Consume change streams from secondary nodes

Ops and Admin Replication
,
1

Hi, reading https://www.mongodb.com/docs/manual/changeStreams/#open-a-change-stream, it mentions:

For a replica set, you can issue the open change stream operation from any of the data-bearing members.

which seems to suggest that we can consume change stream from a replica (secondary node). Some questions around that:

  • Is there any config to only allow read change stream from secondary? If so, how do we handle a secondary being promoted to primary or a secondary node get terminated?
  • Is there a config to only allow read change stream from primary? Or is it actually the default behavior?

Any related example or documentation would also be helpful! Thanks

2

I believe the default behavior is: “allowing read change stream from any node within a replset”, and there is no special config to enforce read only from primary or secondary.

If that’s the case, do we have any special events being sent out if a node being promoted / demoted at all? Or for change stream API, the “role” of a node (primary vs secondary) is not being exposed to clients at all?

3

The read preference is at the client driver level. It is not specific to change stream. You specify the read preference when you establish the connection between the client driver and the server. The change stream will use what ever read preference the connection has.

To use other than the default read preference see:

4

Thanks @steevej , some follow ups:

  • With secondary read preference, will Mongo automatically switch node if the previous secondary node gets promoted to primary?
  • If we create a MongoClient by connecting to a primary node, and start a connection with secondary read preference, would mongo reroute the connection to another node in the replset automatically?
5

Some more context: when we build connection, we pass in a list of server addresses: debezium/MongoClients.java at main · debezium/debezium · GitHub

One of them is primary and I wonder what would happen if we set readPreference to secondary while we are building a connection with the primary node?

6

Related ^

After the connection is built and there is a failover (e.g., promote a secondary to primary), would the change stream be re-created automatically to another secondary node?

7

I have no idea.

Hopefully, someone will jump in the thread.

8

Hi guys. I have the same situation. When Primary and secondary change, streaming does not output any changes, and only work when change back.

Does anyone found how to solve it?

9

Hi @Mykhailo_Kuznietsov welcome to the community!

I did a simple, straightforward test for this. Basically I created a 3-node replica set, and connected a simple Python script to them:

uri = "mongodb://localhost:27017,localhost:27018,localhost:27019/test?replicaSet=replset&readPreference=secondary"
conn = pymongo.MongoClient(uri)
db = conn.test
cursor = db.test.watch()
while 1:
    print(next(cursor))

Note the option readPreference=secondary in the connection string above.

I then inserted a couple of documents to the collection to make sure the script works.

Then I killed the primary, wait for the new primary to be elected, then inserted more documents.

The script stays running and outputting the newly inserted documents as expected.

If this is not working as above for you, could you post:

  • The MongoDB version you used
  • The connection string you used
  • The script you used to open the changestream
  • The scenario you tested

Best regards
Kevin

2 Likes