Replication state does not survive a reboot

Ops and Admin Replication
,
1

Using Mongo 4.2, implementing a single node replicaset to test change streams. We are starting Mongo via a config file.

net:
  bindIp: 0.0.0.0
  port: 27017
processManagement:
  fork: true
  pidFilePath: /var/run/mongodb/mongod.pid
  timeZoneInfo: /usr/share/zoneinfo
replication:
  replSetName: rs0
storage:
  dbPath: /data/mongo
  journal:
    enabled: true
systemLog:
  destination: file
  logAppend: true
  logRotate: reopen
  path: /var/log/mongodb/mongod.log

After enabling replication in the config file and initiating, everything works fine.
But after a reboot, the replication is not enabled. Mongo clearly recognizes that replication needs to be started, but the previous configuration is not applied.

2024-02-29T16:43:20.978+0000 I  CONTROL  [LogicalSessionCacheReap] Failed to reap transaction table: NotYetInitialized: Replication has not yet been configured
2024-02-29T16:43:21.022+0000 W  REPL     [replexec-0] Locally stored replica set configuration does not have a valid entry for the current node; waiting for reconfig or remote heartbeat; Got "NodeNotFound: No host described in new configuration 14556796 for replica set rs0 maps to this node" while validating { _id: "rs0", version: 14556796, protocolVersion: 1, writeConcernMajorityJournalDefault: true, members: [ { _id: 0, host: "server001:27017", arbiterOnly: false, buildIndexes: true, hidden: false, priority: 3.0, tags: {}, slaveDelay: 0, votes: 1 } ], settings: { chainingAllowed: true, heartbeatIntervalMillis: 2000, heartbeatTimeoutSecs: 10, electionTimeoutMillis: 10000, catchUpTimeoutMillis: -1, catchUpTakeoverDelayMillis: 30000, getLastErrorModes: {}, getLastErrorDefaults: { w: 1, wtimeout: 0 }, replicaSetId: ObjectId('65de6426618df0f1646e119f') } }
2024-02-29T16:43:21.022+0000 I  REPL     [replexec-0] New replica set config in use: { _id: "rs0", version: 14556796, protocolVersion: 1, writeConcernMajorityJournalDefault: true, members: [ { _id: 0, host: "server001:27017", arbiterOnly: false, buildIndexes: true, hidden: false, priority: 3.0, tags: {}, slaveDelay: 0, votes: 1 } ], settings: { chainingAllowed: true, heartbeatIntervalMillis: 2000, heartbeatTimeoutSecs: 10, electionTimeoutMillis: 10000, catchUpTimeoutMillis: -1, catchUpTakeoverDelayMillis: 30000, getLastErrorModes: {}, getLastErrorDefaults: { w: 1, wtimeout: 0 }, replicaSetId: ObjectId('65de6426618df0f1646e119f') } }
2024-02-29T16:43:21.022+0000 I  REPL     [replexec-0] This node is not a member of the config

rs.status returns:

{
        "ok" : 0,
        "errmsg" : "Our replica set config is invalid or we are not a member of it",
        "code" : 93,
        "codeName" : "InvalidReplicaSetConfig"
}

If i run rs.reconfig(rs.config(),{force:true}) and restart the mongod service , the previous config is applied, the node is recognized, and everything works again. From what I’ve read here in the community, replication should be stateful so Im confused about why these steps are necessary after restarting the server

2

Hi @Al_Ferone ,

I’ve often seen this in relation to running a replicaset in docker. This could also happen if mongod cannot resolve hostname in the replicaset config.

What OS and version is being used?
How did you install mongodb?
Is it possible that mongod is starting before the network is fully configured?

MongoDB 4.2 was end of life April 2023 I recommended to upgrade to a current version of MongoDB( 5.0 or higher).

3

Thank you, Chris! It was the network. The service file contained “After=network.target” out of the box. After replacing that with:
After=network-online.target
Wants=network-online.target

everything is fine after a reboot

1 Like
4

This topic was automatically closed 5 days after the last reply. New replies are no longer allowed.