rs.reconfig()

term Replica Configuration Field

MongoDB 4.4 adds the term field to the replica set configuration document. The term field is set by the primary replica set member. The primary ignores the term field if set explicitly in the rs.reconfig() operation.

Reconfiguration Can Add or Remove No More than One Voting Member at a Time

Starting in MongoDB 4.4, rs.reconfig() by default allows adding or removing no more than 1 voting member at a time. For example, a new configuration can make at most one of the following changes to the cluster membership:

• Adding a new voting replica set member.

• Removing an existing voting replica set member.

• Modifying the votes for an existing replica set member.

To add or remove multiple voting members, issue a series of rs.reconfig() operations to add or remove one member at a time.

Issuing a force reconfiguration immediately installs the new configuration even if it adds or removes multiple voting members. Force reconfiguration can cause unexpected behavior, such as the rollback of "majority" committed write operations.

Reconfiguration Waits Until a Majority of Members Install the Replica Configuration

Starting in MongoDB 4.4, rs.reconfig() waits until a majority of voting replica set members install the new replica configuration before returning success. A voting member is any replica set member where members[n].votes is 1, including arbiters.

Replica set members propagate their replica configuration via heartbeats. Whenever a member learns of a configuration with a higher version and term, it installs the new configuration. The reconfiguration process has two distinct 'waiting' phases:

1) Wait for the current configuration to be committed before installing the new configuration.

The "current" configuration refers to the replica configuration in use by the primary at the time rs.reconfig() is issued.

A configuration is committed when:

• A majority of voting replica set members have installed the current configuration, and

• All writes which were "majority" committed in the previous configuration have also replicated to a majority in the current configuration.

Typically, the current configuration has already been installed on a majority of voting replica set members. However, the majority committed writes in the previous configuration may not all be committed in the current configuration. Delayed members or members that are lagging behind the primary can increase the time spent in this phase.

If the operation was issued with a maxTimeMS limit and the operation exceeds the limit while waiting, the operation returns an error and discard the new configuration. The limit is cumulative and does not reset after proceeding to the next phase.

2) Wait for a majority of voting members in the new configuration to install the new configuration.

The "new" configuration refers to the replica configuration specified to rs.reconfig().

The primary installs and begins using the new replica configuration before propagating the configuration to the remaining replica set members. The operation only waits for a majority of voting members to install the new configuration, and does not require waiting for the new configuration to be committed.

If the operation was issued with a maxTimeMS limit and the operation exceeds the limit while waiting, the operation returns an error but continues using and propagating the new configuration.

Issuing a force reconfiguration immediately installs the new configuration regardless of the commitment status of the previous configuration. Force reconfiguration can cause unexpected behavior, such as the rollback of "majority" committed write operations.

To check the commitment status of the current replica configuration, issue replSetGetConfig with the commitmentStatus parameter on the replica set primary.

Access Control

To run the method on deployments that enforce access control, the user must have replSetConfigure privilege action on the cluster resource. The clusterManager built-in role, available in the admin database, provides the required privileges for this command.

Locking Behavior

rs.reconfig() obtains a special mutually exclusive lock to prevent more than one rs.reconfig() operation from occurring at the same time.

Mixed Version Replica Set

Availability

The rs.reconfig() shell method can trigger the current primary to step down in some situations. Primary step-down triggers an election to select a new primary:

Starting in MongoDB 4.2, when the primary steps down, it no longer closes all client connections and writes that were in progress are killed. For details, see Behavior.

The median time before a cluster elects a new primary should not typically exceed 12 seconds, assuming default replica configuration settings. This includes time required to mark the primary as unavailable and call and complete an election. You can tune this time period by modifying the settings.electionTimeoutMillis replication configuration option. Factors such as network latency may extend the time required for replica set elections to complete, which in turn affects the amount of time your cluster may operate without a primary. These factors are dependent on your particular cluster architecture.

During the election process, the cluster cannot accept write operations until it elects the new primary.

Your application connection logic should include tolerance for automatic failovers and the subsequent elections. MongoDB drivers can detect the loss of the primary and automatically retry certain write operations a single time, providing additional built-in handling of automatic failovers and elections:

Compatible drivers enable retryable writes by default

To further reduce potential impact to a production cluster, reconfigure only during scheduled maintenance periods.

{ force: true }

Member Priority and Votes

• Members with priority greater than 0 cannot have 0 votes.

• Non-voting (i.e. votes is 0) members must have priority of 0.

Drop Outgoing Connections After Removing a Member

Using rs.reconfig() to remove a replica set member does not automatically drop open outgoing connections from other replica set members to the removed member.

By default, replica set members wait for 5 minutes before dropping connections to the removed member. In sharded replica sets, you can modify this timeout using the ShardingTaskExecutorPoolHostTimeoutMS server parameter.

db.adminCommand(
  {
    "dropConnections" : 1,
    "hostAndPort" : [
      "<hostname>:<port>"
    ]
  }
)

Replace <hostname> and <port> with those of the removed member.

Change Replica Set Member Priority

The following sequence of operations updates the members[n].priority of the second member. The operations are issued through a mongo shell connected to the primary.

cfg = rs.conf();
cfg.members[1].priority = 2;
rs.reconfig(cfg);

1. The first statement uses the rs.conf() method to retrieve a document containing the current configuration for the replica set and sets the document to the local variable cfg.

2. The second statement sets a members[n].priority value to the second document in the members array. For additional settings, see replica set configuration settings.

   To access the member configuration document in the array, the statement uses the array index and not the replica set member's members[n]._id field.

3. The last statement calls the rs.reconfig() method with the modified cfg to initialize this new configuration. Upon successful reconfiguration, the replica set configuration will resemble the following:

{
   "_id" : "rs0",
   "version" : 2,
   "protocolVersion" : NumberLong(1),
   "members" : [
      {
         "_id" : 0,
         "host" : "mongodb0.example.net:27017",
         "arbiterOnly" : false,
         "buildIndexes" : true,
         "hidden" : false,
         "priority" : 1,
         "tags" : {
         },
         "slaveDelay" : NumberLong(0),
         "votes" : 1
      },
      {
         "_id" : 1,
         "host" : "mongodb1.example.net:27017",
         "arbiterOnly" : false,
         "buildIndexes" : true,
         "hidden" : false,
         "priority" : 2,
         "tags" : {
         },
         "slaveDelay" : NumberLong(0),
         "votes" : 1
      },
      {
         "_id" : 2,
         "host" : "mongodb2.example.net:27017",
         "arbiterOnly" : false,
         "buildIndexes" : true,
         "hidden" : false,
         "priority" : 1,
         "tags" : {
         },
         "slaveDelay" : NumberLong(0),
         "votes" : 1
      }
   ],
   "settings" : {
      "chainingAllowed" : true,
      "heartbeatIntervalMillis" : 2000,
      "heartbeatTimeoutSecs" : 10,
      "electionTimeoutMillis" : 10000,
      "catchUpTimeoutMillis" : 2000,
      "getLastErrorModes" : {
      },
      "getLastErrorDefaults" : {
         "w" : 1,
         "wtimeout" : 0
      },
      "replicaSetId" : ObjectId("58858acc1f5609ed986b641b")
   }
}

Change Replica Set Settings

You can also modify the cluster replica set settings document. The settings document contains configuration options that apply to the whole replica set.

The following sequence of operations updates the settings.heartbeatTimeoutSecs of the cluster to 15. The operations are issued through a mongo shell session that is connected to the primary.

cfg = rs.conf();
cfg.settings.heartbeatTimeoutSecs = 15;
rs.reconfig(cfg);