Rich text fundamentals/clusters

Background Wrapper fundamentals/clusters

Then, substitute the following configuration settings (MongoDB Atlas cluster name, database name, and username) in the mongo shell command line window. For example:

Note: When using the mongo shell like above, you will be prompted to type the password that you submitted when the MongoDB deployment was created.

Whether your initial intentions are about developing proof of concept applications or planning for a production environment, a very good starting point is to create a new MongoDB cluster on MongoDB Atlas. By using the free tier, the default setup is to deploy a MongoDB cluster with a 

,  then you have to enable this feature separately and specify the number of shards to have.

A MongoDB replica set ensures replication is enforced by providing data redundancy and high availability over more than one MongoDB server. 

If a MongoDB deployment lacked a replica set, that means all data would be present in exactly one server. In case that main server fails, then all data would be lost - but not when a replica set is enabled. Therefore, we can immediately see the importance of having a replica set for a MongoDB deployment.

In addition to fault tolerance, replica sets can also provide extra read operations capacity for the MongoDB cluster, directing clients to the additional servers, subsequently reducing the overall load of the cluster. 

Replica sets can also be beneficial for distributed applications due to the data locality they offer, so that faster and parallel read operations happen to the replica sets instead of having to go through one main server.

A MongoDB cluster needs to have a primary node and a set of secondary nodes in order to be considered a replica set. 

At most, one primary node must exist, and its role is to receive all write operations. All changes on the primary node’s data sets are recorded in a special capped collection called the 

The role of the secondary nodes is to then replicate the primary node’s operation log and make sure that the data sets reflect exactly the primary’s data sets. This functionality is illustrated in the following diagram:

The replication from the primary node’s oplog to the secondaries happens asynchronously, which allows the replica set to continue to function despite potential failure of one or more of its members. If the current primary node suddenly becomes unavailable, an election will take place to determine the new primary node. In the scenario of having two secondary nodes, one of the secondary nodes will become the primary node:

If for any reason only one secondary node exists, a special instance called 

 can be added to a replica set which can only participate in replica set elections but does not replicate the oplog from the primary. This means it can’t provide data redundancy and it will always be an arbiter, i.e., it can’t become a primary or a secondary node, whereas a primary can become a secondary node and vice versa.

The failure of a primary node is not the only reason a replica set election can occur. Replica set elections can happen for other reasons in response to a variety of events, such as when a new node is added in the replica set, on the initiation of a replica set, during maintenance of the replica set, or if any secondary node loses connectivity to the primary node for a period more than the configured timeout period (‘10sec’ by default).

Read and write operations in a replica set

A replica set in a MongoDB cluster is transparent to client applications. This means they can’t identify whether the cluster has a replica set enabled or whether it’s running on a single server deployment.

However, MongoDB offers additional read and write operations on top of the standard input and output commands. Client applications can optionally address directly to which replica set node a read operation should execute. By default, all read operations are directed to the primary node, but specific routing to secondaries can also be configured; this is called read preference.

Several read preference modes can be configured. For example, if a client application is configured to go directly to secondaries, then the mode parameter in the read preference should be set to  

. If there are specific needs for the least network latency irrespective of whether that happens in the primary or any secondary node, then the 

 read preference mode should be configured. However, in this option, a risk of potentially stale data comes into play (if the nearest node is a secondary node) due to the nature of 

Alternatively, the read preference mode can be set to  

. These two modes also make use of another property called 

 to determine to which node of the replica set should the read operation be directed. In all cases where there is a chance for the read operation to happen on a non-primary node, you must ensure that your application can tolerate stale data.

When writing data in a MongoDB replica set, you can include additional options to ensure that the write has propagated successfully throughout the cluster. This involves adding a 

 property alongside an insert operation. A write concern means what level of acknowledgement we desire to have from the cluster upon each write operation, and it consists of the following options:

, meaning that no write acknowledgement is needed. 

 is the default and it means that it only requires the primary node to acknowledge the write, while any number greater than 1 corresponds to the number of nodes plus the primary that need to acknowledge. For example, 

 secondaries need to signal as well as the primary node.

 value corresponds to whether MongoDB has been written on disk in a special area called the  

. This is used from MongoDB for recovery purposes in case of a hard shutdown, and it is enabled by default.

 value is the time the command should wait before returning any result. If this is not specified and if for any reason the actual write has any network issues, then the command would block indefinitely, so it is a good practice to set this value. It is measured in milliseconds and it is only applicable for 

In the following example, if we had a 5-node replica set, we are requesting that the 

option) of the nodes (3) replies back with a successful write acknowledgment:

 command. The first parameter is the document that will be inserted in the 

 collection, while the second parameter is the write concern. The write concern requests for the MongoDB cluster to acknowledge that the write has succeeded in the majority of data bearing nodes, and overall, this operation should not take more than 5 seconds.

A sharded cluster in MongoDB is a collection of datasets distributed across many shards (servers) in order to achieve horizontal scalability and better performance in read and write operations.

Sharding is very useful for collections that have a lot of data and high query rates. 

Based on the above diagram, Collection1 is sharded, unlike Collection2, which is not sharded. If we kept Collection1 in one server, the CPU of that server could spike because it wouldn't have the necessary capacity to deal with the requests it received. However, adding another shard for Collection1 helps distribute the load by increasing the overall capacity that it can receive.

 allows a shard to also be configured as a replica set. In other words, it is configured to be with high availability and horizontal scalability. In addition, a component called 

 is deployed when configuring a sharded cluster. 

 is a query router; it acts as an intermediary between the client and the server, or the shard that the data resides in. Apart from routing, it can also handle load balancing for all query and write operations to the shards.

 is also deployed. It contains any information and configurations relating to the sharded cluster. This component also stores the location and the ranges of the data chunks in the different shards, as well as authentication information including usernames, roles, and permissions to different databases and collections.

Planning for disaster recovery and zero downtime means having a replica set of the MongoDB cluster across a single region or a number of regions, depending on how crucial the assessment of the risk is. 

 process and other secondary processes. It is recommended that the total number of these processes is odd so that majority is ensured in the case that the master fails and a new master needs to be allocated.

Sharding a MongoDB cluster is also at the cornerstone of deploying a production cluster with huge data loads. 

Obviously, designing your data models, appropriately storing them in collections, and defining corrected indexes is essential. But if you truly want to leverage the power of MongoDB, you need to have a plan regarding sharding your cluster.

Rich text2 fundamentals/clusters

Rich text3 fundamentals/clusters

FAQs

FAQs section overview

In MongoDB, a cluster refers to a group of servers or nodes that work together to store and manage data, while a collection is a grouping of documents within a database, similar to a table in a relational database.

You can have multiple clusters, each consisting of multiple nodes, enabling horizontal scaling and high availability.

A MongoDB cluster is a set of connected MongoDB instances (nodes) that work together to provide redundancy, high availability, and scalability for handling large volumes of data.

MongoDB Clusters

FAQs

What is the difference between a cluster and a collection in MongoDB?

How many clusters can you have in MongoDB?

What is a MongoDB cluster?

Ready to get started?