Below is an example of how to connect with the MongoDB Shell. Substitute the following configuration settings (MongoDB Atlas cluster name, database name, and username) in the mongosh command line window. For example:
Note: When using the MongoDB Shell like above, you will be prompted to type the password that you submitted when the MongoDB deployment was created.
Creating a cluster in MongoDB
Whether your initial intentions are about developing proof of concept applications or planning for a production environment, a very good starting point is to create a new MongoDB cluster on MongoDB Atlas. By using the free tier, the default setup is to deploy a MongoDB cluster with a replica set. However, if you also want to enable sharding, then you have to enable this feature separately and specify the number of shards to have.
MongoDB replica sets
What is a MongoDB replica set?
A MongoDB replica set ensures replication is enforced by providing data redundancy and high availability over more than one MongoDB server.
If a MongoDB deployment lacked a replica set, that means all data would be present in exactly one server. In case that main server fails, then all data would be lost - but not when a replica set is enabled. Therefore, we can immediately see the importance of having a replica set for a MongoDB deployment.
In addition to fault tolerance, replica sets can also provide extra read operations capacity for the MongoDB cluster, directing clients to the additional servers, subsequently reducing the overall load of the cluster.
Replica sets can also be beneficial for distributed applications due to the data locality they offer, so that faster and parallel read operations happen to the replica sets instead of having to go through one main server.
How do MongoDB replica sets work?
A MongoDB replica set consists of a primary node and multiple secondary nodes working together to ensure data availability and durability.
Primary Node Operations
The primary node handles all write operations and maintains an operation log (oplog) - a special capped collection that records all data modifications. There can only be one primary node active at any time in a replica set.
Secondary Node Operations
Secondary nodes continuously replicate the primary's oplog and apply these operations to maintain identical copies of the data. This asynchronous replication mechanism ensures the replica set can continue functioning even if some members fail.
Election Process
When the primary node becomes unavailable, the replica set initiates an election to select a new primary from among the secondary nodes. Elections can be triggered by various events:
- Primary node failure
- Network connectivity issues lasting longer than the timeout period (default: 10 seconds)
- Addition of new nodes to the replica set
- Initial replica set configuration
- Planned maintenance operations
Read and write operations in a replica set
Client Applications and Replica Sets
MongoDB's replica set implementation is transparent to client applications - they interact with the cluster the same way regardless of whether it's running as a single server or with replication enabled.
Read Preferences
While MongoDB directs all write operations to the primary node, it offers flexible read operation routing through read preferences. By default, read operations target the primary node, but applications can optionally specify different read preferences to direct queries to secondary nodes. This capability allows applications to:
- Balance read workloads across the replica set
- Reduce latency in multi-region deployments
- Support different consistency requirements for different types of queries
Several read preference modes can be configured. For example, if a client application is configured to go directly to secondaries, then the mode parameter in the read preference should be set to secondary. If there are specific needs for the least network latency irrespective of whether that happens in the primary or any secondary node, then the nearest read preference mode should be configured. However, in this option, a risk of potentially stale data comes into play (if the nearest node is a secondary node) due to the nature of asynchronous replication from primary to secondaries.
Alternatively, the read preference mode can be set to primary preferred or secondary preferred. These two modes also make use of another property called maxStalenessSeconds to determine to which node of the replica set should the read operation be directed. In all cases where there is a chance for the read operation to happen on a non-primary node, you must ensure that your application can tolerate stale data.
When writing data in a MongoDB replica set, you can include additional options to ensure that the write has propagated successfully throughout the cluster. This involves adding a write concern property alongside an insert operation. A write concern means what level of acknowledgement we desire to have from the cluster upon each write operation, and it consists of the following options:
The w value can be set to 0, meaning that no write acknowledgement is needed. 1 is the default and it means that it only requires the primary node to acknowledge the write, while any number greater than 1 corresponds to the number of nodes plus the primary that need to acknowledge. For example, 4 means 3 secondaries need to signal as well as the primary node.
The j value corresponds to whether MongoDB has been written on disk in a special area called the journal. This is used from MongoDB for recovery purposes in case of a hard shutdown, and it is enabled by default.
Finally the wtimeout value is the time the command should wait before returning any result. If this is not specified and if for any reason the actual write has any network issues, then the command would block indefinitely, so it is a good practice to set this value. It is measured in milliseconds and it is only applicable for w values greater than 1.
In the following example, if we had a 5-node replica set, we are requesting that the majority (w option) of the nodes (3) replies back with a successful write acknowledgment:
In the above example, we are using the mongo shell insert command with two parameters. The first parameter is the document that will be inserted into the products collection. The second parameter is the write concern, which specifies that the write operation must be acknowledged by a majority of the data-bearing nodes within a five-second timeout period.
MongoDB sharding
What is a sharded cluster?
A sharded cluster in MongoDB distributes large datasets across multiple shards (servers) to achieve horizontal scalability and improved performance for both read and write operations.
Sharding is particularly beneficial for collections with large data volumes and high query rates.
In the above diagram, Collection1 is sharded across multiple servers, while Collection2 remains unsharded. If Collection1 were confined to a single server, that server's CPU could become overwhelmed due to insufficient capacity to handle incoming requests. By distributing Collection1 across multiple shards, the system can better handle the workload by spreading it across more computing resources.
A sharded cluster consists of three essential components:
- Shards: Each shard can be configured as a replica set, providing both high availability and horizontal scalability.
- Query Router (mongos): This component serves as an intermediary between clients and shards. The mongos process routes queries to appropriate shards and handles load balancing across the cluster for all read and write operations.
- Config Servers: These servers provide essential metadata services for the cluster. They store cluster configuration data, chunk location and range information, authentication details including usernames, role-based access control settings, and database and collection permissions.
Production environment
A robust production MongoDB deployment requires careful planning around several key aspects:
High Availability and Disaster Recovery
To ensure business continuity and minimize downtime, deploy MongoDB replica sets across one or multiple regions based on your risk assessment and recovery requirements.
Each replica set consists of a primary mongod process and multiple secondary processes. It is recommended to maintain an odd number of mongod processes to ensure a majority vote can be reached when the primary fails and a new primary needs to be elected.
Scalability
While proper data modeling, collection design, and indexing strategies are foundational, sharding is essential for handling large-scale production workloads. A well-planned sharding strategy enables your MongoDB cluster to scale horizontally as data volumes grow, maintain performance under heavy workloads, and distribute data and queries across multiple servers.
When designing your production environment, it's important to plan your sharding strategy early, even if you don't implement it immediately. This forward-thinking approach helps ensure your application can scale smoothly as your data and traffic grow.
