Build a Resilient Application with MongoDB
On this page
To write application code that takes advantage of the capabilities of MongoDB and gracefully handles replica set elections, you should:
Install the latest drivers.
Use a connection string that specifies all hosts.
Use retryable writes and retryable reads.
Use a
majority
write concern and a read concern that makes sense for your application.Handle errors in your application.
Install the Latest Drivers
First, install the latest drivers for your language from MongoDB Drivers. Drivers connect and relay queries from your application to your database. Using the latest drivers enables the latest MongoDB features.
Then, in your application, import the dependency:
If you are using Maven, add the
following to your pom.xml
dependencies list:
<dependencies> <dependency> <groupId>org.mongodb</groupId> <artifactId>mongodb-driver-sync</artifactId> <version>4.0.1</version> </dependency> </dependencies>
If you are using Gradle, add the
following to your build.gradle
dependencies list:
dependencies { compile 'org.mongodb:mongodb-driver-sync:4.0.1' }
// Latest 'mongodb' version installed with npm const MongoClient = require('mongodb').MongoClient;
Connection Strings
Use a connection string that specifies all the hosts in your deployment to connect your application to your database. If your deployment performs a replica set election and a new primary is elected, a connection string that specifies all hosts in your deployment discovers the new primary without application logic.
You can specify all the hosts in your deployment using either:
The connection string can also specify options, notably retryWrites and writeConcern.
Tip
See also:
For help formatting your connection string, see Connect to a Deployment Using a MongoDB Driver.
Use your connection string to instantiate a MongoDB client in your application:
// Create a variable for your connection string String uri = "mongodb://[<username>:<password>@]hostname0<:port>[,hostname1:<port1>][,hostname2:<port2>][...][,hostnameN:<portN>]"; // Instantiate the MongoDB client with the URI MongoClient client = MongoClients.create(uri);
// Create a variable for your connection string const uri = "mongodb://[<username>:<password>@]hostname0<:port>[,hostname1:<port1>][,hostname2:<port2>][...][,hostnameN:<portN>]"; // Instantiate the MongoDB client with the URI const client = new MongoClient(uri, { useNewUrlParser: true, useUnifiedTopology: true });
Retryable Writes and Reads
Note
Starting in MongoDB version 3.6 and with 4.2-compatible drivers, MongoDB retries both writes and reads once by default.
Retryable Writes
Use retryable writes to retry certain write operations a single time if they fail.
Retrying writes exactly once is the best strategy for handling transient network errors and replica set elections in which the application temporarily can't find a healthy primary node. If the retry succeeds, the operation as a whole succeeds and no error is returned. If the operation fails, it is likely due to:
A lasting network error, or
An invalid command.
When an operation fails, your application needs to handle the error itself.
Retryable Reads
Read operations are automatically retried a single time if they fail starting in MongoDB version 3.6 and with 4.2-compatible drivers. You don't need to configure your application to retry reads.
Write and Read Concern
You can tune the consistency and availability of your application using write concerns and read concerns. Stricter concerns imply that database operations wait for stronger data consistency guarantees, whereas loosening consistency requirements provides higher availability.
Example
If your application handles monetary balances, consistency is
extremely important. You might use majority
write and read
concerns to ensure you never read from stale data or data that might
be rolled back.
Alternatively, if your application records temperature data from hundreds of sensors every second, you may not be concerned if you read data that does not include the most recent readouts. You can loosen consistency requirements to provide faster access to that data.
Write Concern
You can set the
write concern level
of your replica set through the connection string URI. Use a
majority
write concern to ensure your data is successfully written
to your database and persisted. This is the recommended default and
sufficient for most use cases.
When you use a write concern that requires acknowledgement, such as
majority
, you may also specify a maximum time limit for writes
to achieve that level of acknowledgement:
The wtimeoutMS connection string parameter for all writes, or
The wtimeout option for a single write operation.
Whether or not you use a time limit and the value you use depends on your application context.
Important
If you do not specify a time limit for writes and the level of write concern is unachievable, the write operation will never complete.
Read Concern
You can set the read concern level of your replica set through the connection string URI. The ideal read concern depends on your application requirements, but the default is sufficient for most use cases. No connection string parameter is required to use default read concerns.
Specifying a read concern can improve guarantees around the data your application receives from your database.
Note
The specific combination of write and read concern your application uses affects order-of-operation guarantees. This is called causal consistency. For more information on causal consistency guarantees, see Causal Consistency and Read and Write Concerns.
Error Handling
Invalid commands, network outages, and network errors that are not handled by retryable writes return errors. Refer to your driver's API documentation for error details.
For example, if an application tries to insert a document with a
duplicate _id
, your driver returns an error that includes:
Unable to insert due to an error: com.mongodb.MongoWriteException: E11000 duplicate key error collection: <db>.<collection> ...
{ "name": : "MongoError", "message": "E11000 duplicate key error collection on: <db>.<collection> ... ", ... }
Without proper error handling, an error might block your application from processing requests until it is restarted.
Your application should handle errors without crashing or side
effects. In the previous example of an application inserting a
duplicate _id
, that application could handle errors as follows:
// Declare a logger instance from java.util.logging.Logger private static final Logger LOGGER = ... ... try { InsertOneResult result = collection.insertOne(new Document() .append("_id", 1) .append("body", "I'm a goofball trying to insert a duplicate _id")); // Everything is OK LOGGER.info("Inserted document id: " + result.getInsertedId()); // Refer to the API documentation for specific exceptions to catch } catch (MongoException me) { // Report the error LOGGER.severe("Failed due to an error: " + me); }
... collection.insertOne({ _id: 1, body: "I'm a goofball trying to insert a duplicate _id" }) .then(result => { response.sendStatus(200) // send "OK" message to the client }, err => { response.sendStatus(400); // send "Bad Request" message to the client });
The insert operation in this example throws a "duplicate key"
error the second time it's invoked because the _id
field must be
unique. The application catches the error, the client is notified, and the app
continues to run. The insert operation fails, however, and it is
up to you to decide whether to show the user a message, retry the
operation, or do something else.
You should always log errors. Common strategies for further processing errors include:
Return the error to the client with an error message. This is a good strategy when you cannot resolve the error and need to inform a user that an action can't be completed.
Write to a backup database. This is a good strategy when you can't resolve the error but don't want to risk losing the request data.
Retry the operation beyond the single default retry. This is a good strategy when you can solve the cause of an error programmatically, then retry it.
You must select the best strategies for your application context.
Example
In the example of a duplicate key error, you should log the error but not retry the operation because it will never succeed. Instead, you could write to a fallback database and review the contents of that database at a later time to ensure that no information is lost. The user doesn't need to do anything else and the data is recorded, so you can choose not to send an error message to the client.
Planning for Network Errors
Returning an error can be desirable behavior when an operation would otherwise never complete and block your application from executing new operations. You can use the maxTimeMS method to place a time limit on individual operations, returning an error for your application to handle if that time limit is exceeded.
The time limit you place on each operation depends on the context of that operation.
Example
If your application reads and displays simple product information
from an inventory
collection, you can be reasonably confident
that those read operations only take a moment. An unusually
long-running query is a good indicator that there is a lasting
network problem. Setting maxTimeMS
on that operation to 5000, or
5 seconds, means that your application receives feedback as soon as
you are confident there is a network problem.
Resilient Example Application
The following example application brings together the recommendations for building resilient applications.
The application is a simple user records API that exposes two endpoints on http://localhost:3000:
Method | Endpoint | Description |
---|---|---|
GET | /users | Gets a list of user names from a users collection. |
POST | /users | Requires a name in the request body. Adds a new user to a
users collection. |
Note
1 // File: App.java 2 3 import java.util.Map; 4 import java.util.logging.Logger; 5 6 import org.bson.Document; 7 import org.json.JSONArray; 8 9 import com.mongodb.MongoException; 10 import com.mongodb.client.MongoClient; 11 import com.mongodb.client.MongoClients; 12 import com.mongodb.client.MongoCollection; 13 import com.mongodb.client.MongoDatabase; 14 15 import fi.iki.elonen.NanoHTTPD; 16 17 public class App extends NanoHTTPD { 18 private static final Logger LOGGER = Logger.getLogger(App.class.getName()); 19 20 static int port = 3000; 21 static MongoClient client = null; 22 23 public App() throws Exception { 24 super(port); 25 26 // Replace the uri string with your MongoDB deployment's connection string 27 String uri = "mongodb://<username>:<password>@hostname0:27017,hostname1:27017,hostname2:27017/?retryWrites=true&w=majority"; 28 client = MongoClients.create(uri); 29 30 start(NanoHTTPD.SOCKET_READ_TIMEOUT, false); 31 LOGGER.info("\nStarted the server: http://localhost:" + port + "/ \n"); 32 } 33 34 public static void main(String[] args) { 35 try { 36 new App(); 37 } catch (Exception e) { 38 LOGGER.severe("Couldn't start server:\n" + e); 39 } 40 } 41 42 43 public Response serve(IHTTPSession session) { 44 StringBuilder msg = new StringBuilder(); 45 Map<String, String> params = session.getParms(); 46 47 Method reqMethod = session.getMethod(); 48 String uri = session.getUri(); 49 50 if (Method.GET == reqMethod) { 51 if (uri.equals("/")) { 52 msg.append("Welcome to my API!"); 53 } else if (uri.equals("/users")) { 54 msg.append(listUsers(client)); 55 } else { 56 msg.append("Unrecognized URI: ").append(uri); 57 } 58 } else if (Method.POST == reqMethod) { 59 try { 60 String name = params.get("name"); 61 if (name == null) { 62 throw new Exception("Unable to process POST request: 'name' parameter required"); 63 } else { 64 insertUser(client, name); 65 msg.append("User successfully added!"); 66 } 67 } catch (Exception e) { 68 msg.append(e); 69 } 70 } 71 72 return newFixedLengthResponse(msg.toString()); 73 } 74 75 static String listUsers(MongoClient client) { 76 MongoDatabase database = client.getDatabase("test"); 77 MongoCollection<Document> collection = database.getCollection("users"); 78 79 final JSONArray jsonResults = new JSONArray(); 80 collection.find().forEach((result) -> jsonResults.put(result.toJson())); 81 82 return jsonResults.toString(); 83 } 84 85 static String insertUser(MongoClient client, String name) throws MongoException { 86 MongoDatabase database = client.getDatabase("test"); 87 MongoCollection<Document> collection = database.getCollection("users"); 88 89 collection.insertOne(new Document().append("name", name)); 90 return "Successfully inserted user: " + name; 91 } 92 }
Note
The following server application uses Express, which you need to add to your project as a dependency before you can run it.
1 const express = require('express'); 2 const bodyParser = require('body-parser'); 3 4 // Use the latest drivers by installing & importing them 5 const MongoClient = require('mongodb').MongoClient; 6 7 const app = express(); 8 app.use(bodyParser.json()); 9 app.use(bodyParser.urlencoded({ extended: true })); 10 11 // Use a connection string that lists all hosts 12 // with retryable writes & majority write concern 13 const uri = "mongodb://<username>:<password>@hostname0:27017,hostname1:27017,hostname2:27017/?retryWrites=true&w=majority"; 14 15 const client = new MongoClient(uri, { 16 useNewUrlParser: true, 17 useUnifiedTopology: true 18 }); 19 20 // ----- API routes ----- // 21 app.get('/', (req, res) => res.send('Welcome to my API!')); 22 23 app.get('/users', (req, res) => { 24 const collection = client.db("test").collection("users"); 25 26 collection 27 .find({}) 28 // In this example, 'maxTimeMS' throws an error after 5 seconds, 29 // alerting the application to a lasting network outage 30 .maxTimeMS(5000) 31 .toArray((err, data) => { 32 if (err) { 33 // Handle errors in your application 34 // In this example, by sending the client a message 35 res.send("The request has timed out. Please check your connection and try again."); 36 } 37 return res.json(data); 38 }); 39 }); 40 41 app.post('/users', (req, res) => { 42 const collection = client.db("test").collection("users"); 43 collection.insertOne({ name: req.body.name }) 44 .then(result => { 45 res.send("User successfully added!"); 46 }, err => { 47 // Handle errors in your application 48 // In this example, by sending the client a message 49 res.send("An application error has occurred. Please try again."); 50 }) 51 }); 52 // ----- End of API routes ----- // 53 54 app.listen(3000, () => { 55 console.log(`Listening on port 3000.`); 56 client.connect(err => { 57 if (err) { 58 console.log("Not connected: ", err); 59 process.exit(0); 60 } 61 console.log('Connected.'); 62 }); 63 });