Local Development with the MongoDB Atlas CLI and Docker
Rate this tutorial
Need a consistent development and deployment experience as developers work across teams and use different machines for their daily tasks? That is where Docker has you covered with containers. A common experience might include running a local version of MongoDB Community in a container and an application in another container. This strategy works for some organizations, but what if you want to leverage all the benefits that come with MongoDB Atlas in addition to a container strategy for your application development?
In this tutorial we'll see how to create a MongoDB-compatible web application, bundle it into a container with Docker, and manage creation as well as destruction for MongoDB Atlas with the Atlas CLI during container deployment.
It should be noted that this tutorial was intended for a development or staging setting through your local computer. It is not advised to use all the techniques found in this tutorial in a production setting. Use your best judgment when it comes to the code included.
If you’d like to try the results of this tutorial, check out the repository and instructions on GitHub.
There are a lot of moving parts in this tutorial, so you'll need a few things prior to be successful:
- Some familiarity with Node.js and JavaScript
The Atlas CLI can create an Atlas account for you along with any keys and ids, but for the scope of this tutorial you'll need one created along with quick access to the "Public API Key", "Private API Key", "Organization ID", and "Project ID" within your account. You can see how to do this in the documentation.
Docker is going to be the true star of this tutorial. You don't need anything beyond Docker because the Node.js application and the Atlas CLI will be managed by the Docker container, not your host computer.
On your host computer, create a project directory. The name isn't important, but for this tutorial we'll use mongodbexample as the project directory.
We're going to start by creating a Node.js application that communicates with MongoDB using the Node.js driver for MongoDB. The application will be simple in terms of functionality. It will connect to MongoDB, create a database and collection, insert a document, and expose an API endpoint to show the document with an HTTP request.
Within the project directory, create a new app directory for the Node.js application to live. Within the app directory, using a command line, execute the following:
1 npm init -y 2 npm install express mongodb
If you don't have Node.js installed, just create a package.json file within the app directory with the following contents:
1 { 2 "name": "mongodbexample", 3 "version": "1.0.0", 4 "description": "", 5 "main": "main.js", 6 "scripts": { 7 "test": "echo \"Error: no test specified\" && exit 1", 8 "start": "node main.js" 9 }, 10 "keywords": [], 11 "author": "", 12 "license": "ISC", 13 "dependencies": { 14 "express": "^4.18.2", 15 "mongodb": "^4.12.1" 16 } 17 }
Next, we'll need to define our application logic. Within the app directory we need to create a main.js file. Within the main.js file, add the following JavaScript code:
1 const { MongoClient } = require("mongodb"); 2 const Express = require("express"); 3 4 const app = Express(); 5 6 const mongoClient = new MongoClient(process.env.MONGODB_ATLAS_URI); 7 let database, collection; 8 9 app.get("/data", async (request, response) => { 10 try { 11 const results = await collection.find({}).limit(5).toArray(); 12 response.send(results); 13 } catch (error) { 14 response.status(500).send({ "message": error.message }); 15 } 16 }); 17 18 const server = app.listen(3000, async () => { 19 try { 20 await mongoClient.connect(); 21 database = mongoClient.db(process.env.MONGODB_DATABASE); 22 collection = database.collection(`${process.env.MONGODB_COLLECTION}`); 23 collection.insertOne({ "firstname": "Nic", "lastname": "Raboy" }); 24 console.log("Listening at :3000"); 25 } catch (error) { 26 console.error(error); 27 } 28 }); 29 30 process.on("SIGTERM", async () => { 31 if(process.env.CLEANUP_ONDESTROY == "true") { 32 await database.dropDatabase(); 33 } 34 mongoClient.close(); 35 server.close(() => { 36 console.log("NODE APPLICATION TERMINATED!"); 37 }); 38 });
There's a lot happening in the few lines of code above. We're going to break it down!
Before we break down the pieces, take note of the environment variables used throughout the JavaScript code. We'll be passing these values through Docker in the end so we have a more dynamic experience with our local development.
The first important snippet of code to focus on is the start of our application service:
1 const server = app.listen(3000, async () => { 2 try { 3 await mongoClient.connect(); 4 database = mongoClient.db(process.env.MONGODB_DATABASE); 5 collection = database.collection(`${process.env.MONGODB_COLLECTION}`); 6 collection.insertOne({ "firstname": "Nic", "lastname": "Raboy" }); 7 console.log("Listening at :3000"); 8 } catch (error) { 9 console.error(error); 10 } 11 });
Using the client that was configured near the top of the file, we can connect to MongoDB. Once connected, we can get a reference to a database and collection. This database and collection doesn't need to exist before that because it will be created automatically when data is inserted. With the reference to a collection, we insert a document and begin listening for API requests through HTTP.
This brings us to our one and only endpoint:
1 app.get("/data", async (request, response) => { 2 try { 3 const results = await collection.find({}).limit(5).toArray(); 4 response.send(results); 5 } catch (error) { 6 response.status(500).send({ "message": error.message }); 7 } 8 });
When the
/data endpoint is consumed, the first five documents in our collection are returned to the user. Otherwise if there was some issue, an error message would be returned.This brings us to something optional, but potentially valuable when it comes to a Docker deployment for local development:
1 process.on("SIGTERM", async () => { 2 if(process.env.CLEANUP_ONDESTROY == "true") { 3 await database.dropDatabase(); 4 } 5 mongoClient.close(); 6 server.close(() => { 7 console.log("NODE APPLICATION TERMINATED!"); 8 }); 9 });
The above code says that when a termination event is sent to the application, drop the database we had created and close the connection to MongoDB as well as the Express Framework service. This could be useful if we want to undo everything we had created when the container stops. If you want your changes to persist, it might not be necessary. For example, if you want your data to exist between container deployments, persistence would be required. On the other hand, maybe you are using the container as part of a test pipeline and you want to clean up when you’re done, the termination commands could be valuable.
So we have an environment variable heavy Node.js application. What's next?
While we have the application, our MongoDB Atlas cluster may not be available to us. For example, maybe this is our first time being exposed to Atlas and nothing has been created yet. We need to be able to quickly and easily create a cluster, configure our IP access rules, specify users and permissions, and then connect with our Node.js application.
This is where the MongoDB Atlas CLI does the heavy lifting!
There are many different ways to create a script. Some like Bash, some like ZSH, some like something else. We're going to be using ZX which is a JavaScript wrapper for Bash.
Within your project directory, not your app directory, create a docker_run_script.mjs file with the following code:
1 #!/usr/bin/env zx 2 3 $.verbose = true; 4 5 const runtimeTimestamp = Date.now(); 6 7 process.env.MONGODB_CLUSTER_NAME = process.env.MONGODB_CLUSTER_NAME || "examples"; 8 process.env.MONGODB_USERNAME = process.env.MONGODB_USERNAME || "demo"; 9 process.env.MONGODB_PASSWORD = process.env.MONGODB_PASSWORD || "password1234"; 10 process.env.MONGODB_DATABASE = process.env.MONGODB_DATABASE || "business_" + runtimeTimestamp; 11 process.env.MONGODB_COLLECTION = process.env.MONGODB_COLLECTION || "people_" + runtimeTimestamp; 12 process.env.CLEANUP_ONDESTROY = process.env.CLEANUP_ONDESTROY || false; 13 14 var app; 15 16 process.on("SIGTERM", () => { 17 app.kill("SIGTERM"); 18 }); 19 20 try { 21 let createClusterResult = await $`atlas clusters create ${process.env.MONGODB_CLUSTER_NAME} --tier M0 --provider AWS --region US_EAST_1 --output json`; 22 await $`atlas clusters watch ${process.env.MONGODB_CLUSTER_NAME}` 23 let loadSampleDataResult = await $`atlas clusters loadSampleData ${process.env.MONGODB_CLUSTER_NAME} --output json`; 24 } catch (error) { 25 console.log(error.stdout); 26 } 27 28 try { 29 let createAccessListResult = await $`atlas accessLists create --currentIp --output json`; 30 let createDatabaseUserResult = await $`atlas dbusers create --role readWriteAnyDatabase,dbAdminAnyDatabase --username ${process.env.MONGODB_USERNAME} --password ${process.env.MONGODB_PASSWORD} --output json`; 31 await $`sleep 10` 32 } catch (error) { 33 console.log(error.stdout); 34 } 35 36 try { 37 let connectionString = await $`atlas clusters connectionStrings describe ${process.env.MONGODB_CLUSTER_NAME} --output json`; 38 let parsedConnectionString = new URL(JSON.parse(connectionString.stdout).standardSrv); 39 parsedConnectionString.username = encodeURIComponent(process.env.MONGODB_USERNAME); 40 parsedConnectionString.password = encodeURIComponent(process.env.MONGODB_PASSWORD); 41 parsedConnectionString.search = "retryWrites=true&w=majority"; 42 process.env.MONGODB_ATLAS_URI = parsedConnectionString.toString(); 43 app = $`node main.js`; 44 } catch (error) { 45 console.log(error.stdout); 46 }
Once again, we're going to break down what's happening!
Like with the Node.js application, the ZX script will be using a lot of environment variables. In the end, these variables will be passed with Docker, but you can hard-code them at any time if you want to test things outside of Docker.
The first important thing to note is the defaulting of environment variables:
1 process.env.MONGODB_CLUSTER_NAME = process.env.MONGODB_CLUSTER_NAME || "examples"; 2 process.env.MONGODB_USERNAME = process.env.MONGODB_USERNAME || "demo"; 3 process.env.MONGODB_PASSWORD = process.env.MONGODB_PASSWORD || "password1234"; 4 process.env.MONGODB_DATABASE = process.env.MONGODB_DATABASE || "business_" + runtimeTimestamp; 5 process.env.MONGODB_COLLECTION = process.env.MONGODB_COLLECTION || "people_" + runtimeTimestamp; 6 process.env.CLEANUP_ONDESTROY = process.env.CLEANUP_ONDESTROY || false;
The above snippet isn't a requirement, but if you want to avoid setting or passing around variables, defaulting them could be helpful. In the above example, the use of
runtimeTimestamp will allow us to create a unique database and collection should we want to. This could be useful if numerous developers plan to use the same Docker images to deploy containers because then each developer would be in a sandboxed area. If the developer chooses to undo the deployment, only their unique database and collection would be dropped.Next we have the following:
1 process.on("SIGTERM", () => { 2 app.kill("SIGTERM"); 3 });
We have something similar in the Node.js application as well. We have it in the script because eventually the script controls the application. So when we (or Docker) stops the script, the same stop event is passed to the application. If we didn't do this, the application would not have a graceful shutdown and the drop logic wouldn't be applied.
Now we have three try / catch blocks, each focusing on something particular.
The first block is responsible for creating a cluster with sample data:
1 try { 2 let createClusterResult = await $`atlas clusters create ${process.env.MONGODB_CLUSTER_NAME} --tier M0 --provider AWS --region US_EAST_1 --output json`; 3 await $`atlas clusters watch ${process.env.MONGODB_CLUSTER_NAME}` 4 let loadSampleDataResult = await $`atlas clusters loadSampleData ${process.env.MONGODB_CLUSTER_NAME} --output json`; 5 } catch (error) { 6 console.log(error.stdout); 7 }
If the cluster already exists, an error will be caught. We have three blocks because in our scenario, it is alright if certain parts already exist.
Next we worry about users and access:
1 try { 2 let createAccessListResult = await $`atlas accessLists create --currentIp --output json`; 3 let createDatabaseUserResult = await $`atlas dbusers create --role readWriteAnyDatabase,dbAdminAnyDatabase --username ${process.env.MONGODB_USERNAME} --password ${process.env.MONGODB_PASSWORD} --output json`; 4 await $`sleep 10` 5 } catch (error) { 6 console.log(error.stdout); 7 }
We want our local IP address to be added to the access list and we want a user to be created. In this example, we are creating a user with extensive access, but you may want to refine the level of permission they have in your own project. For example, maybe the container deployment is meant to be a sandboxed experience. In this scenario, it makes sense that the user created access only the database and collection in the sandbox. We
sleep after these commands because they are not instant and we want to make sure everything is ready before we try to connect.Finally we try to connect:
1 try { 2 let connectionString = await $`atlas clusters connectionStrings describe ${process.env.MONGODB_CLUSTER_NAME} --output json`; 3 let parsedConnectionString = new URL(JSON.parse(connectionString.stdout).standardSrv); 4 parsedConnectionString.username = encodeURIComponent(process.env.MONGODB_USERNAME); 5 parsedConnectionString.password = encodeURIComponent(process.env.MONGODB_PASSWORD); 6 parsedConnectionString.search = "retryWrites=true&w=majority"; 7 process.env.MONGODB_ATLAS_URI = parsedConnectionString.toString(); 8 app = $`node main.js`; 9 } catch (error) { 10 console.log(error.stdout); 11 }
After the first try / catch block finishes, we'll have a connection string. We can finalize our connection string with a Node.js URL object by including the username and password, then we can run our Node.js application. Remember, the environment variables and any manipulations we made to them in our script will be passed into the Node.js application.
At this point, we have an application and we have a script for preparing MongoDB Atlas and launching the application. It's time to get everything into a Docker image to be deployed as a container.
At the root of your project directory, add a Dockerfile file with the following:
1 FROM node:18 2 3 WORKDIR /usr/src/app 4 5 COPY ./app/* ./ 6 COPY ./docker_run_script.mjs ./ 7 8 RUN curl https://fastdl.mongodb.org/mongocli/mongodb-atlas-cli_1.3.0_linux_x86_64.tar.gz --output mongodb-atlas-cli_1.3.0_linux_x86_64.tar.gz 9 RUN tar -xvf mongodb-atlas-cli_1.3.0_linux_x86_64.tar.gz && mv mongodb-atlas-cli_1.3.0_linux_x86_64 atlas_cli 10 RUN chmod +x atlas_cli/bin/atlas 11 RUN mv atlas_cli/bin/atlas /usr/bin/ 12 13 RUN npm install -g zx 14 RUN npm install 15 16 EXPOSE 3000 17 18 CMD ["./docker_run_script.mjs"]
The custom Docker image will be based on a Node.js image which will allow us to run our Node.js application as well as our ZX script.
After our files are copied into the image, we run a few commands to download and extract the MongoDB Atlas CLI.
Finally, we install ZX and our application dependencies and run the ZX script. The
CMD command for running the script is done when the container is run. Everything else is done when the image is built.We could build our image from this Dockerfile file, but it is a lot easier to manage when there is a Compose configuration. Within the project directory, create a docker-compose.yml file with the following YAML:
1 version: "3.9" 2 services: 3 web: 4 build: 5 context: . 6 dockerfile: Dockerfile 7 ports: 8 - "3000:3000" 9 environment: 10 MONGODB_ATLAS_PUBLIC_API_KEY: YOUR_PUBLIC_KEY_HERE 11 MONGODB_ATLAS_PRIVATE_API_KEY: YOUR_PRIVATE_KEY_HERE 12 MONGODB_ATLAS_ORG_ID: YOUR_ORG_ID_HERE 13 MONGODB_ATLAS_PROJECT_ID: YOUR_PROJECT_ID_HERE 14 MONGODB_CLUSTER_NAME: examples 15 MONGODB_USERNAME: demo 16 MONGODB_PASSWORD: password1234 17 # MONGODB_DATABASE: sample_mflix 18 # MONGODB_COLLECTION: movies 19 CLEANUP_ONDESTROY: true
You'll want to swap the environment variable values with your own. In the above example, the database and collection variables are commented out so the defaults would be used in the ZX script.
To see everything in action, execute the following from the command line on the host computer:
1 docker-compose up
The above command will use the docker-compose.yml file to build the Docker image if it doesn't already exist. The build process will bundle our files, install our dependencies, and obtain the MongoDB Atlas CLI. When Compose deploys a container from the image, the environment variables will be passed to the ZX script responsible for configuring MongoDB Atlas. When ready, the ZX script will run the Node.js application, further passing the environment variables. If the
CLEANUP_ONDESTROY variable was set to true, when the container is stopped the database and collection will be removed.The MongoDB Atlas CLI can be a powerful tool for bringing MongoDB Atlas to your local development experience on Docker. Essentially you would be swapping out a local version of MongoDB with Atlas CLI logic to manage a more feature-rich cloud version of MongoDB.
MongoDB Atlas enhances the MongoDB experience by giving you access to more features such as Atlas Search, Charts, and App Services, which allow you to build great applications with minimal effort.