Building a NodeJS App with MongoDB Atlas and AWS Elastic Container Service - Part 2

Jay Gordon
November 9, 2017 | Updated: June 26, 2020

It's that time of year again! This post is part of our Road to AWS re:Invent 2017 blog series. In the weeks leading up to AWS re:Invent in Las Vegas this November, we'll be posting about a number of topics related to running MongoDB in the public cloud. See all posts here.

In my last post, we started preparing an application built on Node.js and MongoDB Atlas for simple CRUD operations. We've completed the initial configuration of the code and are now ready to launch this into production.

As mentioned in part one, I want to minimize the long-term maintenance of this app's hosting environment. Much like we used MongoDB Atlas to offload many of the operational responsibilities for our database, we can make use of Amazon EC2 Container Service to deploy our Docker apps on AWS. By reducing the amount of patching, systems maintenance, and long term security concerns for both our database and our application front-end, we’re able to dedicate more time to application development.

Ready Docker and coldbrew-cli for deploy

Docker and coldbrew-cli have simple configuration files we'll review and write in our repo's root directory.

Docker

Let's take a look at the Dockerfile:


FROM node:boron

RUN mkdir -p /usr/src/app
WORKDIR /usr/src/app

COPY . /usr/src/app

EXPOSE 3000

CMD ["npm", "start"]

This file will tell Docker to follow the instructions to use the "boron" LTS version of Node.js, meaning we won't need to install it manually to work with our application. After establishing the appropriate version of Node.js to use, we'll then make a working directory on the container telling it where the app will live. The app will be copied into the working directory, "/usr/src/app", and finally be started using the npm start command stated in our package.json file.

Place the contents of the Dockerfile in the root of the code repository and save it.

echo "FROM node:boron
>
> RUN mkdir -p /usr/src/app
>WORKDIR /usr/src/app
>
>COPY . /usr/src/app
>
>EXPOSE 3000
>
>CMD ["npm", "start"]" > Dockerfile

Next, we'll start working with coldbrew-cli. Much like Docker, we’ll create a plain text config file that will contain basic instructions on how to configure our infrastructure for our app.

coldbrew-cli

Let’s create a file called coldbrew.conf in the root of our repository directory and then store the following contents in it:

touch coldbrew.conf

name: mern-demo
cluster: mern-demo

port: 3000
units: 2
cpu: 1.0
memory: 500m

load_balancer:
  enabled: true
  health_check:
    path: /
    status: 200

For a full breakdown of all the terms in this file, have a look at the coldbrew-cli docs. I have provided a simple configuration file that will set up our elastic load balancing (ELB) with a compute instance for our container and 500 MB of available memory per app. We can even configure a simple health check for our ELB to ensure our instances are online.

We can now create our environment with coldbrew-cli. For the purposes of this walkthrough, let’s say we want to deploy our ECS cluster in the us-east-2 region and that we don’t need to access our nodes via SSH; we can launch using the following command:


$ coldbrew --aws-region="us-east-2" cluster-create mern-demo --disable-keypair

Once executing this, we'll be shown a list of resources that will be created:

Determining AWS resources to create...
  ECS Cluster: coldbrew-mern-demo
  IAM Role for ECS Services: coldbrew-mern-demo-ecs-service-role
  EC2 Launch Configuration for ECS Container Instances: coldbrew-mern-demo-lc
  EC2 Auto Scaling Group for ECS Container Instances: coldbrew-mern-demo-asg
  IAM Instance Profile for ECS Container Instances: coldbrew-mern-demo-instance-profile
  EC2 Security Group for ECS Container Instances: coldbrew-mern-demo-instance-sg

Do you want to create these resources? [y/N]:

Answer “Yes”, and your resources will start building in the background.


[+] Creating IAM Instance Profile [coldbrew-mern-demo-instance-profile]...
[+] Creating EC2 Security Group [coldbrew-mern-demo-instance-sg]...
[*] Adding inbound rule [tcp:22:0.0.0.0/0] to EC2 Security Group [coldbrew-mern-demo-instance-sg]...
[+] Creating EC2 Launch Configuration [coldbrew-mern-demo-lc]... (this may take long)
[+] Creating EC2 Auto Scaling Group [coldbrew-mern-demo-asg]... (this may take long)
[+] Creating ECS Cluster [coldbrew-mern-demo]...
[+] Creating IAM Role [coldbrew-mern-demo-ecs-service-role]...
</command></pre>
<p>We can query the status with the following command:</p>
<pre><code class="language-plaintext">
$ coldbrew --aws-region="us-east-2" cluster-status mern-demo

Here is the command output similar to what we would get:


Cluster
  Name: mern-demo
AWS
  Region: us-east-2
  VPC: vpc-7935db10
  Subnets: subnet-70f0df3a subnet-58e01531 subnet-49d4db31
ECS
  ECS Cluster: coldbrew-mern-demo
  IAM Role for ECS Services: coldbrew-mern-demo-ecs-service-role
  ECS Services: 0
  ECS Tasks (running/pending): 0/0
  ECS Container Instances: 1
Auto Scaling
  EC2 Launch Configuration: coldbrew-mern-demo-lc
    IAM Instance Profile: coldbrew-mern-demo-instance-profile
    Instance Type: t2.micro
    Image ID: ami-bd3e64d8
    Key Pair:
    Security Groups: coldbrew-mern-demo-instance-sg
  EC2 Auto Scaling Group: coldbrew-mern-demo-asg
    Instances (current/desired/min/max): 1/1/0/1
ECS Container Instance
  ID: a56d40d1-7095-45a0-af81-5f309bbbd728
  Status: ACTIVE
  Tasks (running/pending): 0/0
  CPU (remaining/registered): 1.00/1.00
  Memory (remaining/registered): 995M/995M,
  EC2 Instance ID: i-03fdc038f3d1c71c8
    Private IP: 172.31.44.140
   <strong> Public IP: 18.221.72.130</strong>

Wow, that's a lot of saved work. Everything from our VPC all the way to the ELB was created for us. See the public IP for our compute instance? Let's make sure it's whitelisted in our Atlas cluster so our data can be saved.

If you were to replace the M0 free cluster with one of Atlas’s dedicated clusters, you'd have access to our VPC peering module for AWS, giving you the ability to whitelist the entire range of host servers via a security group entry in the whitelist.

Now it's time to build our Docker image with coldbrew-cli and deploy our app.

First a Docker image is created and saved:


$ coldbrew --aws-region="us-east-2" deploy


[*] Checking cluster availability [mern-demo]...
[+] Creating ECR Repository [coldbrew/mern-demo]...
[*] Building Docker image [722245653955.dkr.ecr.us-east-2.amazonaws.com/coldbrew/mern-demo:latest]... (this may take long)

> docker build -t 722245653955.dkr.ecr.us-east-2.amazonaws.com/coldbrew/mern-demo:latest -f /Users/jaygordon/work/mern-crud/Dockerfile /Users/jaygordon/work/mern-crud

Then the image is pushed to the appropriate nodes in the cluster:

[*] Pushing Docker image [722245653955.dkr.ecr.us-east-2.amazonaws.com/coldbrew/mern-demo:latest]... (this may take long)

> docker push 722245653955.dkr.ecr.us-east-2.amazonaws.com/coldbrew/mern-demo:latest
The push refers to a repository [722245653955.dkr.ecr.us-east-2.amazonaws.com/coldbrew/mern-demo]
a791685d46cb: Pushed
e71eccb6eee4: Pushed
b7f1d9d858aa: Pushed
246ae56dbdbd: Pushed
e271ac6d0c18: Pushed
682e7cee9d37: Pushed
d359ab38b013: Pushed
latest: digest: sha256:c58153d1fe62dacb1644966ffe4acca6b76cb383aee1f76e0efd97ceaa1a306e size: 2425

[*] Updating ECS Task Definition [mern-demo]...
[+] Creating ELB Target Group [mern-demo-elb-tg]...
[+] Creating EC2 Security Group [mern-demo-elb-sg]...
[*] Adding inbound rule [tcp:80:0.0.0.0/0] to EC2 Security Group [mern-demo-elb-sg]...
[*] Adding inbound rule [tcp:0:sg-f8ab7190] to EC2 Security Group [coldbrew-mern-demo-instance-sg]...
[+] Creating ELB Load Balancer [mern-demo-elb]...
[+] Adding listener (HTTP) for ELB Load Balancer [mern-demo-elb]...
[+] Creating ECS Service [mern-demo]...

Application deployment completed.

Our app is now deployed. Let's get the ELB and verify:

$ coldbrew --aws-region="us-east-2" status |egrep elb
  
ELB Target Group: mern-demo-elb-tg
  ELB Load Balancer: mern-demo-elb
    Endpoint: <a href="http://mern-demo-elb-2131866240.us-east-2.elb.amazonaws.com:80" target="_blank">http://mern-demo-elb-2131866240.us-east-2.elb.amazonaws.com:80</a>

The ELB now provides us with an http endpoint to access our app. There's no need to use nginx or any other http server to reverse proxy the Node.js port. The coldbrew-cli deploy process will set up your port forwarding based on the information in the coldbrew.conf file. The cluster creation and deployment takes about five minutes. To test that we're online and running, simply go to the URL provided by coldbrew and add a record to our app (first load may take a minute):

Congratulations, we now have all the tools to build and configure our own ECS cluster using MongoDB Atlas! Destroying the app is pretty simple as well — just run these two commands to terminate all the resources associated with ECS.

$ coldbrew --aws-region="us-east-2" delete
Determining AWS resources that need to be deleted...
  ECS Service: mern-demo
  ECR Repository: coldbrew/mern-demo
  ELB Target Group: mern-demo-elb-tg
  ELB Load Balancer: mern-demo-elb
  EC2 Security Group for ELB Load Balancer: mern-demo-elb-sg
> Do you want to delete these resources? [y/N]: y

After we answer “yes”, the created resources will begin terminating:

[*] Updating ECS Service to stop all tasks [mern-demo]...
[-] Deleting ELB Load Balancer [mern-demo-elb]...
[-] Deleting ELB Target Group [mern-demo-elb-tg]... (this may take long)
[-] Removing inbound rule [tcp:0:sg-f8ab7190] from EC2 Security Group [coldbrew-mern-demo-instance-sg]...
[-] Deleting EC2 Security Group for ELB Load Balancer [mern-demo-elb-sg]... (this may take long)
[-] Deleting ECR Repository [coldbrew/mern-demo]...
[-] Deleting (and draining) ECS Service [mern-demo]... (this may take long)

And now delete the remaining cluster elements:

$ coldbrew --aws-region="us-east-2" cluster-delete mern-demo

Determining AWS resources that need to be deleted...
  ECS Cluster: coldbrew-mern-demo
  IAM Role for ECS Services: coldbrew-mern-demo-ecs-service-role
  EC2 Launch Configuration for ECS Container Instances: coldbrew-mern-demo-lc
  EC2 Auto Scaling Group for ECS Container Instances: coldbrew-mern-demo-asg
  IAM Instance Profile for ECS Container Instances: coldbrew-mern-demo-instance-profile
  EC2 Security Group for ECS Container Instances: coldbrew-mern-demo-instance-sg

> Do you want to delete these resources? [y/N]: y

Answer “yes” and we'll see the remaining compute cluster elements terminated:

[*] Terminating instances in EC2 Auto Scaling Group [coldbrew-mern-demo-asg]... (this may take long)
[-] Deleting EC2 Auto Scaling Group [coldbrew-mern-demo-asg]... (this may take long)
[-] Deleting EC2 Launch Configuration [coldbrew-mern-demo-lc]... 
[-] Deleting IAM Instance Profile [coldbrew-mern-demo-instance-profile]... 
[-] Deleting EC2 Security Group [coldbrew-mern-demo-instance-sg]... 
[-] Deleting ECS Cluster [coldbrew-mern-demo]... 
[-] Deleting IAM Role [coldbrew-mern-demo-ecs-service-role]...

What's next?

Try this process out with your own application or even with a dedicated MongoDB Atlas cluster to enable VPC peering. You can get a full tutorial on how to configure VPC Peering in MongoDB Atlas by viewing the "Peering your MongoDB Atlas Cluster to AWS" video. You'll be able to use a completely service-based deployment of your application that requires no operating systems to be managed, no kernels to update, and less overall manual work.

If you'd like to sign up for a free MongoDB Atlas cluster, check out our signup page here!

← Previous

Announcing MongoDB 3.6

Today we’re announcing the release of MongoDB 3.6 , the latest milestone in our efforts to give developers the most productive general-purpose data management systems for modern applications. 3.6 makes it easier than ever to work with data in the most natural, efficient, and frictionless way possible. Allowing developers to persist rich, nested data without flattening it out is one of MongoDB’s defining strengths. Documents can model any kind of data: key value, graph, and relational data sets are as much at home in documents as heterogeneous, nested structures are. With MongoDB Server 3.6, we’re making the query language much more powerful, with new array update operators that allow you to specify in-place updates to specific, matching array items, at any depth of nesting. Extensions to the $lookup aggregation stage now allow uncorrelated subqueries and multiple matching conditions, so referencing and joining documents in complex combinations can be handled in the database. Modern applications need to respond instantaneously to changes, providing notifications to their users and interfaces that update live. To enable this, MongoDB 3.6 introduces Change Streams, which applications can use to get real-time notification of updates to collection data. Just a few lines of code yield a tailable cursor that continuously delivers notifications on updates to the watched collection. A key characteristic of robust systems is that they handle network outages gracefully, but the defensive coding to handle them can represent a significant burden on developers. MongoDB 3.6 alleviates that burden with Retryable Writes, a new feature ensuring that writes are performed exactly once, even in the face of outages. MongoDB 3.6 also concludes our march to uniformly secure defaults, with localhost only binding implemented directly in the core server. Since 2.6.0, this behavior has been the default configuration in the most popular packaged versions of MongoDB, but the server itself did not default to localhost only binding, because many customer infrastructures use border security and relied on the previously permissive behavior. As of this release, MongoDB Server itself will default to refusing all connections unless they are from a whitelisted IP. The flexibility of documents is fully compatible with data validation, and MongoDB 3.6 improves on its previous capabilities with the introduction of JSON Schema. Using JSON Schema, you can specify (in JSON), on a per-collection basis, exactly what qualifies as a valid document, such as the types a field can have, whether it is required, and if a document permits fields not listed in the spec. With MongoDB 3.6, schema isn’t a straightjacket, it’s framework of validation you can tune to exactly the degree you need. MongoDB Atlas has seen an incredible collection of improvements in 3.6. Cross-Region Clusters provide a whole new level of high-availability at the click of a button, whether you run Atlas on AWS, Google Cloud Platform, or Azure. We’ve also introduced two new instance classes, M2 and M5. These low-cost, shared tenancy instances bridge the gap between the free tier and the fully dedicated tiers, and are perfect for the development phase of a project between toy and production. On the operations side, Atlas and Ops Manager have both gained a Data Explorer for quickly digging into the data in your database, and a Performance Advisor, which analyzes the queries running in your cluster and finds opportunities for improvement, like missing indexes to add. The BI Connector , introduced last year, has been completely re-written for MongoDB 3.6 making it run much faster and easier to manage. The BI connector 2.0 is much better at translating SQL queries into MongoDB’s native aggregation framework, so it can push much more work directly down to MongoDB, instead of having to do it itself in memory. With over a hundred new and updated features , MongoDB 3.6 is our biggest release yet, and we’re incredibly excited to get it into your hands and see what amazing things you do with it. Learn more About the Author Eliot Horowitz is the CTO and Co-Founder of MongoDB. He wrote the core code base for MongoDB starting in 2007, and subsequently built the engineering and product teams. Today, Eliot oversees those teams, and continues to drive technology innovations at MongoDB. Prior to MongoDB, Eliot co-founded and built ShopWiki, a groundbreaking online retail search engine. He built its technology, it’s team, and presided over its private sale in 2010. Before that, Eliot was a software developer in the R&D group at DoubleClick. Eliot is on the board of the NY Tech Talent Pipeline. In 2006, he was selected as one of BusinessWeek’s Top 25 Entrepreneurs Under Age 25, and in 2015 was named to the Business Insider “Under 35 and Crushing it” list. He was also recently named to Crain’s NY Business 40 Under 40 Class of 2017 list. Eliot received a B.S. in Computer Science from Brown University.

November 8, 2017

Next →

MongoDB and Partners: Building the AI Future, Together

If you’re like me, over the past year you’ve closely watched AI’s developments—and the world’s reactions to them. From infectious excitement about AI’s capabilities, to impatience with its cost and return on investment, every day has been filled with AI twists and turns. It’s been quite the roller coaster. During the ride, from time to time I’ve wondered where AI falls on the Gartner hype cycle, which gives "a view of how a technology or application will evolve over time." Have we hit the "peak of inflated expectations" only to fall into the "trough of disillusionment?" Or is the hype cycle an imperfect guide, as The Economist argues? The reality is that it takes time for any new technology—even transformative ones like AI—to take hold. And every advance, no matter how big, has had its detractors. A famous example is that of Picasso (!), who in 1968 said, “Computers are useless. They can only give you answers.” (!!) For our part, MongoDB is convinced that AI is a once-in-a-generation technology that will enhance every future application—a belief that has been reinforced by the incredible work our partners have shared at MongoDB’s 2024 events. Speeding AI development MongoDB is committed to helping organizations of all sizes succeed with AI, and one way we’re doing that is by collaborating with the MongoDB partner ecosystem to create powerful, user-friendly AI development tools and solutions. For example, Fireworks.ai —which is a member of the MongoDB AI Applications Program ecosystem —created an inference solution that hosts gen AI models and supports containerized deployments. This tool makes it easier for developers to build and deploy powerful applications with a range of easy-to-use tools and customization options. They can choose to use state-of-the-art, open-source language, image, and multimodal foundation models off the shelf, or they can customize and fine-tune models to their needs. Jointly, Fireworks.ai and MongoDB provide a solution for developers who want to leverage highly curated and optimized open-source models and combine these with their organization’s own proprietary data—and to do so with unparalleled speed and security. “MongoDB is one of the most sophisticated database providers, and it’s very easy to use,” said Benny Chen , cofounder of Fireworks.ai. "We want developers to be able to use these tools, and we want to work with providers who enable and empower developers." Nomic , another MAAP ecosystem member, also enables developers with best-in-class solutions across the entire unstructured data workflow. Their Embed offering, available through the Nomic API , allows users to vectorize large-scale datasets for use in text, image, and multimodal retrieval applications, including retrieval-augmented generation (RAG), using only their web browser. The Nomic-MongoDB solution is a highly efficient, open-weight model that developers can use to visualize the unstructured datasets they store in MongoDB Atlas . These insights help users quickly discover trends and articulate data-driven value propositions. Nomic also supported the recently announced vector quantization in MongoDB Atlas Vector Search , which reduces vector sizes while preserving performance. Last—but hardly least!—there’s our new reference architecture with MAAP partners AWS and Anthropic. Announced at MongoDB.local London , the reference architecture supports building memory-enhanced AI agents, and is designed to streamline complex processes and develop smarter, more responsive applications. For more—including a link to the code on Github— check out the MongoDB Developer Center . Making AI work for anyone and everyone The companies MongoDB partners with aren’t just making gen AI easier for developers—they’re building tools for everyone. For example, Capgemini has invested $2 billion in gen AI and is training 100,000 of its employees in the technology. GenYoda, a solution that helps insurance professionals with their daily work, is a product of this investment. GenYoda leverages MongoDB Atlas Vector Search to analyze large amounts of customer data, like policy statements, premiums, claims history, and health information. Using GenYoda, insurance professionals can quickly analyze underwriters’ reports to make informed decisions, create longitudinal health summaries, and streamline customer interactions to improve contact center efficiency. GenYoda can ingest 100,000 documents in just a few hours and respond to users’ queries in two to three seconds—a metric on par with the most widely used gen AI models. And it produces results: in one example, by using Capgemini’s solution an insurer was able to increase productivity by 15%, add new reports 25% faster (thus speeding decision-making), and reduce the manual effort of searching PDFs, increasing efficiency by 10%. Building the future of AI together So, what’s next? Honestly, I’m as curious as you are. But I’m also incredibly excited. At MongoDB, we’re active participants in the AI revolution, working to embrace the possibilities that lie ahead. The future of gen AI is bright, and I can’t wait to see what we’ll build together. To learn more about how MongoDB can accelerate your AI journey, explore the MongoDB AI Applications Program .

November 4, 2024