Building a NodeJS App with MongoDB Atlas and AWS Elastic Container Service - Part 2

Jay Gordon
November 9, 2017 | Updated: June 26, 2020

It's that time of year again! This post is part of our Road to AWS re:Invent 2017 blog series. In the weeks leading up to AWS re:Invent in Las Vegas this November, we'll be posting about a number of topics related to running MongoDB in the public cloud. See all posts here.

In my last post, we started preparing an application built on Node.js and MongoDB Atlas for simple CRUD operations. We've completed the initial configuration of the code and are now ready to launch this into production.

As mentioned in part one, I want to minimize the long-term maintenance of this app's hosting environment. Much like we used MongoDB Atlas to offload many of the operational responsibilities for our database, we can make use of Amazon EC2 Container Service to deploy our Docker apps on AWS. By reducing the amount of patching, systems maintenance, and long term security concerns for both our database and our application front-end, we’re able to dedicate more time to application development.

Ready Docker and coldbrew-cli for deploy

Docker and coldbrew-cli have simple configuration files we'll review and write in our repo's root directory.

Docker

Let's take a look at the Dockerfile:


FROM node:boron

RUN mkdir -p /usr/src/app
WORKDIR /usr/src/app

COPY . /usr/src/app

EXPOSE 3000

CMD ["npm", "start"]

This file will tell Docker to follow the instructions to use the "boron" LTS version of Node.js, meaning we won't need to install it manually to work with our application. After establishing the appropriate version of Node.js to use, we'll then make a working directory on the container telling it where the app will live. The app will be copied into the working directory, "/usr/src/app", and finally be started using the npm start command stated in our package.json file.

Place the contents of the Dockerfile in the root of the code repository and save it.

echo "FROM node:boron
>
> RUN mkdir -p /usr/src/app
>WORKDIR /usr/src/app
>
>COPY . /usr/src/app
>
>EXPOSE 3000
>
>CMD ["npm", "start"]" > Dockerfile

Next, we'll start working with coldbrew-cli. Much like Docker, we’ll create a plain text config file that will contain basic instructions on how to configure our infrastructure for our app.

coldbrew-cli

Let’s create a file called coldbrew.conf in the root of our repository directory and then store the following contents in it:

touch coldbrew.conf

name: mern-demo
cluster: mern-demo

port: 3000
units: 2
cpu: 1.0
memory: 500m

load_balancer:
  enabled: true
  health_check:
    path: /
    status: 200

For a full breakdown of all the terms in this file, have a look at the coldbrew-cli docs. I have provided a simple configuration file that will set up our elastic load balancing (ELB) with a compute instance for our container and 500 MB of available memory per app. We can even configure a simple health check for our ELB to ensure our instances are online.

We can now create our environment with coldbrew-cli. For the purposes of this walkthrough, let’s say we want to deploy our ECS cluster in the us-east-2 region and that we don’t need to access our nodes via SSH; we can launch using the following command:


$ coldbrew --aws-region="us-east-2" cluster-create mern-demo --disable-keypair

Once executing this, we'll be shown a list of resources that will be created:

Determining AWS resources to create...
  ECS Cluster: coldbrew-mern-demo
  IAM Role for ECS Services: coldbrew-mern-demo-ecs-service-role
  EC2 Launch Configuration for ECS Container Instances: coldbrew-mern-demo-lc
  EC2 Auto Scaling Group for ECS Container Instances: coldbrew-mern-demo-asg
  IAM Instance Profile for ECS Container Instances: coldbrew-mern-demo-instance-profile
  EC2 Security Group for ECS Container Instances: coldbrew-mern-demo-instance-sg

Do you want to create these resources? [y/N]:

Answer “Yes”, and your resources will start building in the background.


[+] Creating IAM Instance Profile [coldbrew-mern-demo-instance-profile]...
[+] Creating EC2 Security Group [coldbrew-mern-demo-instance-sg]...
[*] Adding inbound rule [tcp:22:0.0.0.0/0] to EC2 Security Group [coldbrew-mern-demo-instance-sg]...
[+] Creating EC2 Launch Configuration [coldbrew-mern-demo-lc]... (this may take long)
[+] Creating EC2 Auto Scaling Group [coldbrew-mern-demo-asg]... (this may take long)
[+] Creating ECS Cluster [coldbrew-mern-demo]...
[+] Creating IAM Role [coldbrew-mern-demo-ecs-service-role]...
</command></pre>
<p>We can query the status with the following command:</p>
<pre><code class="language-plaintext">
$ coldbrew --aws-region="us-east-2" cluster-status mern-demo

Here is the command output similar to what we would get:


Cluster
  Name: mern-demo
AWS
  Region: us-east-2
  VPC: vpc-7935db10
  Subnets: subnet-70f0df3a subnet-58e01531 subnet-49d4db31
ECS
  ECS Cluster: coldbrew-mern-demo
  IAM Role for ECS Services: coldbrew-mern-demo-ecs-service-role
  ECS Services: 0
  ECS Tasks (running/pending): 0/0
  ECS Container Instances: 1
Auto Scaling
  EC2 Launch Configuration: coldbrew-mern-demo-lc
    IAM Instance Profile: coldbrew-mern-demo-instance-profile
    Instance Type: t2.micro
    Image ID: ami-bd3e64d8
    Key Pair:
    Security Groups: coldbrew-mern-demo-instance-sg
  EC2 Auto Scaling Group: coldbrew-mern-demo-asg
    Instances (current/desired/min/max): 1/1/0/1
ECS Container Instance
  ID: a56d40d1-7095-45a0-af81-5f309bbbd728
  Status: ACTIVE
  Tasks (running/pending): 0/0
  CPU (remaining/registered): 1.00/1.00
  Memory (remaining/registered): 995M/995M,
  EC2 Instance ID: i-03fdc038f3d1c71c8
    Private IP: 172.31.44.140
   <strong> Public IP: 18.221.72.130</strong>

Wow, that's a lot of saved work. Everything from our VPC all the way to the ELB was created for us. See the public IP for our compute instance? Let's make sure it's whitelisted in our Atlas cluster so our data can be saved.

If you were to replace the M0 free cluster with one of Atlas’s dedicated clusters, you'd have access to our VPC peering module for AWS, giving you the ability to whitelist the entire range of host servers via a security group entry in the whitelist.

Now it's time to build our Docker image with coldbrew-cli and deploy our app.

First a Docker image is created and saved:


$ coldbrew --aws-region="us-east-2" deploy


[*] Checking cluster availability [mern-demo]...
[+] Creating ECR Repository [coldbrew/mern-demo]...
[*] Building Docker image [722245653955.dkr.ecr.us-east-2.amazonaws.com/coldbrew/mern-demo:latest]... (this may take long)

> docker build -t 722245653955.dkr.ecr.us-east-2.amazonaws.com/coldbrew/mern-demo:latest -f /Users/jaygordon/work/mern-crud/Dockerfile /Users/jaygordon/work/mern-crud

Then the image is pushed to the appropriate nodes in the cluster:

[*] Pushing Docker image [722245653955.dkr.ecr.us-east-2.amazonaws.com/coldbrew/mern-demo:latest]... (this may take long)

> docker push 722245653955.dkr.ecr.us-east-2.amazonaws.com/coldbrew/mern-demo:latest
The push refers to a repository [722245653955.dkr.ecr.us-east-2.amazonaws.com/coldbrew/mern-demo]
a791685d46cb: Pushed
e71eccb6eee4: Pushed
b7f1d9d858aa: Pushed
246ae56dbdbd: Pushed
e271ac6d0c18: Pushed
682e7cee9d37: Pushed
d359ab38b013: Pushed
latest: digest: sha256:c58153d1fe62dacb1644966ffe4acca6b76cb383aee1f76e0efd97ceaa1a306e size: 2425

[*] Updating ECS Task Definition [mern-demo]...
[+] Creating ELB Target Group [mern-demo-elb-tg]...
[+] Creating EC2 Security Group [mern-demo-elb-sg]...
[*] Adding inbound rule [tcp:80:0.0.0.0/0] to EC2 Security Group [mern-demo-elb-sg]...
[*] Adding inbound rule [tcp:0:sg-f8ab7190] to EC2 Security Group [coldbrew-mern-demo-instance-sg]...
[+] Creating ELB Load Balancer [mern-demo-elb]...
[+] Adding listener (HTTP) for ELB Load Balancer [mern-demo-elb]...
[+] Creating ECS Service [mern-demo]...

Application deployment completed.

Our app is now deployed. Let's get the ELB and verify:

$ coldbrew --aws-region="us-east-2" status |egrep elb
  
ELB Target Group: mern-demo-elb-tg
  ELB Load Balancer: mern-demo-elb
    Endpoint: <a href="http://mern-demo-elb-2131866240.us-east-2.elb.amazonaws.com:80" target="_blank">http://mern-demo-elb-2131866240.us-east-2.elb.amazonaws.com:80</a>

The ELB now provides us with an http endpoint to access our app. There's no need to use nginx or any other http server to reverse proxy the Node.js port. The coldbrew-cli deploy process will set up your port forwarding based on the information in the coldbrew.conf file. The cluster creation and deployment takes about five minutes. To test that we're online and running, simply go to the URL provided by coldbrew and add a record to our app (first load may take a minute):

Congratulations, we now have all the tools to build and configure our own ECS cluster using MongoDB Atlas! Destroying the app is pretty simple as well — just run these two commands to terminate all the resources associated with ECS.

$ coldbrew --aws-region="us-east-2" delete
Determining AWS resources that need to be deleted...
  ECS Service: mern-demo
  ECR Repository: coldbrew/mern-demo
  ELB Target Group: mern-demo-elb-tg
  ELB Load Balancer: mern-demo-elb
  EC2 Security Group for ELB Load Balancer: mern-demo-elb-sg
> Do you want to delete these resources? [y/N]: y

After we answer “yes”, the created resources will begin terminating:

[*] Updating ECS Service to stop all tasks [mern-demo]...
[-] Deleting ELB Load Balancer [mern-demo-elb]...
[-] Deleting ELB Target Group [mern-demo-elb-tg]... (this may take long)
[-] Removing inbound rule [tcp:0:sg-f8ab7190] from EC2 Security Group [coldbrew-mern-demo-instance-sg]...
[-] Deleting EC2 Security Group for ELB Load Balancer [mern-demo-elb-sg]... (this may take long)
[-] Deleting ECR Repository [coldbrew/mern-demo]...
[-] Deleting (and draining) ECS Service [mern-demo]... (this may take long)

And now delete the remaining cluster elements:

$ coldbrew --aws-region="us-east-2" cluster-delete mern-demo

Determining AWS resources that need to be deleted...
  ECS Cluster: coldbrew-mern-demo
  IAM Role for ECS Services: coldbrew-mern-demo-ecs-service-role
  EC2 Launch Configuration for ECS Container Instances: coldbrew-mern-demo-lc
  EC2 Auto Scaling Group for ECS Container Instances: coldbrew-mern-demo-asg
  IAM Instance Profile for ECS Container Instances: coldbrew-mern-demo-instance-profile
  EC2 Security Group for ECS Container Instances: coldbrew-mern-demo-instance-sg

> Do you want to delete these resources? [y/N]: y

Answer “yes” and we'll see the remaining compute cluster elements terminated:

[*] Terminating instances in EC2 Auto Scaling Group [coldbrew-mern-demo-asg]... (this may take long)
[-] Deleting EC2 Auto Scaling Group [coldbrew-mern-demo-asg]... (this may take long)
[-] Deleting EC2 Launch Configuration [coldbrew-mern-demo-lc]... 
[-] Deleting IAM Instance Profile [coldbrew-mern-demo-instance-profile]... 
[-] Deleting EC2 Security Group [coldbrew-mern-demo-instance-sg]... 
[-] Deleting ECS Cluster [coldbrew-mern-demo]... 
[-] Deleting IAM Role [coldbrew-mern-demo-ecs-service-role]...

What's next?

Try this process out with your own application or even with a dedicated MongoDB Atlas cluster to enable VPC peering. You can get a full tutorial on how to configure VPC Peering in MongoDB Atlas by viewing the "Peering your MongoDB Atlas Cluster to AWS" video. You'll be able to use a completely service-based deployment of your application that requires no operating systems to be managed, no kernels to update, and less overall manual work.

If you'd like to sign up for a free MongoDB Atlas cluster, check out our signup page here!

← Previous

Announcing MongoDB 3.6

Today we’re announcing the release of MongoDB 3.6 , the latest milestone in our efforts to give developers the most productive general-purpose data management systems for modern applications. 3.6 makes it easier than ever to work with data in the most natural, efficient, and frictionless way possible. Allowing developers to persist rich, nested data without flattening it out is one of MongoDB’s defining strengths. Documents can model any kind of data: key value, graph, and relational data sets are as much at home in documents as heterogeneous, nested structures are. With MongoDB Server 3.6, we’re making the query language much more powerful, with new array update operators that allow you to specify in-place updates to specific, matching array items, at any depth of nesting. Extensions to the $lookup aggregation stage now allow uncorrelated subqueries and multiple matching conditions, so referencing and joining documents in complex combinations can be handled in the database. Modern applications need to respond instantaneously to changes, providing notifications to their users and interfaces that update live. To enable this, MongoDB 3.6 introduces Change Streams, which applications can use to get real-time notification of updates to collection data. Just a few lines of code yield a tailable cursor that continuously delivers notifications on updates to the watched collection. A key characteristic of robust systems is that they handle network outages gracefully, but the defensive coding to handle them can represent a significant burden on developers. MongoDB 3.6 alleviates that burden with Retryable Writes, a new feature ensuring that writes are performed exactly once, even in the face of outages. MongoDB 3.6 also concludes our march to uniformly secure defaults, with localhost only binding implemented directly in the core server. Since 2.6.0, this behavior has been the default configuration in the most popular packaged versions of MongoDB, but the server itself did not default to localhost only binding, because many customer infrastructures use border security and relied on the previously permissive behavior. As of this release, MongoDB Server itself will default to refusing all connections unless they are from a whitelisted IP. The flexibility of documents is fully compatible with data validation, and MongoDB 3.6 improves on its previous capabilities with the introduction of JSON Schema. Using JSON Schema, you can specify (in JSON), on a per-collection basis, exactly what qualifies as a valid document, such as the types a field can have, whether it is required, and if a document permits fields not listed in the spec. With MongoDB 3.6, schema isn’t a straightjacket, it’s framework of validation you can tune to exactly the degree you need. MongoDB Atlas has seen an incredible collection of improvements in 3.6. Cross-Region Clusters provide a whole new level of high-availability at the click of a button, whether you run Atlas on AWS, Google Cloud Platform, or Azure. We’ve also introduced two new instance classes, M2 and M5. These low-cost, shared tenancy instances bridge the gap between the free tier and the fully dedicated tiers, and are perfect for the development phase of a project between toy and production. On the operations side, Atlas and Ops Manager have both gained a Data Explorer for quickly digging into the data in your database, and a Performance Advisor, which analyzes the queries running in your cluster and finds opportunities for improvement, like missing indexes to add. The BI Connector , introduced last year, has been completely re-written for MongoDB 3.6 making it run much faster and easier to manage. The BI connector 2.0 is much better at translating SQL queries into MongoDB’s native aggregation framework, so it can push much more work directly down to MongoDB, instead of having to do it itself in memory. With over a hundred new and updated features , MongoDB 3.6 is our biggest release yet, and we’re incredibly excited to get it into your hands and see what amazing things you do with it. Learn more About the Author Eliot Horowitz is the CTO and Co-Founder of MongoDB. He wrote the core code base for MongoDB starting in 2007, and subsequently built the engineering and product teams. Today, Eliot oversees those teams, and continues to drive technology innovations at MongoDB. Prior to MongoDB, Eliot co-founded and built ShopWiki, a groundbreaking online retail search engine. He built its technology, it’s team, and presided over its private sale in 2010. Before that, Eliot was a software developer in the R&D group at DoubleClick. Eliot is on the board of the NY Tech Talent Pipeline. In 2006, he was selected as one of BusinessWeek’s Top 25 Entrepreneurs Under Age 25, and in 2015 was named to the Business Insider “Under 35 and Crushing it” list. He was also recently named to Crain’s NY Business 40 Under 40 Class of 2017 list. Eliot received a B.S. in Computer Science from Brown University.

November 8, 2017

Next →

Hanabi Technologies Uses MongoDB to Power AI Assistant, Hana

For all the hype surrounding generative AI, cynics tend to view the few real-world implementations as little more than “fancy chatbots.” But for Abhinav Aggarwal, CEO of Hanabi Technologies, the idea of a generative AI-powered bot that is more than just an assistant was intriguing. “I’d been using ChatGPT since it launched,” said Aggarwal. “That got me thinking: How could we make a chatbot that was like a team member?” And with that concept, Hana was born. The problem with bots “Most generative AI chatbots do not act like people; they wait for a command and give a response,” said Aggarwal. “We wanted to create a human-like chatbot that would proactively help people based on what they wanted—automating reminders, for example, or fetching time zones from your calendar to correctly schedule meetings.” Hanabi’s flagship product, Hana, is an AI assistant designed to enhance team collaboration within Google Chat, working in concert with Google Workspace and its suite of products. “Our target customers are smaller companies of between 10 and 50 people. At this size you’re not going to build your own agent from scratch,” he said. Hana integrates with Google APIs to deliver a human-like assistant that chimes in with helpful interventions, such as automatically setting reminders and making sure meetings are booked in the right time zone for each participant. “Hana is designed to bring AI to smaller companies and help them collaborate in a space where they are already working—Google Workspace,” Aggarwal explained. The MongoDB Atlas solution For Hana to act like a member of the team, Hanabi needed to process massive amounts of data to support advanced features like retrieval-augmented generation (RAG) for better information retrieval across Google Docs and many other sources. And with a rapidly growing user base of over 600 organizations and 17,000+ installs, Hanabi also required a secure, scalable, and high-performing data storage solution. MongoDB Atlas provided a flexible document model, built-in vector database, and scalable cloud-based infrastructure, freeing Hanabi engineers to build new features for Hana rather than focusing on rote tasks like data extract, transform, and load processes or manual scaling and provisioning. Now, MongoDB Atlas handles a variety of responsibilities: Scalability and security: MongoDB Atlas’s auto-scaling and automatic backup features have enabled Hanabi to seamlessly grow its user base without the need for manual database management. RAG: MongoDB Atlas plays a critical role in Hana’s RAG functionality. The platform enables Hanabi to split Google Docs into small sections, create embeddings, and store these sections in Atlas’s vector database. Development Processes: According to Aggarwal, MongoDB’s flexibility in managing changing schemas has been essential to the company’s fast-paced development cycle. Data Visualization: Using MongoDB Atlas Charts has enabled Hanabi to create comprehensive dashboards for real-time data visualization. This has helped the team track usage, set reminders, and optimize performance without needing to build a manual dashboard. Impact and results With MongoDB Atlas, Hanabi can successfully scale Hana to meet the demands of its rapidly expanding user base. The integration is also enabling Hana to offer powerful features like automatic interactions with customers, advanced information retrieval from Google Docs, and manually added memory snippets, making it an essential tool for teams around the world. Next steps Hanabi plans to continue integrating more tools into Hana while expanding its reach to personal Gmail users. The company is also rolling out a new automatic-interaction feature, further enhancing Hana’s ability to proactively assist users without direct commands. MongoDB Atlas remains a key component of Hanabi’s stack, alongside Google Kubernetes Engine, NestJS, and LangChain, enabling Hanabi to focus on innovating to improve the customer experience. Tech Stack MongoDB Atlas Google Kubernetes Engine NestJS LangChain Are you building AI apps? Join the MongoDB AI Innovators Program today! Successful participants gain access to free MongoDB Atlas credits, technical enablement, and invaluable connections within the broader AI ecosystem. If your company is interested in being featured, we’d love to hear from you. Connect with us at ai_adopters@mongodb.com.

November 21, 2024