Vyopta: Shipping fast and reducing operational costs with MongoDB Atlas

Brent Mills
August 2, 2017 | Updated: August 3, 2017
#Customer Stories

This is a guest post from Brent Mills, Principal Infrastructure Engineer at Vyopta, an analytics company that enterprises use to monitor, support, and optimize their video and unified collaboration systems.

MongoDB has always been very attractive to developers in part due to the ease with which you can set up, manage, and interact with the database. Time-to-market is increasingly important, as businesses require increasing levels of speed and flexibility in how they deliver new products and features to their customers.

But as with any other back-end system, you have to manage monitoring and alerting, the clusters (and all of the idiosyncrasies of every distributed system), backups, encryption, provisioning for scale, etc.; These are all things operations teams have to consider in order to deliver production-ready applications, but they’re also things that can delay innovation velocity and can present operational overhead.

By moving our database to the MongoDB cloud service, we’ve been able to remove much of the operational load, saving us over 15 hours of work per month, enabling us to increase efficiency as we’ve moved apps from development to production.

Our Use Case

Below is a simplified version of our pre-MongoDB Atlas setup. I would imagine this looks familiar to many of you:

First off, there’s nothing wrong with this infrastructure, in fact it’s served us pretty well we’ve experienced very few issues. Even so, we were spending a non-trivial amount of time each month on operational overhead. Troubleshooting backup failures, relocating instances because of a network revamp, spinning up new environments for testing, configuring encryption for SOC2 compliance and additional requirements for customers storing data outside of the United States. All of this maintenance work adds up, slowing us down. We’d rather have our development team pushing out new features, and our DevOps team running our CI/CD process, or supporting new initiatives like our move to a microservices architecture.

MongoDB Atlas to the Rescue

Personally I’ve never been a big fan of managed cloud services. They always seem to be lacking important features, have frustrating usability issues, and/or monetarily punish you in creative ways. MongoDB Atlas, however, was very attractive to us for the following reasons:

  • Fully functional MongoDB instance. With the ability to upgrade versions in a few clicks you can always have the newest feature set. It’s also close to identical to what you would deploy on-premise, not some specific “hosted” variation that can’t do x, y, or z.
  • Easy backup and restore. Click to enable, edit the schedule in one pane with cost estimation and options for separate daily, weekly, and monthly retention periods. Straight-forward restore with option to download or query data in place, without having to restore the complete backup.
  • Integrated metrics and alerts. No more setting up agents on every node. Built in alert integrations with services like PagerDuty.
  • Easy scale up/down. Need more RAM? Edit the cluster configuration and select a bigger size. Need more disk space? Drag a slider.
  • Easy to provision new clusters. A few mouse clicks or an API call and we can have a cluster with encryption, sharding, and backups. The GUI also shows you a direct cost estimation.
  • No nickel and diming. The folks at MongoDB could have easily added extra charges for each cluster modification, encryption, restores, alerts, and various other things to muddy the water, but they did not. You pay for the managed instances and backups you use and that’s it.
  • No lock-in. Migrating data in and out of Atlas is extremely simple.

After experimenting with the various features it became pretty obvious this would eliminate the vast majority of our overhead

With Atlas it now takes about 10 minutes to provision a full cluster with monitoring and any other features we need. We can even create a new group, which is sort of like a linked account, and give developers the ability to create their own clusters in an isolated environment without waiting on us.

This is Going to Hurt...Oh Wait...

So with all of this the only potential downside has to be the price, right? It turns out this was the most pleasant surprise of all.

Since our Atlas clusters are running on AWS and the cluster sizes are based on EC2 instances, we can make some rough comparisons on price:

  • M30 cluster w/ 8gb RAM and 100gb storage - ~$430/mo
  • 3 m4.large AWS instances w/ 8gb RAM and 100gb storage - ~265/mo

This does not include storage cost for backups or data transfer but gives us a good idea. If we’re running on an M30 instance in production, we don’t have to save many man hours to justify a ~$165/month difference. Another thing to note is that most people will over-provision their own clusters because it’s not easy to scale. With Atlas, we can eliminate this waste.

Going Forward

As we continue to build out our continuous deployment systems we keep looking for ways to dynamically provision environments and test for scale. Another thing Atlas offers in this respect is an API that encompasses most of their GUI features. This allows us to programmatically spin up a cluster as part of a test run:

We can also use a Docker container for this, but if we’re trying to replicate our production environment or test performance it can be quite difficult with containers.

The one thing missing from the API at the moment is access to metrics. It would be a killer feature if we could spin up a cluster, run some tests, then include production-like database performance stats in our test results. I know after speaking with the Atlas team, that this is coming in the next few months.

Conclusion

We have been using MongoDB Atlas in production for around 6 months and have been very happy with it. Our migration consisted of stopping MongoDB, running a mongodump, then simply restoring it with mongorestore using the new Atlas connection strings. We were able to schedule a maintenance window to accomplish this but there are also now options for doing a live migration if you require the uptime. Because of the simplicity and minimal effort involved in migrating the data, we were able to see cost savings in the first month. Clear, upfront cost estimates have been especially nice since I seem to spend a lot of my time in convoluted pricing calculators. Hopefully our experience inspires others to to give Atlas a try (which you can do without cost using the MongoDB Atlas free tier).

← Previous

How to Alert Your DevOps Team About Your MongoDB Database

Monitoring and alerting are key ingredients of the DevOps methodology put into practice. The systems that you run or the applications on them will generate a ton of telemetry data that you can use as actionable insight to inform your database monitoring and management strategy. Whether it's the number of connections currently being made to a database or the amount of RAM in use, this monitoring data can help us proactively prevent problems that can lead to poor customer experience MongoDB Atlas , the database as a service platform, provides you deep visibility into database and platform monitoring, with full integration into services relied upon by DevOps teams such as Slack or PagerDuty. You can ensure notifications are also available via email or SMS. Configuring proper monitoring and ensuring the correct members of your team are notified can keep your database performing at optimal levels, and increase uptime of your application. By monitoring your application for specific metrics that are key to performance, you can ensure you've properly scaled your managed database environment, and can re-scale at any time, so you minimize your costs accordingly. In this blog post I'll discuss the reasons why configuring alerts is important, and show you how to: Review the existing MongoDB Atlas default alerts Set up Slack as an outbound alert endpoint Configure your alerts Notify your team via SMS Why do we need Alerts for our DBaaS? Even when using a Database-as-a-Service (DBaaS) such as MongoDB Atlas, there is still a need to implement a monitoring policy. MongoDB Atlas comes with a number of default alerts to notify your team of specific triggered events; of course these alerts are also configurable. This is a great start for most teams, but there maybe times in which a custom alert for your environment is required. Proactive Maintenance Being proactive is important to ensure the success of an application once it's been deployed to a production environment. If our MongoDB database were to run short on disk space, we certainly need to be notified before our disk is full, to ensure uptime. By configuring proper thresholds to alert us when our disk is near full, we can proactively provision more disk capacity, without interruption to the application. If your database is saturated by connections, you may have errors that require intervention by your team, and the need to scale your cluster. These types of preventative measures can be configured for many specific conditions relative to disk performance, IOPS, CPU and available memory. By utilizing alerts that notify you before your database cluster has reached any performance limit, you can plan a strategy for scaling up and down elastically as usage dictates. Change notification If you are working in an environment with multiple administrators, there could be many people concurrently making changes to your application. This could require change notification alerts to inform your team and help avoid confusion about why modifications are occurring to your Atlas deployment. MongoDB Atlas makes it easy to configure a range of custom alerts that can be as granular as triggering when new users being added to a group. Performance Indication Custom alerts can be used to ensure you are getting the best possible performance from your MongoDB cluster. Some good examples of alerts that can help you maximize your database's performance include: Monitoring request latency per host Configure scanned objects alerts that notify you if queries are scanning more than a certain number of objects in a single query. This can assist in assuring appropriate fields are indexed If CPU, memory, or IOPS consumption is close to its limits Connections to your database are reaching a maximum threshold The great thing about MongoDB Atlas is when you get these alerts, you can easily and seamlessly do something about them. If you need more CPU or RAM, just upsize your instances, or add more of them; more or faster disks, scale the disks; taking a break for a while, drop your costs to a minimum by removing excess capacity. And all of these reconfigurations can be accomplished with no application downtime. All the cluster needs do is perform one a rolling replica set election, orchestrated by a couple of mouse clicks (or API calls) in the clean Atlas interface. Notifications Monitoring and alerts are no good to us without the actual notification reaching us on our mobile device if we are on-call. If I don't get a notice about a potential incident, how am I ever to fix it? MongoDB Atlas comes with some solutions to solving this problem by permitting you a number of ways to notify your team. The following are all outbound notification options for MongoDB Atlas: Email - to any standard email client SMS - Text message to alert your phone HipChat / Slack / Flowdock - chat notifications pushed via the MongoDB Atlas API directly to a specified channel PagerDuty - integration with on-call schedule and alerting Webhooks - Sends an HTTP POST request to an endpoint for programmatic processing We can use any or all of the above to ensure that our team is notified of changes or potential issues with our MongoDB Atlas cluster. You can find the configuration section for the service-based alerts in the "Settings" section of your MongoDB Atlas UI. You will need the API keys from your different service providers. Notify via Slack Slack has a direct integration with MongoDB Atlas that sends database alerts to the Slack channel of your choosing. To get started, make sure you have an existing Slack account with permission to add an API key. In this example, we’ve created a channel for our team called "#alerts" that will receive notifications of any alert status provided by MongoDB Atlas. Next, we will generate an API key to place in my Settings for my Slack integration. Go to the Slack API App Token generator , then click the "Create an App" button. Select the team, and then title the name of the app. In this case we have just used "mongodb-alerts" to make it easy to identify the purpose of the created token in the future. Now we will need to add permissions for this, so let's click the "Bot Users" button, give the bot the username @mongodbalerts, and then enable it: Once this is finished, we will click “Save changes”. Now let's finally "Install" the new app by clicking the "Install App" link. We'll be brought to a section that will ask us for final authorization; clicking “OK” will bring us to the API Token Section. We have obscured our full token, but you can see that we are provided with two different tokens. The token required for MongoDB Atlas is the "Bot User OAuth Access Token"; copy it and then open a new browser window. In the new browser tab, bring up the "Settings Section" and then scroll to the bottom of the page and find the "Slack Integration" tab. The "#alerts" channel is where the token and the channel for notifications are specified. After you click the "Post test message to Slack" you should see a notification into the #alerts channel: Create an alert MongoDB Atlas's comprehensive monitoring and alerting system is highly customizable for your team. You can get started with creating an alert by going to the Alerts section in your MongoDB Atlas UI. Here we see our Alert History, a section which lets us know what alerts were triggered in the past, and who acknowledged this alert. Next, go to the upper right hand corner and click the green "ADD" button. You'll now have a section permitting you to create an alert. If you wanted to create an alert for any time a host with the type “primary” has a connection number totaling over 500 per second, that is then pushed out to all members of our group via SMS and sends a notification to Slack, you'd follow the method as shown in this GIF: Click SAVE and you're all set. Your team will be SMS'd any time your Atlas database has over 500 concurrent connections. You can now customize your metrics for your database to ensure you're always properly notified. With the appropriate alerts now configured and our team's notification endpoints set, you'll be able to sleep soundly at night. By using correct monitoring settings and alerting appropriate team members, your application can remain resilient and provide a great experience for your users. Get started with MongoDB Atlas today, try our free tier by signing up now!

July 27, 2017
Next →

Staff Engineering at MongoDB: Your Path to Making Broad Impact

Andrew Whitaker is a Senior Staff Engineer at MongoDB. His previous experience spans tiny startups to enormous organizations like AWS, where he held several different roles focusing on databases. Before joining MongoDB, he worked at a startup building optimized machine learning models in the cloud. Read on to learn more about why Andrew decided to join MongoDB in a senior-level engineering role and how his work is driving improvement within our engineering organization. Why MongoDB I have long been a fan of MongoDB’s products and services. MongoDB the database has always been a pleasure to work with – the system “brings joy” to quote a phrase. As a Python developer, I appreciate how the Python driver feels “Pythonic” in a completely natural way. The programmer interacts with the database using Python constructs: dictionaries, lists, and primitive types. By contrast, SQL databases force me to change my mental model, and the query language feels like an add-on that does not blend with the core language. As an engineer, I am always looking to expand my knowledge and grow my skills. The scope of challenges engineers face at MongoDB is what triggered my interest in the company. We obviously have people working on core databases and distributed systems. But, we also have teams dedicated to machine learning, streaming data, analytics, networking, developer tooling, drivers, and many more areas. It is very hard to get bored working at MongoDB. Finally, I would be remiss if I did not mention the people. Overall, MongoDB’s engineering culture prioritizes intelligence, low ego, and an ability to get stuff done. CL/CI (Continuous Learning, Continuous Improvement) Working at MongoDB has provided me with opportunities for continued learning and growth. Though I do not program as much as I did earlier in my career, I have recently been exploring the Rust language. I’m excited by Rust because it avoids the tradeoffs between predictable performance and safety. My work in the search space has given me exposure to the fast moving world of AI: vector embeddings, RAG, etc. For various reasons, I think MongoDB is uniquely positioned to do well in this area. On top of this, I’m working on some initiatives that are not fully public. I can say that one focus area is improving the sharding experience for our customers. We believe MongoDB sharding is best-in-breed. Still, the process requires more manual configuration than we think is ideal: customers select the shard key, cluster type, shard count, etc. We give guidance here, but I think we can raise the bar in terms of offering a seamless experience with less “futz”. I’m also working with the search team. We believe there is a natural affinity between MongoDB’s document model and AI/ML workloads. We have some features in the works that extend this integration in new and interesting ways. I also spend a fair bit of time driving quality improvements across our suite of products. Our CTO Jim Scharf frequently refers to our “ big 4 ” goals: security, durability, availability, and performance. These goals are more important than any feature we build. I’ve been working across the company to help teams define their availability SLO/SLAs. It turns out that measuring availability is a subtle topic. For example, a naive approach of counting the percentage of failed requests can underestimate downtime because customers make fewer requests when a service is unavailable. So, the first step is to clarify the definition of availability. Finally, as a lapsed academic (in a distant life, I was a graduate student at the University of Washington Department of Computer Science and Engineering), I’m always interested in finding ways to bridge theory and practice. I’ve been collaborating with some folks in our research team to drive improvements to our replication protocols. There are theoretical results that suggest it is impossible to simultaneously achieve low latency and strong consistency (“linearizability” in the technical jargon). However, we believe there are intermediate points in the consistency/latency spectrum that have not been fully explored. This work hasn't been made into a product yet, but stay tuned. Flexible working MongoDB is a hybrid company. Like many of our engineers, I work outside the company headquarters in New York City (I live in Seattle). I appreciate MongoDB’s approach to hybrid working and that company leadership, starting with Dev , cares about the well-being of their employees. It seems there are companies that don’t seem to trust their employees to make decisions, such as which days to come into the office, so I’m thankful for the autonomy I receive at MongoDB to work in a way that’s best for me. Remote work has its challenges, but I would say that the benefit for my work/life balance has been transformative. Final thoughts I have found MongoDB engineers demonstrate a strong mix of technical depth, pragmatism, and empathy. I have yet to find the “smart jerk” prototype that seems to exist throughout the tech industry. Overall, I have found MongoDB is open to change and growth at both the team level and the individual level. There is a willingness to evolve and improve that aligns with the company’s values and leadership principles and enables the success of our technology and people. Find out more about MongoDB culture and career opportunities by joining our talent community .

November 20, 2024