Ways to Integrate MongoDB Atlas in Your DevOps Processes

Darshana Paithankar
October 11, 2023 | Updated: January 11, 2024
#DevOps

MongoDB Atlas - the industry-leading developer data platform integrates all of the data services you need to build modern applications in a unified developer experience. We want to meet you where you are and offer various ways to begin with Atlas and make the most of its features. Starting with the Atlas User Interface is a good initial step. Now what if your requirement is to automate the deployment of Atlas clusters at scale, and by leveraging tools that are already integral to your application ecosystem? Through this blog, we will address this question.

We will discuss the programmatic methods for starting with Atlas and deploying Atlas resources and infrastructure according to your specific needs. The tools we provide cover most of the control plane management ( easily deploy, manage, scale) Atlas clusters. Thereby, enabling you to create the building blocks of Atlas such as clusters, database users, projects, Atlas Search indexes, backups, alerts, and more. You can interact directly with the data plane through the command line or programmatically using tools like MongoDB Shell, Compass, language drivers, etc., to work with data and perform CRUD operations.

1. Atlas Administration API: Leveraging your choice of client

No matter what your choice of client: be it cURL commands, Postman, Insomnia etc. You can use a client of your choice to directly interact with Atlas through the Atlas Administration API. The Atlas Administration API gives you a RESTful interface to interact with Atlas resources and perform various actions within MongoDB Atlas. Each endpoint represents a specific resource in Atlas ( e.g. cluster). You can programmatically deploy and manage all of the Atlas resources from an administrative standpoint such as creating a cluster, database user, projects, advanced clusters, backups, monitoring etc.

All of the tools that we will cover below are built on top of the Atlas Administration API and abstract away the complexities of using the Atlas Atlas Administration API directly.

2. GoSDK Client: A simplified way to get started with the Atlas Admin API

One of the mechanisms that simplify the interaction with the API is the availability of SDK (Software Development Kit). If you are a Go developer, the GoSDK client gives you a much simpler experience of getting started with the Atlas Administration API. It also has full endpoint coverage of the Administration API and improves the speed of getting started with the Atlas Administration API. Getting started with the Admin API through the Go SDK client only takes a few lines of code since the SDK includes pre-built functions, structs, and methods that encapsulate the complexity of HTTP requests, authentication, error handling, versioning, and other low-level details. We will be adding SDK support for more languages in the future. Feel free to add feedback on what other languages you would like to see supported.

3. MongoDB Atlas CLI: Simple command-line tool to easily deploy Atlas resources

If you prefer to manage your Atlas resources using simple commands in the terminal, the MongoDB Atlas Command-Line Interface (CLI) is your answer. With the Atlas CLI, you can seamlessly manage your clusters, automate database user creation, control network access, and perform various other administrative tasks, all from the command line. You can also script these actions out with the CLI for even easier repeatability. It is available for multiple operating systems, including Windows, Linux, and macOS. It is often used in conjunction with other command-line tools to automate workflows and integrate with CI/CD pipelines. An easy way to get started with the Atlas CLI is through the quickstart.

4. Infrastructure as Code (IaC) integrations: Automating deployment of Atlas using Infrastructure as Code tools

There are several advantages of using Infrastructure as Code ( IaC) tools to provision application infrastructure. By treating your infrastructure code similar to your application code, IaC tools offer benefits such as version control, scalability, security, repeatability etc. What if you could easily deploy your Atlas resources using your preferred IaC tools? MongoDB Atlas gives you that flexibility. Whether you're an AWS CloudFormation or a HashiCorp Terraform enthusiast, you can provision and manage Atlas resources using these tools with ease through the integrations we offer:

AWS CloudFormation integration: MongoDB Atlas supports three ways to provision resources using AWS CloudFormation

  • The first is by leveraging Atlas resources directly from the CloudFormation Public Registry. We have 33+ resources available today. The configurations are defined in JSON/ YAML and can be executed using the AWS CLI/ AWS management console.
  • If you want a faster way to get started, you can explore our AWS Partner Solutions ( formerly known as quickstarts), which have pre-built Cloudformation templates to help you provision a group of Atlas resources for specific use cases instead of deploying them one by one.
  • And if you are passionate about using languages you are comfortable with such as JavaScript/TypeScript, Python, Java, C#, and Go instead of learning YAML/JSON, you can leverage the AWS Cloud Development Kit (CDK) to deploy Atlas resources. Under the hood, when your AWS CDK applications are run, they translate your code to CloudFormation templates which use the CloudFormation service for provisioning.

HashiCorp Terraform integration: If you are already using Terraform as your IaC tool of choice, we have integrations with Hashicorp Terraform as well. There are two easy ways to get started with Hashicorp Terraform

  • By directly provisioning Atlas resources on AWS, Azure, and Google Cloud using the Terraform MongoDB Atlas provider.
  • If you prefer to use your favorite language, e.g. Typescript, Python, C#, Java, Go, you can use CDKTF (Cloud Development Kit for Terraform), which will allow you to deploy Atlas using Terraform under the hood without knowing the specifics of Terraform’s configuration language

5. Atlas Kubernetes Operator: Use your existing Kubernetes tooling to manage Atlas resources

For organizations leveraging Kubernetes for container orchestration, the Atlas Kubernetes Operator provides seamless integration between Kubernetes and MongoDB Atlas. This operator allows you to deploy and manage Atlas resources using your existing Kubernetes tooling, streamlining the process of spinning up and scaling databases alongside your applications. You can manage Atlas in exactly the same way you manage your applications running in Kubernetes. This is done by managing Atlas directly via custom resources in Kubernetes. These custom resources can be created, managed, and stored as the source of truth in your repository. You can then leverage a Continuous Deployment tool of your choice such as ArgoCD to apply them into Kubernetes.

Whether you prefer working with the command line, a RESTful API, Kubernetes, or IaC tools, MongoDB Atlas provides a diverse set of tools to help you achieve your automation goals. By embracing these methods, you can streamline your operations, improve efficiency, and pave the way for a more agile and responsive development process.

Learn more from our Atlas Programmatic Access documentation page

← Previous

Boost the Accuracy of E-commerce Search Results with Atlas Vector Search

Artificial Intelligence’s (AI) growth has led to transformative advancements in the retail industry, including natural language processing, image recognition, and data analysis. These capabilities are pivotal to enhancing the efficiency and accuracy of e-commerce search results. E-commerce, characterized by its vast product catalogs and diverse customer base, generates enormous amounts of data every day. From user preferences and search histories to product reviews and purchase patterns — and add to that images, video, and audio associated with product campaigns and user search — the data is both a goldmine and a challenge. Traditional search mechanisms, which rely on exact keyword matches, are inadequate at handling such nuanced and voluminous data. This is where vector search comes into play as the perfect data mining tool . As a sophisticated search mechanism, it leverages AI-driven algorithms to understand the intrinsic relationships between data points. This enables it to discern complex patterns, similarities, and contexts that conventional keyword-based searches might overlook. Let’s dig deeper into the differences between traditional keyword matching search and vector search, and answer questions like: What type of queries does vector search improve in the retail search landscape? What are the challenges associated with it? And how can your business tap into the competitive advantage it represents? Check out our AI resource page to learn more about building AI-powered apps with MongoDB. Traditional Keyword Matching vs. Vector Search Traditional search functionalities for e-commerce platforms — keyword matching, typo tolerance, autocomplete, highlighting, facets, and scoring — are often built in-house or implemented on top of typical search engines like Apache Lucene, AtlasSearch, or ElasticSearch, relying heavily on metadata textual descriptions. While this has served the industry well for years, it often falls short of understanding the nuanced needs of modern consumers. For instance, a customer might be looking for a "blue floral summer dress," but if the product description lacks these terms, it might not appear in the search results, even if it perfectly matches the visual description. Figure 1: As embeddings encode numerically the meaning of documents, semantically close documents will be geometrically close as well. Vector search is a method that finds similar items in a dataset based on their vector representations, and offers a more efficient and accurate way to sift through large datasets. Instead of relying on exact matches, it uses mathematical techniques to measure the similarity between vectors, allowing it to retrieve items that are semantically similar to the user's query, even if the query and the item descriptions don't contain exact keyword matches. Figure 2: Data flow diagram showcasing how applications, vector embedding algorithms, and search engines work together at a high level. One great thing about Vector search is that by encoding any type of data, i.e. text, images or sound, you can perform queries on top of that, creating a much more comprehensive way of improving the relevance of your search results. Let’s explore examples of queries that involve context, intent, and similarity. Visual similarity queries Query: "Find lipsticks in shades similar to this coral lipstick." Vector Search Benefit: Vector search can recognize the color tone and undertones of the specified lipstick and suggest similar shades from the same or different brands. Data type: image or text Contextual queries Query: "Affordable running shoes for beginners." Vector Search Benefit: Vector search can consider both the price range and the context of "beginners," leading to relevant shoe suggestions tailored to the user's experience level and budget. Data type: text, audio (voice) Natural language queries Query: "Show me wireless noise-canceling headphones under $100." Vector Search Benefit: Capture intent. Vector search can parse the query's intent to filter headphones with specific features (wireless, noise-canceling) and a price constraint, offering products that precisely match the request. Data type: text, audio (voice) Complementary product queries Query: "Match this dress with elegant heels and a clutch." Vector Search Benefit: Vector search can comprehend the user's request to create a coordinated outfit by suggesting shoes and accessories that complement the selected dress. Data type: text, audio (voice), image Challenging landscape, flexible stack Now that we've explored different queries and their associated data types that could be used in vector embeddings for search, we can see how much more information can be used to deliver more accurate results and fuel growth. Let’s consider some of the challenges associated with a vector search solution data workflow and how MongoDB Atlas Vector Search helps bridge the gap between challenges and opportunities. Data overload The sheer volume of products and user-generated data can be overwhelming, making it challenging to offer relevant search results. By embedding different types of data inputs like images, audio (voice), and text queries for later use with vector search, we can simplify this workload. Storing your vector encoding in the same shared operational data layer your applications are built on top of, but also generating search indexes based on those vectors, makes it simple to add context to your application search functionalities. Using Atlas Vector Search combined with MongoDB App Services , you can reduce operational overhead by creating a trigger that could “see” when a new document is created in your collections and automatically make the call to the embedding API of your preference, pushing the document to it and storing the retrieved embedding data in the same document stored in your collection. Figure 3: Storing vectors with the data simplifies the overall architecture of your application. As the number of documents or vectors grows, efficient indexing structures ensure that search performance remains reasonable. By simply creating an index based on the embedded data field, you can leverage the optimized retrieval of the data, reduce the computational load, and accelerate its performance, especially for nearest neighbor search tasks, where the goal is to find items that are most similar to a given query. Altogether, the combination of MongoDB Vector Search capabilities with App Services and indexing provides a robust and scalable solution to achieve real-time responsiveness. An indexed vector search database can provide rapid query results, making it suitable for applications like recommendation engines or live search interfaces. Changing consumer behavior Developing an effective vector search solution involves understanding the nuances of the retail domain. Retailers must consider factors like seasonality, trends, and user behavior to improve the accuracy of search results. To overcome this challenge, retailers will need to be able to adjust their business model by categorizing their product catalogs and user data according to different criteria, for example: So as you can see all this vast amount of information can be embedded to build more comprehensive criteria for relevance, but first it needs to be properly captured and organized. This is where the value of the flexible document model comes into play. The document model allows you to define different fields and attributes for each category of data. This can be used to capture the various categorization criteria. Retailers could also utilize embedded subdocuments to associate relevant information with products or customers. For instance, you can embed a subdocument containing marketing campaign data, engagement channels, and geographic location within products to track their performance. As categorization criteria evolve, dynamic schema evolution allows you to add or modify fields without disrupting existing data. This flexibility easily accommodates changing business needs. Retailers may also use embedded arrays to record purchase history for customers. Each array element can represent a transaction, including product details and purchase date, facilitating segmentation based on recency and frequency. By embedding all these different data types, and leveraging the flexible capabilities of the document model, retailers can create a comprehensive and dynamic system that effectively categorizes data according to diverse criteria in a fast and resilient way. This enables personalized search experiences and enhanced customer engagement in the e-commerce space. Sitting on a goldmine Every retailer worldwide now realizes that with their customer data, they are sitting on a goldmine. Using the proper enabling technologies would allow them to build better experiences for their customers while infusing their applications with automated, data-driven decision-making. Retailers offering more intuitive and contextual search results can ensure their customers find what they're looking for by personalizing the relevance of their search results, enhancing satisfaction, and increasing the likelihood of successful transactions. The future of e-commerce search lies in harnessing the power of technologies like Atlas Vector Search , as it’s not only another vector search database, but also an extended product for the developer data platform , providing them with an integrated set of data and application services. For retailers, the message is clear: to offer unparalleled shopping experiences, embracing and integrating vector search functionalities with a performant and reliant platform that simplifies your data organization and storage is not just beneficial, it's essential. Learn more and discover How to Implement Databricks Workflows and Atlas Vector Search for Enhanced E-commerce Search Accuracy with our developer guide, and check out our GitHub repository explaining the full code for deploying an AI-Enhanced e-commerce search solution

October 11, 2023
Next →

Nokia Corteca Scales Wi-Fi Connectivity to Millions of Devices With MongoDB Atlas

Nokia’s home Wi-Fi connectivity cloud platform was launched in 2019 as the Nokia WiFi Cloud Controller (NWCC). In 2023, it was renamed and relaunched as the Corteca Home Controller, becoming part of the Corteca software suite that delivers smarter broadband for a better experience. The Corteca Home Controller can be hosted on Amazon Web Services, Google Cloud, or Microsoft Azure, and is the industry’s first platform to support three management services—device management, Wi-Fi management, and application management. Supporting TR-369 (a standardized remote device management protocol) also allows the Home Controller to work in a multi-vendor environment, managing both Nokia broadband devices and third-party broadband devices. By solving connectivity issues before the end-user detects them, and by automatically optimizing Wi-Fi performance, the Home Controller helps deliver excellent customer experiences to millions of users, 24/7. During the five years that Nokia Corteca has been a MongoDB Atlas customer, the Home Controller has successfully scaled from 500,000 devices to over 4.5 million. There are now 75 telecommunications customers of Home Controller spread across all regions of the globe. Having the stability, efficiency, and performance to scale Nokia Corteca's solution is end-to-end, from applications embedded in the device, through the home, and into the cloud. Algorithms assess data extracted from home networks, based on which performance parameters automatically adjust as needed—changing Wi-Fi channels to avoid network interference, for example—thereby ensuring zero downtime. The Home Controller processes real-time data sent from millions of devices, generating massive volumes of data. With a cloud optimization team tasked with deploying the solution across the globe to ever more customers, the Home Controller needed to store and manage its vast dataset and to onboard new telecommunication organizations more easily without incurring any downtime. Prior to Nokia Corteca moving to MongoDB Atlas, its legacy relational database lacked stability and required both admin and application teams to manage operations. A flexible model with time series capabilities That's where MongoDB Atlas came in. Nokia was familiar with the MongoDB Atlas database platform, having already worked with it as part of a previous company acquisition and solution integration. As Nokia's development team had direct experience with the scalability, manageability, and ease of use offered by MongoDB Atlas, they knew it had the potential to address the Home Controller’s technical and business requirements. There was another key element: Nokia wanted to store time-series data—a sequence of data points in which insights are gained by analyzing changes over time. MongoDB Atlas has the unique ability to store operational and time series data in parallel and provides robust querying capabilities on that data. Other advantages include MongoDB's flexible schema, which helps developers store data to match the application's needs and adapt as data changes over time. MongoDB Atlas also provides features such as Performance Advisor that monitors the performance of the database and makes intelligent recommendations to optimize and improve the performance and resource consumption Fast real time data browsing and scalability made easy Previously, scaling the database had been time-consuming and manual. With MongoDB Atlas, the team can easily scale up as demand increases with very little effort and no downtime. This also means it is much more straightforward to add new clients, such as large telecommunications companies. Having started with 100GB of data, the team now has more than 1.3 terabytes, and can increase the disc space in a fraction of a second, positioning the team to be able to scale with the business. As the Home Controller grows and onboards more telcos, the team anticipates a strengthening relationship with MongoDB. “We have a very good relationship with the MongoDB team,” said Jaisankar Gunasekaran, Head of Cloud Hosting and Operations at Nokia. “One of the main advantages is their local presence—they’re accessible, they’re friendly, and they’re experts. It makes our lives easier and lets us concentrate on our products and solutions.” To learn more about how MongoDB can help drive innovation and capture customer imaginations, check out our MongoDB for Telecommunications page.

July 2, 2024