How to Deploy Vector Search, Atlas Search, and Search Nodes With the Atlas Kubernetes Operator
Rutuja Rajwade, Tim Kelly10 min read • Published Dec 20, 2024 • Updated Dec 20, 2024
FULL APPLICATION
Rate this tutorial
The Atlas Kubernetes Operator enables Kubernetes users to manage Atlas resources using the same tools and processes they already employ for their existing services across any Kubernetes offering, such as AWS EKS or Red Hat OpenShift. It works by extending the Kubernetes API through custom resources (CR), enabling developers to manage MongoDB resources as native Kubernetes objects. Developers can apply custom resource definition configuration files for Atlas building blocks into the Kubernetes cluster that the Atlas Operator is installed into, and the Atlas Kubernetes Operator automates the application of that configuration via the Atlas Admin API.
Now, teams whose RAG applications are underpinned with Kubernetes, can take advantage of custom resources for Atlas Vector Search, Atlas Search, and Search Nodes.
Atlas Vector Search unlocks the full potential of your data, no matter whether it’s structured or unstructured, taking advantage of the rise in popularity and usage of AI and LLMs to solve critical business challenges. This is possible due to Vector Search being part of the MongoDB Atlas developer data platform, which starts with our flexible document data model and unified API providing one consistent experience.
Atlas Search adds powerful, relevant, full-text search of your data. A simple Atlas Search index on your collection will make all string fields searchable and can provide drastic user experience improvements in finding relevant content quickly.
Both Atlas Vector Search and Atlas Search provide Search Nodes dedicated architecture, enabling better optimization for the right level of resourcing for specific workload needs. Search Nodes provide dedicated infrastructure for both Atlas Search and Atlas Vector Search workloads, allowing you to optimize compute resources and fully scale search needs independent of the database. Search Nodes provide better performance at scale, delivering workload isolation and higher availability. In some cases, we’ve seen 60% faster query time for some users' workloads, leveraging concurrent querying in Search Nodes.
In this tutorial, we’ll go over how to deploy Atlas Search, and Vector Search resources using the MongoDB sample datasets and the Atlas Kubernetes Operator.
Before starting, ensure we have the following set up and ready:
- A MongoDB Atlas account
- A running Kubernetes cluster with nodes running processors with the x86-64, AMD64, or ARM64 architecture
- API keys associated with your organization to configure Atlas Kubernetes Operator access to Atlas
First, we will need to install the operator into a running Kubernetes cluster. Make sure that it’s running processors with the x86-64, AMD64, or ARM64 architecture. In this example, we are running a Kubernetes cluster using Minikube and Docker, and using Kubectl to apply commands.
Once we have logged into our Atlas account, we will first create API keys for use by the operator.
In the top left-hand corner, click on the “Access Manager” drop-down and the API keys tab. Click on “Create API Key.”
Note: If you want the Atlas Kubernetes Operator to create a new Atlas project to work in, you will need to create and use the API keys for an organization. For an existing project, you will need to create them for a project. For this example, we’ll be creating a new Atlas project. For either type of project, it is best practice to configure an API access list.
Here, we will input a name for our API keys (for example, AKO/Vector Search Keys) and select the organization role. We must assign the API key the Organization Project Creator organization role or higher.
From there, we will find a set of unique API keys generated for you. Be sure to save these somewhere secure. Once we leave this page, we will not be able to recover them!
Next, we will install/deploy the operator into the Kubernetes cluster we have running.
Once deployed, the Atlas Kubernetes Operator can monitor resources and events across a single or multiple Kubernetes namespaces. This capability allows the operator to manage MongoDB Atlas custom resources comprehensively across the entire Kubernetes cluster or specific designated namespaces.
For this example, we’ll set up the operator to watch all namespaces in our cluster, but you can learn how to set up the operator to watch a single namespace in our Atlas Kubernetes Operator documentation.
We’ll use the following command in our terminal:
1 kubectl apply -f https://raw.githubusercontent.com/mongodb/mongodb-atlas-kubernetes/v2.3.1/deploy/all-in-one.yaml
Make sure to replace the
<version> with the latest update in the following format: v2.x.x (eg. v2.3.1).Verify the installation:
1 kubectl get pods -n mongodb-atlas-system
Next, we’ll create a secret in Kubernetes. A Kubernetes secret is an object used to store and manage sensitive information, such as passwords, OAuth tokens, and SSH keys, securely within a Kubernetes cluster. In this case, it will be the API keys you created earlier.
If you use external secret storage, you don't need to put sensitive information directly into Kubernetes secrets. To learn more, see Configure Secret Storage.
Input the following command and replace
<atlas_organization_id> with your organization ID, <atlas_api_public_key> with your public API key, and <atlas_api_private_key> with your private key.Troubleshooting: If the following command produces an error, try to run the command without quotes (e.g.,
–”from-literal=orgId=<atlas_organization_id>” \)1 kubectl create secret generic mongodb-atlas-operator-api-key \ 2 --from-literal="orgId=<atlas_organization_id>" \ 3 --from-literal="publicApiKey=<atlas_api_public_key>" \ 4 --from-literal="privateApiKey=<atlas_api_private_key>" \ 5 -n mongodb-atlas-system
Next, we use the following command to label our secret:
1 kubectl label secret mongodb-atlas-operator-api-key atlas.mongodb.com/type=credentials -n mongodb-atlas-system
Projects allow us to isolate different database environments (for instance, development/qa/prod environments) from each other, as well as users/teams. Here, we’ll use the operator to create a project in the Atlas UI through an Atlas project custom resource.
Next, we input the following command in our terminal.
1 cat <<EOF | kubectl apply -f - 2 apiVersion: atlas.mongodb.com/v1 3 kind: AtlasProject 4 metadata: 5 name: my-project 6 spec: 7 name: Test Atlas Operator Project 8 projectIpAccessList: 9 - cidrBlock: "0.0.0.0/0" 10 comment: "Allowing access to database from everywhere (only for Demo!)" 11 EOF
Note: The above commands will add the CIDR 0.0.0.0/0 to your IP access list if input as is. This CIDR allows any client to connect to the Atlas cluster. Do not use this IP address in production.
If you are working off an existing project in Atlas, be sure to change
name: my-project to the name of your project in Atlas, and update the cidrBlock to your current IP address.After entering this command, we can check our Atlas UI. We should see a project created in the Atlas UI called “Test Atlas Operator Project.”
Deployments are our instances of MongoDB running on a cloud provider. Here, we’ll set up our deployments and learn how to enable Atlas Search and Atlas Vector Search.
Text search allows users to quickly find relevant information by entering just a few keywords, and allowing users to scan large bodies of data. When a query is entered, the text search engine scours through our indexed documents to locate and return documents most relevant to our query, based on the keywords. Atlas Search is a good solution for queries requiring exact, or near exact, matches where the overarching semantic meaning isn't as critical. Think online shopping, searching for men's large pajamas.
Enabling this feature requires an Atlas deployment. Below is the updated
AtlasDeployment configuration to enable Atlas Search.Here’s the base deployment configuration:
1 apiVersion: atlas.mongodb.com/v1 2 kind: AtlasDeployment 3 metadata: 4 name: my-atlas-cluster 5 namespace: mongodb-atlas-system 6 spec: 7 deploymentSpec: 8 backupEnabled: true 9 clusterType: REPLICASET 10 name: Test-cluster-M10 11 replicationSpecs: 12 - regionConfigs: 13 - backingProviderName: AWS 14 electableSpecs: 15 instanceSize: M10 16 nodeCount: 3 17 providerName: AWS 18 regionName: US_EAST_1 19 priority: 7 20 terminationProtectionEnabled: false 21 projectRef: 22 name: my-project 23 namespace: mongodb-atlas-system
To enable Atlas Search, add the following
searchIndexes field to the AtlasDeployment:1 apiVersion: atlas.mongodb.com/v1 2 kind: AtlasDeployment 3 metadata: 4 name: my-atlas-cluster 5 namespace: mongodb-atlas-system 6 spec: 7 deploymentSpec: 8 backupEnabled: true 9 clusterType: REPLICASET 10 name: Test-cluster-M10 11 replicationSpecs: 12 - regionConfigs: 13 - backingProviderName: AWS 14 electableSpecs: 15 instanceSize: M10 16 nodeCount: 3 17 providerName: AWS 18 regionName: US_EAST_1 19 priority: 7 20 searchIndexes: 21 - DBName: sample_airbnb 22 collectionName: listingsAndReviews 23 name: my-index 24 search: 25 mappings: 26 dynamic: true 27 searchConfigurationRef: 28 name: atlassearchindexconfig-sample 29 namespace: mongodb-atlas-system 30 type: search 31 terminationProtectionEnabled: false 32 projectRef: 33 name: my-project 34 namespace: mongodb-atlas-system
DBName: Specifies the name of the database (e.g.,sample_airbnb)collectionName: Specifies the collection for which the index is created (e.g.,listingsAndReviews)type: Specifies the type of index, which in this case issearchsearchConfigurationRef: References anAtlasSearchIndexConfigfor reusable configurations
Save the configuration in a YAML file (e.g.,
atlas-search-config.yaml) and apply it using the following command:1 kubectl apply -f atlas-search-config.yaml
1 apiVersion: atlas.mongodb.com/v1 2 kind: AtlasSearchIndexConfig 3 metadata: 4 name: atlassearchindexconfig-sample 5 namespace: mongodb-atlas-system 6 spec: 7 analyzer: lucene.standard 8 searchAnalyzer: lucene.standard
This configuration can be shared across multiple deployments, reducing duplication. For more information, visit MongoDB Atlas Search documentation.
Atlas Vector Search is like using a compass in a sea of data, guiding you to what’s most semantically relevant rather than just matching words. Traditional search engines rely on keywords, hunting for exact terms or patterns in text. Vector search adds a whole new perspective to how you find data you are looking for—it's all about meaning.
Imagine every piece of unstructured data (songs, documents, images, etc.) as a single point in a multi-dimensional space, mapped based on its underlying meaning or context. These points, or vector embeddings, are numeric representations of data learned by machine learning models. A query gets transformed into a similar dot, and Vector Search finds the closest matches in this space, even if the words don’t directly match.
It’s the tech behind how Spotify recommends songs that share a similar emotional tone to your favorite playlist, how e-commerce platforms suggest products that align with your browsing history, and how image search engines find visually similar photos based on an uploaded picture. With tools like MongoDB Atlas Vector Search, it’s now easier to unlock the full potential of unstructured data and deliver value via real-world apps.
Like Atlas Search, enabling Atlas Vector Search requires an update to the
AtlasDeployment configuration. Below is an example deployment with Vector Search enabled:1 apiVersion: atlas.mongodb.com/v1 2 kind: AtlasDeployment 3 metadata: 4 name: my-atlas-cluster 5 namespace: mongodb-atlas-system 6 spec: 7 deploymentSpec: 8 backupEnabled: true 9 clusterType: REPLICASET 10 name: Test-cluster-M10 11 replicationSpecs: 12 - regionConfigs: 13 - backingProviderName: AWS 14 electableSpecs: 15 instanceSize: M10 16 nodeCount: 3 17 providerName: AWS 18 regionName: US_EAST_1 19 priority: 7 20 searchIndexes: 21 - DBName: sample_airbnb 22 collectionName: listingsAndReviews 23 name: vector-index 24 vectorSearch: 25 fields: | 26 [ 27 {"description": {"type": "vector"}} 28 ] 29 type: vectorSearch 30 terminationProtectionEnabled: false 31 projectRef: 32 name: my-project 33 namespace: mongodb-atlas-system
DBName: Name of the database (e.g.,sample_airbnb)collectionName: Name of the collection for the vector index (e.g.,listingsAndReviews)type: Specifies the type of index, set tovectorSearchfields: Defines the fields to be indexed as vectors. In this example, thedescriptionfield in thesample_airbnbdataset is indexed
Save this configuration in a YAML file (e.g.,
atlas-vector-search-config.yaml) and apply it with:1 kubectl apply -f atlas-vector-search-config.yaml
Important Note: Unlike Atlas Search, Atlas Vector Search does not require a separate
AtlasSearchIndexConfig resource because the configurations are specific to each deployment and not reusable across multiple deployments. For more information, see the MongoDB Atlas Vector Search documentation.With these configurations, your Atlas deployment supports both Atlas Search and Atlas Vector Search. In the Atlas UI, verify the indexes under the Indexes tab in your collection. This unified platform lets you store and search operational, metadata, and vector data seamlessly, eliminating the need for multiple systems.
Atlas Search Nodes are purpose-built resources in MongoDB Atlas for dedicated infrastructure for Atlas Search and Vector Search workloads.
On M10 clusters or higher, on Atlas clusters running MongoDB v6.0 or higher, we can configure Search Nodes separately from database nodes.
- Dedicated power for search:
Search Nodes aren’t part of your core database cluster—they’re a separate layer optimized specifically for search queries. This means no more competition between search and your transactional workloads for CPU or memory. - Independent scaling:
As your application grows, you can scale Search Nodes independently of your database. Got more users hammering your search bar? Just add more nodes without touching the rest of your cluster.
- Faster queries:
By offloading search to specialized nodes, you get faster response times, even for complex queries, whether it’s keyword search or vector similarity. - Scale independently:
Heavy search usage won’t bog down your database. Need more search power? Scale just the Search Nodes while your database runs uninterrupted. - No resource clashes:
With search and database operations handled separately, resource contention is a thing of the past. Your database keeps humming along, even during high search query traffic. - Optimized costs:
Only pay for the search capacity you need. Instead of over-provisioning database nodes to handle search, you can scale Search Nodes independently and save money. - Improved developer experience:
Use a single, unified API across database and search operations. Create a search index with a few clicks or a single API call.
- High-volume search:
If your app relies on constant, high-speed searches—think e-commerce, media streaming, or social platforms - AI features:
When you’re running Atlas Vector Search for semantic queries, like recommending products or powering chatbots with contextual understanding - Workload isolation:
When you want to guarantee search doesn’t interfere with the database’s core operations, like writes and analytics.
To add Search Nodes, include the
searchNodes field in the AtlasDeployment configuration:1 apiVersion: atlas.mongodb.com/v1 2 kind: AtlasDeployment 3 metadata: 4 name: my-atlas-cluster 5 namespace: mongodb-atlas-system 6 spec: 7 deploymentSpec: 8 backupEnabled: true 9 clusterType: REPLICASET 10 name: Test-cluster-M10 11 replicationSpecs: 12 - regionConfigs: 13 - backingProviderName: AWS 14 electableSpecs: 15 instanceSize: M10 16 nodeCount: 3 17 additionalNodeSpecs: 18 searchNodes: 19 - instanceSize: M10 20 nodeCount: 2 21 providerName: AWS 22 regionName: US_EAST_1 23 priority: 7 24 terminationProtectionEnabled: false 25 projectRef: 26 name: my-project 27 namespace: mongodb-atlas-system
searchNodes: Specify the size (instanceSize) and number (nodeCount) of dedicated Search Nodes.- Independent scaling: You can scale the Search Nodes independently of the primary database nodes.
Save the updated configuration in a YAML file (e.g.,
atlas-search-nodes-config.yaml) and apply it using:1 kubectl apply -f atlas-search-nodes-config.yaml
This ensures our MongoDB Atlas deployment includes dedicated Search Nodes, optimizing search query performance while isolating search workloads from your main database operations.
Now, if you want to learn more about how to create database users and how to get a connection string for your app, we cover all these steps in How to Deploy MongoDB Atlas With the Atlas Kubernetes Operator.
In this tutorial, you’ve seen how to deploy MongoDB Atlas Search, Vector Search, and Search Nodes using the Atlas Kubernetes Operator. These tools make it easy to manage advanced search capabilities directly within your database, whether you’re handling full-text queries, powering recommendations, or building applications that require semantic understanding.
By isolating search workloads with Search Nodes, you can keep your database performing smoothly while scaling search independently. With the Atlas Kubernetes Operator, you can bring MongoDB into your existing workflows, keeping everything consistent and manageable across your infrastructure.
Now, your setup is ready to handle scalable, high-performance search workloads. Whether it’s supporting real-time applications or complex queries, MongoDB Atlas has you covered.
If you'd like to learn more, head over to Developer Center where you can learn MongoDB Orchestration With Spring & Atlas Kubernetes Operator, or head over to MongoDB Community Forums to ask us questions or see what other people are doing with MongoDB.
Top Comments in Forums
There are no comments on this article yet.
Rate this tutorial