Rutuja Rajwade

Product Marketing Manager

MongoDB

Tim Kelly

Developer Advocate

Learn how to deploy MongoDB Atlas Vector Search, Atlas Search, and Search Nodes using the Atlas Kubernetes Operator. This tutorial covers step-by-step instructions to integrate advanced search capabilities into Kubernetes clusters, enabling scalable, high-performance workloads with MongoDB Atlas.

How to Deploy Vector Search, Atlas Search, and Search Nodes With the Atlas Kubernetes Operator

Kubernetes, a container orchestration tool, enables developers to automate the deployment, scaling, and management of their containerized applications. Kubernetes is popular among developers for its security benefits and the fact that it comes in many different flavors, including AWS' EKS, Microsoft Azure's AKS, and Google Cloud’s GKE. MongoDB Atlas enables developers to seamlessly integrate Atlas into their current Kubernetes deployment pipeline through our MongoDB Atlas Kubernetes Operator. 

The Atlas Kubernetes Operator enables customers using Kubernetes to create and manage Atlas resources in one of the supported public clouds with exactly the same tools and processes that they use for their existing services in Kubernetes. The operator works by extending the Kubernetes API to support custom resources, enabling developers to manage MongoDB resources as native Kubernetes objects, even though Atlas is running outside of the cluster. Developers can create and apply custom resources into Kubernetes to define Atlas configuration, and the Atlas Kubernetes Operator automates the application of that configuration via the Atlas Admin API. 

There are many ways to get started with the Atlas Kubernetes Operator, such as [via the Atlas CLI](https://www.mongodb.com/docs/atlas/cli/current/command/atlas-kubernetes-operator-install/) or through a [Helm Chart](https://www.mongodb.com/docs/atlas/operator/current/ak8so-quick-start-helm/#std-label-ak8so-quick-start-helm-ref). This tutorial will walk you through how to get started using the MongoDB Atlas Kubernetes Operator [Quick Start](https://www.mongodb.com/docs/atlas/operator/current/ak8so-quick-start/). Let’s get started!  

## Pre-requisites
- A MongoDB [Atlas account](https://www.mongodb.com/cloud/atlas/register)
- A running Kubernetes cluster with nodes running processors with the x86-64, AMD64, or ARM64 architecture — you can use some like Minikube to test with
- [Kubectl](https://kubernetes.io/docs/tasks/tools/) and [jq](https://jqlang.github.io/jq/download/) (version 1.6 or higher) installed 

## 1. Set up your Organization ID and API keys
First, you will need to install the operator into a running Kubernetes cluster. Make sure that it’s running processors with the x86-64, AMD64, or ARM64 architecture. In this example, we are running a Kubernetes cluster using [Minikube](https://minikube.sigs.k8s.io/docs/start/?arch=%2Fmacos%2Farm64%2Fstable%2Fbinary+download) and Docker, and using Kubectl to apply commands.
 
If you don’t already have an Atlas account, you can [sign up for one](https://account.mongodb.com/account/register?_ga=2.256169867.1898390183.1720909380-1592895210.1668436484&_gac=1.180422485.1719957183.CjwKCAjwyo60BhBiEiwAHmVLJURlqXVVADnlIGLoEkPSRGZe51Zz6AiMfc2vm8WSHqM3lLHpcPI1phoCuLkQAvD_BwE).

Once you have logged into your Atlas account, you will first create API keys for use by the operator. 

In the top left-hand corner, click on the “Access Manager” drop-down and the API keys tab. Click on “Create API Key.” 

![Atlas Organization Access Manager Page][1]

>Note: If you want the Atlas Kubernetes Operator to create a new Atlas project to work in, you will need to create and use the API keys for [an organization](https://www.mongodb.com/docs/atlas/configure-api-access/#std-label-create-org-api-key). For an existing project, you will need to create them for [a project](https://www.mongodb.com/docs/atlas/configure-api-access/#std-label-create-project-api-key). For this example, we’ll be creating a new Atlas project. For either type of project, it is best practice to configure an [API access list](https://www.mongodb.com/docs/atlas/configure-api-access-project/#std-label-edit-project-api-key-access-list).

Here, you will input a name for your API keys (for example, AKO/Vector Search Keys) and select the organization role. You must assign the API key the Organization Project Creator organization role or higher.

![Atlas UI “Create API Keys” page][2]

From there, you will find a set of unique API keys generated for you. Be sure to save these somewhere secure. Once you leave this page, you will not be able to recover them!

## 2. Install the MongoDB Atlas Kubernetes Operator
Next, we will install/deploy the operator into the Kubernetes cluster you have running. 
Once deployed, the Atlas Kubernetes Operator can monitor resources and events across a single or multiple Kubernetes namespaces. This capability allows the operator to manage MongoDB Atlas custom resources comprehensively across the entire Kubernetes cluster or specific designated namespaces.

For this example, we’ll set up the operator to watch all namespaces in our cluster, but you can learn how to set up the operator to watch a single namespace in our documentation. 

Use the following command in your terminal: 

```bash
kubectl apply -f https://raw.githubusercontent.com/mongodb/mongodb-atlas-kubernetes/v2.3.1/deploy/all-in-one.yaml
```
Make sure to replace the `<version>` with the latest update in the following format: “v2.x.x”.

## 3. Create a secret with your API keys and Organization ID
Next, we’ll create a secret in Kubernetes. A Kubernetes secret is an object used to store and manage sensitive information, such as passwords, OAuth tokens, and SSH keys, securely within a Kubernetes cluster. In this case, it will be the API keys you created earlier. 

If you use external secret storage, you don't need to put sensitive information directly into Kubernetes [secrets](https://kubernetes.io/docs/concepts/configuration/secret/). To learn more, see [Configure Secret Storage](https://www.mongodb.com/docs/atlas/operator/current/ak8so-secret-storage/#std-label-ak8so-secret-storage).

Input the following command and replace `<atlas_organization_id>` with your organization ID, `<atlas_api_public_key>` with your public API key, and `<atlas_api_private_key>` with your private key. 

>Troubleshooting: If the following command produces an error, try to run the command without quotes (eg. `--from-literal=orgId=<atlas_organization_id> \`)

```bash
kubectl create secret generic mongodb-atlas-operator-api-key \
    --from-literal="orgId=<atlas_organization_id>" \
    --from-literal="publicApiKey=<atlas_api_public_key>" \
    --from-literal="privateApiKey=<atlas_api_private_key>" \
    -n mongodb-atlas-system
```

Next, use the following command to label your secret:
```bash
kubectl label secret mongodb-atlas-operator-api-key atlas.mongodb.com/type=credentials -n mongodb-atlas-system
```

## 4. Create an Atlas project custom resource
Projects allow you to isolate different database environments (for instance, development/qa/prod environments) from each other, as well as users/teams. Here, we’ll use the operator to create a project in the Atlas UI through an Atlas project custom resource.

Input the following command in your terminal. 

```bash
cat <<EOF | kubectl apply -f -
apiVersion: atlas.mongodb.com/v1
kind: AtlasProject
metadata:
  name: my-project
spec:
  name: Test Atlas Operator Project
  projectIpAccessList:
    - cidrBlock: "0.0.0.0/0"
      comment: "Allowing access to database from everywhere (only for Demo!)"
EOF
```
>Note: The above commands will add the CIDR 0.0.0.0/0 to your IP access list if input as is. This CIDR allows any client to connect to the Atlas cluster. Do not use this IP address in production. 

If you are working off an existing project in Atlas, be sure to change `name: my-project` to the name of your project in Atlas, and update the `cidrBlock` to your current IP address.

After entering this command, check your Atlas UI. You should see a project created in the Atlas UI called “Test Atlas Operator Project.” 

![Atlas UI “Projects” Page][3]

## 5. Create an Atlas deployment
Deployments are instances of MongoDB running on a cloud provider. This custom resource will create a free tier M0 cluster, but you can always adjust the size of your cluster depending on your workload needs in the custom resource. In your terminal, enter this command: 

```bash
cat <<EOF | kubectl apply -f -
apiVersion: atlas.mongodb.com/v1
kind: AtlasDeployment
metadata:
  name: my-test-cluster
spec:
  projectRef:
    name: my-project
  deploymentSpec:
    clusterType: REPLICASET
    name: test-atlas-operator-project
    replicationSpecs:
       - regionConfigs:
         - regionName: US_EAST_1
           providerName: TENANT
           backingProviderName: AWS
           electableSpecs:
              instanceSize: M0
              nodeCount: 3
EOF
```

Now, within Projects in the Atlas UI, you should see “1 Cluster” created under the Clusters column.

![Atlas UI “Projects” Page][4]

Next, you’ll want to create a secret for your password.

Input the following command. Make sure to replace `P@@ssword%` with a password of your choice. 

```bash
kubectl create secret generic the-user-password --from-literal="password=P@@sword%"
```

Next, use the following command to label your password.

```bash
kubectl label secret the-user-password atlas.mongodb.com/type=credentials
```

With the cluster configured, we’ll create a AtlasDatabaseUser custom resource using the following command. Make sure to replace the password you defined before in the code line that says `the-user-password`. 

```bash
cat <<EOF | kubectl apply -f -
apiVersion: atlas.mongodb.com/v1
kind: AtlasDatabaseUser
metadata:
  name: my-database-user
spec:
  roles:
    - roleName: "readWriteAnyDatabase"
      databaseName: "admin"
  projectRef:
    name: my-project
  username: theuser
  passwordSecretRef:
    name: the-user-password
EOF
```

You will need to keep running the following command until you get a `True` response. This may take some time and you may need to run it multiple times before you get the correct response. 

```bash
kubectl get atlasdatabaseusers my-database-user -o=jsonpath='{.status.conditions[?(@.type=="Ready")].status}'
```

## 6. Retrieve secret and create connection string
Finally, you’ll need to retrieve the secret that the operator created. Getting your connection string to the newly created database can also be done through Kubernetes. For this step, you will need to have [jq](https://jqlang.github.io/jq/download/) (version 1.6 or higher) installed. 

Input the following command after you’ve replaced the following parts of the code with what is in the metadata from the custom resources you applied before:

- `my-project`: Specify the value of the metadata field of your AtlasProject Custom Resource (in this example, `my-project`).
- `my-atlas-cluster`: Specify the value of the metadata field of your AtlasDeployment Custom Resource (in this example, `my-test-cluster`).
- `my-database-user`: Specify the value of the metadata field of your AtlasDatabaseUser Custom Resource (in this example, `my-database-user`)
 
```bash
kubectl get secret {my-project}-{my-atlas-cluster}-{my-database-user} -o json | jq -r '.data | with_entries(.value |= @base64d)';
```

Next, run this command to obtain a connection string. Make sure to input the password and username you had before next to the fields where you see `“password”` and `“username”`.

```bash
{
   "connectionStringStandard": "mongodb://theuser:P%40%40sword%25@test-cluster-shard-00-00.peqtm.mongodb.net:27017,test-cluster-shard-00-01.peqtm.mongodb.net:27017,test-cluster-shard-00-02.peqtm.mongodb.net:27017/?ssl=true&authSource=admin&replicaSet=atlas-pk82fl-shard-0",
   "connectionStringStandardSrv": "mongodb+srv://theuser:P%40%40sword%25@test-cluster.peqtm.mongodb.net",
   "password": "P@@sword%",
   "username": "theuser"
 }
```

Your connection strings will look different from the connection string in this example. 

Finally, you can consume the secret as an environment variable with your application:. 

```bash
containers:
 - name: test-app
   env:
    - name: "CONNECTION_STRING"
      valueFrom:
        secretKeyRef:
          name: test-atlas-operator-project-test-cluster-theuser
          key: connectionStringStandardSrv
```

With that, you have successfully leveraged the Atlas Operator to create and connect to a new Atlas cluster, via Kubernetes! 

To learn more about the Atlas Kubernetes Operator, you can go to the [Atlas Operator documentation](https://www.mongodb.com/docs/atlas/atlas-operator/?_ga=2.142396786.1582366874.1719839208-1592895210.1668436484&_gac=1.54601049.1719957183.CjwKCAjwyo60BhBiEiwAHmVLJURlqXVVADnlIGLoEkPSRGZe51Zz6AiMfc2vm8WSHqM3lLHpcPI1phoCuLkQAvD_BwE). If you are using MongoDB Enterprise Advanced instead of Atlas, we have the [Enterprise Kubernetes Operator](https://www.mongodb.com/try/download/enterprise-kubernetes-operator) available for running MongoDB within your own Kubernetes environment.

If you have questions or want to share your work, head to the [MongoDB Developer Community](https://www.mongodb.com/community/forums/)!


  [1]: https://images.contentstack.io/v3/assets/blt39790b633ee0d5a7/blt98e746cefdf2bd44/66a8bfd4d14106d14c555d3c/image3.png
  [2]: https://images.contentstack.io/v3/assets/blt39790b633ee0d5a7/blt1366d0b416b07a38/66a8bfd5c10344eae6067631/image2.png
  [3]: https://images.contentstack.io/v3/assets/blt39790b633ee0d5a7/blt328055b7b3bd4feb/66a8bfd5d141068402555d40/image4.png
  [4]: https://images.contentstack.io/v3/assets/blt39790b633ee0d5a7/bltc747a9108c60c28f/66a8bfd4adec839e912d0c84/image1.png

Learn how to deploy MongoDB Atlas with the Atlas Kubernetes Operator

How to Deploy MongoDB Atlas With the Atlas Kubernetes Operator

Zuhair Ahmed

Sr. Product Manager

An event-driven architecture is a highly effective framework for creating applications that are responsive, scalable, and maintainable with loosely coupled components. This approach has become essential in developing applications that can rapidly respond to customer demands and dynamic market conditions.

For example, in healthcare, event-driven systems process data emitted by medical devices (like ECGs and blood glucose monitors) to provide real-time insights and alerts to healthcare providers, enabling timely interventions. Financial institutions utilize event-driven architectures to process and analyze transactions in real time. This allows for immediate fraud detection, risk assessment, and dynamic pricing based on current market conditions.

In this tutorial, we cover how to get started with MongoDB Atlas for near real-time data processing with HashiCorp Terraform. By integrating [MongoDB Atlas Stream Processing](https://www.mongodb.com/products/platform/atlas-stream-processing) with the [HashiCorp Terraform MongoDB Atlas Provider](https://registry.terraform.io/providers/mongodb/mongodbatlas/latest), developers can efficiently manage real-time data workflows and automate resource provisioning.

Let’s get started!

## Prerequisites

- Create a [MongoDB Atlas](https://www.mongodb.com/cloud/atlas/register) account. (More configuration details will be shown below.)
- Install [HashiCorp Terraform](https://developer.hashicorp.com/terraform/tutorials/aws-get-started/install-cli) on your terminal or sign up for a free [Terraform Cloud](https://app.terraform.io/public/signup/account) account.
- Create [MongoDB Atlas programmatic API keys](https://www.mongodb.com/docs/atlas/configure-api-access/#grant-programmatic-access-to-service) and [associate them with Terraform](https://www.mongodb.com/developer/products/atlas/deploy-mongodb-atlas-terraform-aws/) (further details below).
- Choose your IDE. For this tutorial, we will be using [VS Code](https://code.visualstudio.com/).

## Deploy foundational Atlas Stream Processing resources using the Terraform provider 

There are three main components to consider with Atlas Stream Processing: 

- [Stream Processing Instance](https://www.mongodb.com/docs/atlas/atlas-sp/overview/#mongodb-expression-exp.Stream-Processing-Instance) (SPI) — This acts as the primary environment where your streaming data processes are executed. It's responsible for managing and executing the stream processing tasks, maintaining performance, and ensuring data integrity throughout the streaming lifecycle.
- [Connection registry](https://www.mongodb.com/docs/atlas/atlas-sp/overview/#mongodb-expression-exp.Connection-Registry) — This serves as the conduit between your data sources/sinks and the processing instance. It's tasked with the secure and efficient transport of streaming data, facilitating continuous data ingestion from various sources into the SPI.
- [Stream processor](https://www.mongodb.com/docs/atlas/atlas-sp/overview/#mongodb-expression-exp.Stream-Processor) — This is where the business logic for data transformation and analysis is defined and executed. Using MongoDB's expressive querying and aggregation capabilities, this component processes incoming data streams continuously, enabling immediate data manipulation and enrichment before forwarding it for storage or further processing.

At this time, the Terraform Atlas Provider supports management of the SPI ([mongodbatlas_stream_instance](https://registry.terraform.io/providers/mongodb/mongodbatlas/latest/docs/resources/stream_instance)) and connections in the Connection Registry ([mongodbatlas_stream_connection](https://registry.terraform.io/providers/mongodb/mongodbatlas/latest/docs/resources/stream_connection)), with programmatic support for stream processors coming in the future. 

First, we need to deploy basic Atlas resources to get started. As part of this tutorial, this includes an Atlas project, M10 dedicated Atlas cluster, database user, and IP access list entry. 

In addition, we will be deploying one Stream Processing Instance and two connections in the connection registry. See our Atlas Admin [Open API Specification documentation](https://www.mongodb.com/docs/atlas/reference/api-resources-spec/v2/#tag/Streams/operation/createStreamInstance) for a full list of supported cloud providers, regions, and all other parameter inputs for these resources:

- In this example, our Stream Processing Instance will be hosted on AWS us-east-1 using an SP30 tier. More information on Stream Processing tiers is available in the [documentation](https://www.mongodb.com/docs/atlas/billing/stream-processing-costs/).
- For the connection registry, we will point our data source to type [Sample](https://www.mongodb.com/docs/atlas/atlas-stream-processing/tutorial/#add-the-sample-stream-connection-to-the-connection-registry.) (solar data that we provide as a sample source to use when getting started) which is great for testing purposes. (Type Kafka is also supported.)
- For a connection to the built-in sample_stream_solar, provide a configuration file with the following syntax:

```
{
  "name": "sample_stream_solar",
  "type": "Sample"
}
```

- Our data sink will point to the M10 dedicated Atlas cluster we mentioned earlier. 
- In the sample Terraform script provided below, replace the “org_id” field, and enter your own organization’s ID and your unique username and password in the “username” and “password” fields.
- Ensure that the IP address added to the script matches the one in Terraform. If you are unsure of the IP address that you are running Terraform on (and you are performing this step from that machine), simply click “Use Current IP Address and Save”. Another option is to open up your IP access list to all, but this comes with significant potential risks. To do this, you can add the following two CIDRs: 0.0.0.0/1 and 128.0.0.0/1. These entries will open your IP access list to, at most, 4,294,967,296 (or 2^32) IPv4 addresses and should be used with caution.

See the below Terraform script as part of our `main.tf` file.

```
terraform {
  required_providers {
    mongodbatlas = {
      source = "mongodb/mongodbatlas"
    }
  }
  required_version = ">= 0.13"
}

resource "mongodbatlas_project" "testProject" {
  name   = "testProject16"
  org_id = "[YOUR ORG_ID]"
}

resource "mongodbatlas_project_ip_access_list" "testIPAccessList" {
  project_id = mongodbatlas_project.testProject.id
  ip_address = "[YOUR IP ADDRESS]"
}

resource "mongodbatlas_database_user" "testUser" {
  project_id         = mongodbatlas_project.testProject.id
  username           = "[YOUR USERNAME]"
  password           = "[YOUR PASSWORD]"
  auth_database_name = "admin"
  roles {
    role_name     = "readWrite"
    database_name = "dbforApp"
  }
}

resource "mongodbatlas_advanced_cluster" "testCluster" {
  project_id   = mongodbatlas_project.testProject.id
  name         = "testCluster"
  cluster_type = "REPLICASET"
  replication_specs {
    region_configs {
      electable_specs {
        instance_size = "M10"
        node_count    = 3
      }
      provider_name = "AWS"
      priority      = 7
      region_name   = "US_EAST_1"
    }
  }
}

resource "mongodbatlas_stream_instance" "myFirstStreamInstance" {
  project_id    = mongodbatlas_project.testProject.id
  instance_name = "myFirstStreamInstance"
  data_process_region = {
    region         = "VIRGINIA_USA"
    cloud_provider = "AWS"
  }
  stream_config = {
    tier = "SP30"
  }
}

resource "mongodbatlas_stream_connection" "dataSource" {
    project_id = mongodbatlas_project.testProject.id
    instance_name = mongodbatlas_stream_instance.myFirstStreamInstance.instance_name
    connection_name = "sample_stream_solar"
    type = "Sample"
}

resource "mongodbatlas_stream_connection" "dataSink" {
    project_id = mongodbatlas_project.testProject.id
    instance_name = mongodbatlas_stream_instance.myFirstStreamInstance.instance_name
    connection_name = "dataSink"
    type = "Cluster"
    cluster_name = mongodbatlas_advanced_cluster.testCluster.name
    db_role_to_execute = {
      role = "atlasAdmin"
      type = "BUILT_IN"
    }
}
```

Note: Before deploying, be sure to store your MongoDB Atlas programmatic API keys created as part of prerequisites as [Environment Variables](https://developer.hashicorp.com/terraform/cli/config/environment-variables). To deploy, you can use the below commands from the terminal: 

```
terraform init 
terraform plan
terraform apply 
```

If your deployment was successful, you should be greeted with “Apply complete!” 

![Screenshot of completed deployment][1]

To confirm, you should be able to see your newly created SPI in the Atlas UI. (To navigate from the home screen, click on “Stream Processing” on the left-hand menu.) 

![Screenshot of Atlas UI after creation of stream processing instance][2]

By going into the Connection Registry, we can also verify that our connections were successfully created. 

![Screenshot of Atlas UI showcasing connection registry][3]

## Get connection string and create stream processor 

Next, you’ll need to retrieve the Atlas Stream Processing connection string and create your first stream processor.

While still in the Atlas UI, click “Connect” and select the tool to access your data through MongoDB Shell or MongoDB for VS Code. 

![Screenshot of Atlas UI showcasing connection to MongoDB Shell and MongoDB VSCode][4]

This will generate your connection string which you can use to connect to your Stream Processing Instance. 

At this point, your first stream processors can be created to define how your data will be processed in your SPI. As mentioned earlier, there are currently no Terraform resources as of yet to provide this configuration. To learn more about how stream processors can be configured, refer to [Manage Stream Processors](https://www.mongodb.com/docs/atlas/atlas-sp/manage-stream-processor/#manage-stream-processors).

For more detailed information, also see our [Atlas Stream Processing example](https://github.com/mongodb/terraform-provider-mongodbatlas/blob/master/examples/mongodbatlas_stream_instance/atlas-streams-user-journey.md) in our Terraform repo. 

## All done 

Congratulations! You have everything that you need now to run your first [Atlas Stream Processing](https://www.mongodb.com/docs/atlas/atlas-sp/overview/)-based workflow. 

With the above steps, teams can utilize Atlas Stream Processing capabilities within the Terraform MongoDB Atlas Provider. Whether you are developing complex event processing systems, real-time analytics, or interactive user engagement features, Atlas Stream Processing integrated with Terraform provides a robust infrastructure to support scalable and resilient applications. 

The HashiCorp Terraform Atlas Provider is open-sourced under the [Mozilla Public License v2.0](https://github.com/mongodb/terraform-provider-mongodbatlas?tab=MPL-2.0-1-ov-file) and we welcome community contributions. To learn more, see our [contributing guidelines](https://github.com/mongodb/terraform-provider-mongodbatlas/blob/master/CONTRIBUTING.md). 

The fastest way to get started is to create a MongoDB Atlas account from the [AWS Marketplace](https://aws.amazon.com/marketplace/seller-profile?id=c9032c7b-70dd-459f-834f-c1e23cf3d092). To learn more about the Terraform Provider, check out our [documentation](https://registry.terraform.io/providers/mongodb/mongodbatlas/latest/docs), [tutorials](https://www.mongodb.com/developer/search/?s=Terraform&sortMode=0), [solution brief](https://www.mongodb.com/collateral/build-resilient-applications-with-mongodb-atlas-and-hashicorp-terraform), or [get started today](https://cloud.mongodb.com).

Go build with MongoDB Atlas and the HashiCorp Terraform Atlas Provider today! 


  [1]: https://images.contentstack.io/v3/assets/blt39790b633ee0d5a7/bltd8b6fd1c026e0dfa/664bc07e1983ea56f5ac2899/1.png
  [2]: https://images.contentstack.io/v3/assets/blt39790b633ee0d5a7/bltc57e0777ae3b69f3/664bc0907853391c269e3c1f/2.png
  [3]: https://images.contentstack.io/v3/assets/blt39790b633ee0d5a7/bltb73dbc3b43a2e8a0/664bc09f941bc85c3f9df7b1/3.png
  [4]: https://images.contentstack.io/v3/assets/blt39790b633ee0d5a7/blt6e9c931aa8a32787/664bc0ae55b39716c16049d8/4.png

How to Get Started with MongoDB Atlas Stream Processing and the HashiCorp Terraform MongoDB Atlas Provider

Last year, MongoDB announced the general availability of [Atlas Vector Search](https://www.mongodb.com/blog/post/introducing-atlas-vector-search-build-intelligent-applications-semantic-search-ai), a new capability in Atlas that allows developers to search across data stored in MongoDB based on its semantic meaning using high dimensional vectors (i.e., “embeddings”) created by machine learning models. 

This allows developers to build intelligent applications that can understand and process human language in a way traditional, text-based search methods cannot since they will only produce an exact match for the query. 

For example, searching for “warm winter jackets” on an e-commerce website that only supports text-based search might return products with the exact match keywords "warm," "winter," and "jackets." Vector search, on the other hand, understands the semantic meaning of "warm winter jackets'' as apparel designed for cold temperatures. It retrieves items that are not only labeled as "winter jackets” but are specifically designed for warmth, including products that might be described with related terms like "insulated," giving users more helpful search results. 

Integrating Atlas Vector Search with infrastructure-as-code (IaC) tools like HashiCorp Terraform can then streamline and optimize your development workflows, ensuring that sophisticated search capabilities are built directly into the infrastructure deployment process. 

This guide will walk you through how to get started with Atlas Vector Search through our [HashiCorp Terraform Atlas provider](https://registry.terraform.io/providers/mongodb/mongodbatlas/latest). Let’s get started! 

### Pre-requisites

 - Create a [MongoDB Atlas](https://www.mongodb.com/cloud/atlas/register) account. 
 - Install [HashiCorp Terraform](https://developer.hashicorp.com/terraform/tutorials/aws-get-started/install-cli) on your terminal or sign up for a free [Terraform Cloud](https://app.terraform.io/public/signup/account) account.
 - Create [MongoDB Atlas programmatic API keys](https://www.mongodb.com/docs/atlas/configure-api-access/#grant-programmatic-access-to-service) and [associate them with Terraform](https://www.mongodb.com/developer/products/atlas/deploy-mongodb-atlas-terraform-aws/). 
 - Select an IDE of your choice. For this tutorial, we will be using [VS Code](https://code.visualstudio.com/).

## Step 1: Deploy Atlas dedicated cluster with Atlas Search Nodes

First, we need to deploy basic Atlas resources to get started. This includes an Atlas project, an M10 dedicated Atlas cluster (which is pay-as-you-go, great for development and low-traffic applications), a database user, and an IP Access List Entry. 

**Note**: When configuring your MongoDB Atlas cluster with Terraform, it's important to restrict IP access to only the IP address from which the Terraform script will be deployed. This minimizes the risk of unauthorized access. 

In addition, as part of this tutorial, we will be using [Atlas Search Nodes](https://www.mongodb.com/blog/post/search-nodes-now-public-preview-performance-scale-dedicated-infrastructure) (optional). These provide dedicated infrastructure for [Atlas Search](https://www.mongodb.com/cloud/atlas/lp/search) and [Vector Search](https://www.mongodb.com/products/platform/atlas-vector-search) workloads, allowing you to fully scale search independent of database needs. Incorporating Search Nodes into your Atlas deployment allows for better performance at scale and delivers workload isolation, higher availability, and the ability to optimize resource usage. 

Lastly, when using Terraform to manage infrastructure, it is recommended to maintain organized file management practices. Typically, your Terraform configurations/scripts will be written in files with the `.tf` extension, such as `main.tf`. This file, which we are using in this tutorial, contains the primary configuration details for deploying resources and should be located ideally in a dedicated project directory on your local machine or on Terraform Cloud. 

See the below Terraform script as part of our `main.tf` file: 

```
terraform {
  required_providers {
    mongodbatlas = {
      source = "mongodb/mongodbatlas"
    }
  }
  required_version = ">= 0.13"
}

resource "mongodbatlas_project" "exampleProject" {
  name   = "exampleProject"
  org_id = "63234d3234ec0946eedcd7da"
}

resource "mongodbatlas_advanced_cluster" "exampleCluster" {
  project_id   = mongodbatlas_project.exampleProject.id
  name         = "ClusterExample"
  cluster_type = "REPLICASET"

  replication_specs {
    region_configs {
      electable_specs {
        instance_size = "M10"
        node_count    = 3
      }
      provider_name = "AWS"
      priority      = 7
      region_name   = "US_EAST_1"
    }
  }
}

resource "mongodbatlas_search_deployment" "exampleSearchNode" {
  project_id   = mongodbatlas_project.exampleProject.id
  cluster_name = mongodbatlas_advanced_cluster.exampleCluster.name
  specs = [
    {
      instance_size = "S20_HIGHCPU_NVME"
      node_count    = 2
    }
  ]
}

resource "mongodbatlas_database_user" "testUser" {
  username           = "username123"
  password           = "password-test123"
  project_id         = mongodbatlas_project.exampleProject.id
  auth_database_name = "admin"

  roles {
    role_name     = "readWrite"
    database_name = "dbforApp"
  }
}

resource "mongodbatlas_project_ip_access_list" "test" {
  project_id = mongodbatlas_project.exampleProject.id
  ip_address = "174.218.210.1"
}
```

**Note**: Before deploying, be sure to store your MongoDB Atlas programmatic API keys created as part of the prerequisites as [environment variables](https://developer.hashicorp.com/terraform/cli/config/environment-variables). To deploy, you can use the below commands from the terminal: 

```
terraform init 
terraform plan
terraform apply 
```

## Step 2: Create your collections with vector data 

For this tutorial, you can create your own collection of vectorized data if you have data to use. 

Alternatively, you can use our [sample data](https://www.mongodb.com/docs/atlas/sample-data/). This is great for testing purposes. The collection you can use is the "sample_mflix.embedded_movies" which already has embeddings generated by Open AI. 

To use sample data, from the Atlas UI, go into the Atlas cluster Overview page and select “Atlas Search” at the top of the menu presented. 
![(Cluster Overview in the Atlas UI)](https://images.contentstack.io/v3/assets/blt39790b633ee0d5a7/blt5c0ebd39059d0e07/6627cb3b776d0cbe1d24d57a/Screenshot_2024-04-23_at_10.52.09_AM.png)

Then, click “Load a Sample Dataset.”
![(Atlas Search menu in the Atlas UI)](https://images.contentstack.io/v3/assets/blt39790b633ee0d5a7/bltae4597ed96ca2b63/6627cb85210d90f4f03a44b4/Screenshot_2024-04-23_at_10.53.44_AM.png)

## Step 3: Add vector search index in Terraform configuration 

Now, head back over to Terraform and create an Atlas Search index with type “vectorSearch.” If you are using the sample data, also include a reference to the database “sample_mflix” and the collection “embedded_movies.” 

Lastly, you will need to set the “fields” parameter as per our example below. See our documentation to learn more about [how to index fields for vector search](https://www.mongodb.com/docs/atlas/atlas-vector-search/vector-search-type/#how-to-index-fields-for-vector-search) and the associated required parameters. 

```
resource "mongodbatlas_search_index" "test-basic-search-vector" {
  name   = "test-basic-search-index"  
  project_id = mongodbatlas_project.exampleProject.id
  cluster_name = mongodbatlas_advanced_cluster.exampleCluster.name
  type = "vectorSearch"
  database = "sample_mflix"
  collection_name = "embedded_movies"
  fields = <<-EOF
    [{
        "type": "vector",
        "path": "plot_embedding",
        "numDimensions": 1536,
        "similarity": "euclidean"
    }]
    EOF
}
```

To deploy again, you can use the below commands from the terminal: 
```
terraform init 
terraform plan
terraform apply 
```

If your deployment was successful, you should be greeted with “Apply complete!” 
![(Terraform in terminal showcasing deployment)](https://images.contentstack.io/v3/assets/blt39790b633ee0d5a7/bltded5c4cdf9b4a083/6627cbeae7ce95db12de9b09/Screenshot_2024-04-23_at_10.55.27_AM.png)

To confirm, you should be able to see your newly created Atlas Search index resource in the Atlas UI with Index Type “vectorSearch” and Status as “ACTIVE.” 
![(Atlas UI showcasing Vector Search deployment)](https://images.contentstack.io/v3/assets/blt39790b633ee0d5a7/blt23db256850afad56/6627cc13bb637218621defb9/Screenshot_2024-04-23_at_10.56.10_AM.png)

## Step 4: Get connection string and connect to the MongoDB Shell to begin Atlas Vector Search queries 

While still in the Atlas UI, go back to the homepage, click “Connect” on your Atlas cluster, and select “Shell.” 

![(Atlas UI page where you can choose a cluster connection method)](https://images.contentstack.io/v3/assets/blt39790b633ee0d5a7/blt95d147e9c622bd5d/6627cc47e7ce95a8b8de9b1a/Screenshot_2024-04-23_at_10.56.59_AM.png)

This will generate your connection string which you can use in the MongoDB Shell to connect to your Atlas cluster. 

### All done

Congratulations! You have everything that you need now to run your first [Vector Search queries](https://www.mongodb.com/docs/atlas/atlas-vector-search/vector-search-stage/).

With the above steps, teams can leverage Atlas Vector Search indexes and dedicated Search Nodes for the Terraform MongoDB Atlas provider to build a retrieval-augmented generation, semantic search, or recommendation system with ease. 

The HashiCorp Terraform Atlas provider is open-sourced under the [Mozilla Public License v2.0](https://github.com/mongodb/terraform-provider-mongodbatlas?tab=MPL-2.0-1-ov-file) and we welcome community contributions. To learn more, see our [contributing guidelines](https://github.com/mongodb/terraform-provider-mongodbatlas/blob/master/CONTRIBUTING.md).

The fastest way to get started is to create a MongoDB Atlas account from the [AWS Marketplace](https://aws.amazon.com/marketplace/seller-profile?id=c9032c7b-70dd-459f-834f-c1e23cf3d092) or [Google Cloud Marketplace](https://console.cloud.google.com/marketplace/product/mongodb/mdb-atlas-self-service?pli=). To learn more about the Terraform provider, check out the [documentation](https://registry.terraform.io/providers/mongodb/mongodbatlas/latest/docs), [solution brief](https://www.mongodb.com/collateral/build-resilient-applications-with-mongodb-atlas-and-hashicorp-terraform), and [tutorials](https://www.mongodb.com/developer/search/?s=Terraform&sortMode=0), or [get started today](https://account.mongodb.com/account/login?signedOut=true&n=https%3A%2F%2Fcloud.mongodb.com%2Fgo%3Fl%3Dhttps%253A%252F%252Fcloud.mongodb.com%252Fv2%252F%253Cproject%253E%2523%252FdataFederation%26_ga%3D2.121590028.567858162.1702935684-1592895210.1668436484%26_gac%3D1.90809704.1702935684.CjwKCAiA-P-rBhBEEiwAQEXhH1-PuNDJyW2Z4QzJDKtcrNGmvK965-IyQbQ-bitpEZxRCffLHhX-KRoC1HsQAvD_BwE).  

Go build with MongoDB Atlas and the HashiCorp Terraform Atlas provider today! 



Learn how to leverage Atlas Vector Search with HashiCorp Terraform in this tutorial.

Leveraging Atlas Vector Search With HashiCorp Terraform: Empowering Semantic Search in Modern Applications

MongoDB Atlas is the multi-cloud developer data platform that provides an integrated suite of cloud database and data services. We help to accelerate and simplify how you build resilient and performant global applications on the cloud provider of your choice.

[AWS CloudFormation](https://www.mongodb.com/atlas/aws-cloudformation) lets you model, provision, and manage AWS and third-party resources like MongoDB Atlas by treating infrastructure as code (IaC). CloudFormation templates are written in either JSON or YAML. 

While there are multiple ways to use CloudFormation to provision and manage your Atlas clusters, such as with Partner Solution Deployments or the AWS CDK, today we’re going to go over how to create your first YAML CloudFormation templates to deploy Atlas clusters with CloudFormation.

These pre-made templates directly leverage MongoDB Atlas resources from the CloudFormation Public Registry and execute via the AWS CLI/AWS Management Console. Using these is best for users who seek to be tightly integrated into AWS with fine-grained access controls. 

Let’s get started! 

*Prerequisites:* 

- Install and configure an [AWS Account](https://docs.aws.amazon.com/accounts/latest/reference/welcome-first-time-user.html) and the [AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-quickstart.html).
- Install and configure the [MongoDB Atlas CLI](https://www.mongodb.com/tools/atlas-cli) (optional but recommended). 

## Step 1: Create a MongoDB Atlas account

Sign up for a [free MongoDB Atlas account](https://www.mongodb.com/cloud/atlas/register), verify your email address, and log into your new account.

Already have an AWS account? Atlas supports paying for usage via the AWS Marketplace (AWS MP) without any upfront commitment — simply <a href="https://aws.amazon.com/marketplace/pp/prodview-pp445qepfdy34?trk=38088f03-4928-4b00-9a39-86781df6ab9b&sc_channel=el" target="_blank" rel="noreferrer">sign up for MongoDB Atlas via AWS Marketplace.<a>

![a sign-up page for MongoDB Atlas with a on the left hand side and form on the right hand side.][1]

## Step 2: Generate a MongoDB Atlas Programmatic API Key

Once you have an account created and are logged into the MongoDB Atlas UI (user interface), you will need to generate a Programmatic API Key (PAK) to authenticate with the CloudFormation resources to MongoDB Atlas. 

Go to the top of the Atlas UI, click the gear icon to the right of the organization name you created, click “Access Manager” in the left-hand menu bar, click the “Create API Keys” tab, and then click the green “Create API Key” box.

![An interface for the Atlas Access Manager.][2]

Enter a description for the API key that will help you remember what it’s being used for — for example, “CloudFormation API Key.” Next, you’ll select the appropriate user permission for what you want to accomplish with CloudFormation. Both the Organization Owner and Organization Project Creator roles (see role descriptions below) provide access to complete this task, but by using the principle of least privilege, let’s select the Organization Project Creator role in the dropdown menu and click “Next.” Create a public key (which acts as a username) and then create a private key (which acts as a password). 

**IMPORTANT: Make sure to copy your private key and store it in a secure location. After you leave this page, your full private key will not be accessible.**

## Step 3: Add API key Access List Entry

When creating MongoDB programmatic API keys for Atlas Admin API, you will need to add the IP address where CloudFormation will be executed on to the IP Whitelist to ensure that only requests from specific IP addresses can access your Atlas cluster using that key. The IP Whitelist provides an additional layer of security by limiting the IP addresses that can connect to your Atlas resources. If an IP address is not whitelisted, any request originating from it will be denied.

On the same page, scroll down and click “Add Access List Entry.” If you are unsure of the IP address that CloudFormation is using to deploy your infrastructure, review the [AWS IP address ranges ](https://docs.aws.amazon.com/vpc/latest/userguide/aws-ip-ranges.html)and contact AWS support directly, who can help confirm the CIDR range to be used in your Atlas PAK IP Whitelist. 

![A screen that displays a form for creating an API key][3]

Another option is to open up your IP Access List to all, but this comes with significant potential risk. To do this, make sure to add in the appropriate organization permissions to your API key. Next, add the following two CIDRs: 0.0.0.0/1 and 128.0.0.0/1. These entries will open your IP Access List to all access — or at most, 4,294,967,296 (or 2^32) IPv4 addresses — and should be used with caution. Learn more about [how to create a Programmatic API Key](https://www.mongodb.com/docs/atlas/configure-api-access/#create-an-api-key-in-an-organization) on MongoDB Atlas.

![An input box that you put an IP address into][4]

## Step 4: Store your MongoDB PAK in AWS Secrets Manager

Before using CloudFormation, you’ll need to store the programmatic API key you created in AWS Secrets Manager. Go to the Secrets Manager tab and click “Add New Secret.” You should be taken to a page that looks like this: 

![An interface with boxes in which a user inputs their secret values][5]

From there, select “Other Type of Secret” and then add in your public and private keys. You can enter them manually or add them in the plaintext section (see below).

![A terminal for inputting secret key/value pairs][6]

In plaintext, format the secret as such: 

```json
{ "PublicKey": "YourPublicKey", "PrivateKey": "YourPrivateKey" }
```

On the next page, give your secret a name. In this case, it would be **cfn/atlas/profile/default**, the same as the one used in the previous step.

## Step 5: Activate AWS third-party extensions in the same region as where your AWS Secrets Manager secret is stored

In the AWS Management Console, click on the CloudFormation button in the navigation bar. Select your region, and then in the left-hand column, select “Public Extensions” under the Registry drop-down. Select “Resource types” under extension type and “Third party” under publisher. 

![A repository of public extensions available on AWS][7]

Make sure to activate your AWS CloudFormation resources in the AWS region from which you wish to deploy your CloudFormation resources. In this example, we deploy from N. Virginia (us-east-1). 

Once a list of resources populates after making your selections, you’ll need to select which resources you need. You will need to use the resources defined by your template. Some example resources include: 

- MongoDB::Atlas::Project
- MongoDB::Atlas::Cluster
- MongoDB::Atlas::DatabaseUser
- MongoDB::Atlas::ProjectIpAccessList

You can find the resources your template needs outlined in the template.

![A selection for the MongoDB Atlas Cluster public extension][8]

When you select your resources, you will need to provide an ARN ([Amazon Resource Names](https://docs.aws.amazon.com/IAM/latest/UserGuide/reference-arns.html)). You can set this up with AWS IAM ([Identity and Access Management](https://aws.amazon.com/iam/)). You can find that in the navigation bar of your AWS. You can find the ARN in the user information in the “Roles” button. Once there, find the role whose ARN you want to use and add it to the Extension Details in CloudFormation. Learn [how to create user roles/permissions](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_users_create.html) in the IAM. 

![An interface in which a user inputs details of their extension, such as an ARN][9]

Once your ARN has been added in, click “Activate” and your permissions will be activated. See a [sample AWS ARN](https://github.com/mongodb/mongodbatlas-cloudformation-resources/blob/master/cfn-resources/resource-role.yaml) required from our GitHub repo. It’s important that you use an ARN with sufficient permissions each time it’s asked for.

![The Identity and Access Management interface where the ARN is highlighted in blue.][10]

Additionally, you can add in this trust relationship string to ensure that the admin or other users are able to perform CloudFormation functions. The following example outlines how to add this string to the space in the Trust relationships section:

```json
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "Statement1",
            "Effect": "Allow",
            "Principal": {
                "AWS": "resource.cloudformation.amazonaws.com"
            },
            "Action": "sts:AssumeRole"
        }
    ]
}
```

Note: Broad IAM access is not recommended for production workloads but may be sufficient for testing.

![Interface in the AWS extension registry where users input the ARN to activate extension. Automatic updates option is highlighted in blue.][11]

In this example, I have opted in to automatically update versions. This step is optional. 

## Step 6: Sample a CloudFormation template deploying MongoDB Atlas resources

If you would rather use a sample CloudFormation template in YAML than create your own, you can find it in the [MongoDB git repository](https://github.com/aws-quickstart/quickstart-mongodb-atlas/blob/8cd6e87ed36f467202432eed170a25ff10abf566/templates/mongodb-atlas.template.yaml). 

## Step 7: Deploy the CloudFormation template

In the AWS management console, go to the CloudFormation tab. Then, in the left-hand navigation, click on “Stacks.” In the window that appears, hit the “Create Stack” drop-down. Select “Create new stack with existing resources.” 

Next, select “template is ready” in the “Prerequisites” section and “Upload a template” in the “Specify templates” section. From here, you will choose the YAML (or JSON) file containing the MongoDB Atlas deployment that you created in the prior step.

![Interface in CloudFormation where user creates a stack. Options for “Template is ready” and “Upload a template file” are highlighted in blue.][12]

Next, you’ll need to specify stack details by filling out the stack name and the parameters. It’s important that the information on this page lines up with the script in the template. For example, you’ll need to make sure that the name of your AWS Secret Manager secret (in this case, “default”) is the same as what you used in your deployment. 

Then, configure your stack options with tags, permissions, and stack failure options. Choose a unique database user password.

![The last page of the “Create Stack” interface. Acknowledgements are highlighted in blue and two boxes are checked.][13]

Finally, review your stack, hit “Submit,” and you’re set! Note deploying a MongoDB Atlas cluster for the first time can take seven to 10 minutes. You can check the status in the Atlas UI. 

Congratulations! You have just deployed MongoDB Atlas resources with AWS CloudFormation.

The MongoDB Atlas CloudFormation resources are open-sourced under the Apache-2.0 license and we welcome community contributions. To learn more, see our [contributing guidelines](https://github.com/mongodb/awscdk-resources-mongodbatlas/blob/main/CONTRIBUTING.md). 

The fastest way to get started is to create a [MongoDB Atlas account from the AWS Marketplace](https://aws.amazon.com/marketplace/pp/prodview-pp445qepfdy34). 

Additionally, you can watch our [demo](https://www.youtube.com/watch?v=I7ljHD0-ZwM) to learn about the other ways to get started with MongoDB Atlas and CloudFormation

Go build with MongoDB Atlas and AWS CloudFormation today!


  [1]: https://images.contentstack.io/v3/assets/blt39790b633ee0d5a7/blt6a7a0aace015cbb5/6504a623a8cf8bcfe63e171a/image4.png
  [2]: https://images.contentstack.io/v3/assets/blt39790b633ee0d5a7/blt471e37447cf8b1b1/6504a651ea4b5d10aa5135d6/image8.png
  [3]: https://images.contentstack.io/v3/assets/blt39790b633ee0d5a7/blt3545f9cbf7c8f622/6504a67ceb5afe6d504a833b/image13.png
  [4]: https://images.contentstack.io/v3/assets/blt39790b633ee0d5a7/blt3582d0a3071426e3/6504a69f0433c043b6255189/image12.png
  [5]: https://images.contentstack.io/v3/assets/blt39790b633ee0d5a7/bltb4253f96c019874e/6504a6bace38f40f4df4cddf/image1.png
  [6]: https://images.contentstack.io/v3/assets/blt39790b633ee0d5a7/blt2840c92b6d1ee85d/6504a6d7da83c92f49f9b77e/image7.png
  [7]: https://images.contentstack.io/v3/assets/blt39790b633ee0d5a7/bltd4a32140ddf600fc/6504a700ea4b5d515f5135db/image5.png
  [8]: https://images.contentstack.io/v3/assets/blt39790b633ee0d5a7/blt49dabfed392fa063/6504a73dbb60f713d4482608/image9.png
  [9]: https://images.contentstack.io/v3/assets/blt39790b633ee0d5a7/blt592e3f129fe1304b/6504a766a8cf8b5ba23e1723/image11.png
  [10]: https://images.contentstack.io/v3/assets/blt39790b633ee0d5a7/bltbff284987187ce16/6504a78bb8c6d6c2d90e6e22/image10.png
  [11]: https://images.contentstack.io/v3/assets/blt39790b633ee0d5a7/blt0ae450069b31dff9/6504a7b99bf261fdd46bddcf/image3.png
  [12]: https://images.contentstack.io/v3/assets/blt39790b633ee0d5a7/blt7f24645eefdab69c/6504a7da9aba461d6e9a55f4/image2.png
  [13]: https://images.contentstack.io/v3/assets/blt39790b633ee0d5a7/blt7e1c20eba155233a/6504a8088606a80fe5c87f31/image6.png

Learn how to quickly and easily deploy MongoDB Atlas instances with Amazon Web Services (AWS) CloudFormation.

Rutuja Rajwade

Content by Rutuja Rajwade

How to Deploy Vector Search, Atlas Search, and Search Nodes With the Atlas Kubernetes Operator

How to Deploy MongoDB Atlas With the Atlas Kubernetes Operator

How to Get Started with MongoDB Atlas Stream Processing and the HashiCorp Terraform MongoDB Atlas Provider

Leveraging Atlas Vector Search With HashiCorp Terraform: Empowering Semantic Search in Modern Applications

How to Deploy MongoDB Atlas with AWS CloudFormation