Community

Samuel Molling

You’re responsible for a payroll system at a fast-growing tech company. Employees rely on you to protect highly sensitive data—things like salaries, bonuses, and personal information. Meanwhile, privacy regulations such as GDPR and LGPD demand strict compliance, leaving no room for mistakes. The stakes are high: Secure the data, or risk penalties and broken trust.

But traditional encryption isn’t enough. Protecting data while it’s in transit or stored only solves part of the problem. What happens when HR needs to run a query on that data? You need an encryption solution that works seamlessly even during data processing—one that keeps everything secure but still allows meaningful searches, like “find all employees earning over $10,000.” That’s where MongoDB’s CSFLE and Queryable Encryption come in, offering cutting-edge tools to balance security and usability.
## A quick overview of encryption
Before diving into the specifics of CSFLE and Queryable Encryption, let’s revisit the core encryption methods MongoDB offers:
### Encryption in transit
MongoDB secures data in motion by encrypting client-server traffic with TLS/SSL. This method is essential for protecting data during network transmission. Also, all communications between nodes and processes are encrypted with TLS, ensuring comprehensive security for data in motion.
### Encryption at rest
Encryption at rest secures data stored on disk, available in MongoDB Enterprise Advanced and Atlas versions. This approach protects data even if an adversary gains access to the disk or underlying database files, ensuring confidentiality beyond just server security.
### Encryption in-use
Protects data at all stages of its lifecycle transmission, storage, and processing. MongoDB offers two approaches for in-use encryption:
- **Queryable Encryption**
- **Client-side field-level encryption (CSFLE)**

## Differences between CSFLE and Queryable Encryption
While there is some overlap, there are key differences between CSFLE and Queryable Encryption.
### Encryption method and inference security
#### CSFLE
- Uses deterministic encryption for fields needing equality queries, producing identical ciphertexts for identical values 
- Enables exact match queries but can lead to significant information leakage in some cases, as patterns may emerge, especially with low-cardinality data
#### Queryable Encryption
- Uses non-deterministic encryption, generating unique ciphertexts for identical values, making attacks more difficult
- Supports equality queries and, from MongoDB 8.0 onward, range queries using operators like `$lt`, `$lte`, `$gt`, and `$gte`

### Supported query types
#### CSFLE
- Supports only exact match (equality) queries on deterministically encrypted fields.
- Suitable for scenarios that don’t require complex query operations; this makes CSFLE ideal where equality is the only required query type
#### Queryable Encryption
- Offers advanced query support for encrypted data; besides equality queries, MongoDB 8.0 adds support for range queries using operators like `$lt`, `$lte`, `$gt`, and `$gte`
- Will soon expand this functionality to include prefix, suffix, and substring queries, enhancing flexibility for encrypted data querying

### When to use each approach
#### CSFLE
- Ideal for scenarios where full control over encryption keys is needed, and equality queries suffice for application requirements
- Particularly useful for highly sensitive data that requires protection throughout client-server communication, offering granular control over the encryption process
#### Queryable Encryption
- Recommended for applications requiring complex encrypted data queries, such as date ranges or numerical values
- Its design leaks less information about the encrypted data than CSFLE; this is open source and the algorithms are published and reviewed by external cryptography experts who analyze the security of its design and implementation
## Implementing CSFLE and Queryable Encryption in Python
To demonstrate the implementation, we’ll create a simple Python application to securely manage employee information, storing and querying sensitive data (such as names and salaries) using MongoDB’s advanced encryption.

### Prerequisites
The following table shows which MongoDB editions support CSFLE and Queryable Encryption:
| Product Name            | Automatic Encryption Support | Explicit Encryption Support |
|--------------------------|------------------------------|-----------------------------|
| **MongoDB Atlas**        | Yes                          | Yes                         |
| **MongoDB Enterprise Advanced** | Yes                    | Yes                         |
| **MongoDB Community Edition**   | No                     | Yes                         |

1. Download the Shared Encryption Library from [MongoDB’s Shared Library for CSFLE](https://www.mongodb.com/pt-br/docs/v8.0/core/csfle/reference/install-library/#std-label-csfle-reference-shared-library-download).
2. Install the Python module:
```bash
pip install pymongo
python -m pip install 'pymongo[encryption]'
```
### Default steps
#### Load local master key
To encrypt data, we use a master key, which can be loaded from a local file (for demonstration purposes only). The following code checks if the key file exists and creates it, if necessary.
To create a new local provider, create a key with this command:
For Shell Unix:
```bash
echo $(head -c 96 /dev/urandom | base64 | tr -d '\n')
```

For PowerShell:

```powershell
$r=[byte[]]::new(64);$g=[System.Security.Cryptography.RandomNumberGenerator]::Create();$g.GetBytes($r);[Convert]::ToBase64String($r)
```
>Note: A local key provider is not secure for production environments. For production, use a remote key management system (KMS) such as AWS KMS, Azure Key Vault, or Google Cloud KMS for enhanced security and access control. MongoDB also supports local KMIP providers, like HashiCorp Vault, for production-grade key management.
Place the key in the code below in YOUR_MASTER_KEY_HERE.
```python
def load_local_master_key(filename):
    if not os.path.exists(filename):
        key = "YOUR_MASTER_KEY_HERE"
        with open(filename, "w") as f:
            f.write(key)
    with open(filename, "r") as f:
        return f.read()
```

### Implementing Queryable Encryption with Python
#### Define the variables

Define the variables for the MongoDB connection string, database, and collection names, as well as the key vault namespace and KMS providers.
> Note: Replace URI with your MongoDB Atlas URI.
```python
uri = "<URI>"
key_vault_namespace = "encryption.__keyVault"
encrypted_database_name = "employee_data"
encrypted_collection_name = "employee_salary"

local_master_key_file = "local_master_key.txt"
local_master_key = load_local_master_key(local_master_key_file)

kms_providers = {"local": {"key": local_master_key}}
encrypted_fields_map = get_encrypted_fields_map()
```
### Define encrypted fields
The getEncryptedFieldsMap function specifies which fields will be encrypted and the allowed query types. In this example, name supports equality queries, and salary supports range queries.
```python
def get_encrypted_fields_map():
    return {
        "fields": [
            {
                "keyId": None,
                "path": "name",
                "bsonType": "string",
                "queries": [{"queryType": "equality"}],
            },
            {
                "keyId": None,
                "path": "salary",
                "bsonType": "int",
                "queries": [{"queryType": "range", "min": 0, "max": 1000000}],
            },
        ]
    }
```
### Configure automatic encryption options
Configure automatic encryption options with the AutoEncryptionOpts function, specifying the key vault namespace, KMS provider, and shared encryption library path.
> Note: In the AutoEncryptionOpts function, replace the path to the shared encryption library with the correct path on your system.
```python
auto_encryption_opts = AutoEncryptionOpts(
	kms_providers=kms_providers,
	key_vault_namespace=key_vault_namespace,
	crypt_shared_lib_path="./mongo_crypt_shared_v1-macos-arm64-enterprise-8.0.3/lib/mongo_crypt_v1.dylib",
	encrypted_fields_map=encrypted_fields_map,
)
```
### Create MongoDB client with automatic encryption
Create a MongoDB client with automatic encryption enabled.
```python
client = MongoClient(uri, auto_encryption_opts=auto_encryption_opts)
db = client[encrypted_database_name]
collection = db[encrypted_collection_name]
collection.drop()
```
![Creating the KeyVault collection][1]

![Creating the __safeContent___1 index][2]

### Insert encrypted documents
Insert example documents into the encrypted collection, with sensitive information protected by encryption.
```python
employees = [
	EmployeeDocument("Alice Johnson", "Software Engineer", "MongoDB", 100000, "USD", datetime.datetime(2019, 3, 5)),
	EmployeeDocument("Bob Smith", "Product Manager", "MongoDB", 150000, "USD", datetime.datetime(2018, 6, 15)),
	EmployeeDocument("Charlie Brown", "Data Analyst", "MongoDB", 200000, "USD", datetime.datetime(2020, 4, 20)),
	EmployeeDocument("Diana Prince", "HR Specialist", "MongoDB", 250000, "USD", datetime.datetime(2021, 12, 3)),
	EmployeeDocument("Evan Peters", "Marketing Coordinator", "MongoDB", 80000, "USD", datetime.datetime(2022, 10, 7)),
]

for employee in employees:
	collection.insert_one(employee.__dict__)
	print(f"Inserted document for {employee.name}")
```
![Inserting documents with encrypted fields][3]

Without the key, we can’t see the data. It worked.
### Perform encrypted queries
Finally, perform queries on the encrypted collection. The search_by_name function searches documents by name, while search_by_salary_range uses a salary range filter.

```python
def search_by_name(coll, name):
    filter = {"name": name}
    cursor = coll.find(filter)
    print(f"Documents with name '{name}':")
    for doc in cursor:
        print(json.dumps(doc, indent=4, default=json_util.default))

def search_by_salary_range(coll, min_salary, max_salary):
    filter = {"salary": {"$gte": min_salary, "$lte": max_salary}}
    cursor = coll.find(filter)
    print(f"Documents with salary between {min_salary} and {max_salary}:")
    for doc in cursor:
        print(json.dumps(doc, indent=4, default=json_util.default))

search_by_name(collection, "Alice Johnson")
search_by_salary_range(collection, 150000, 200000)
```
![Executing the Python script][4]

As we can see, in MongoDB version 8, we can also do this with data ranges.
You can check the entire code in the [GitHub repository](https://github.com/SamuelMolling/mongodb-lab/blob/main/queryable-encryption-py/main.py) and more informations in the MongoDB documentation.
## Implementing CSFLE with Python
### Define the variables
Define the variables for the MongoDB connection string, database, and collection names, as well as the key vault namespace and KMS providers.
Note: Replace URI with your MongoDB Atlas URI.
```python
uri = "<URI>"
local_master_key_file = "local_master_key.txt"
local_master_key = load_local_master_key(local_master_key_file)
kms_providers = setup_kms_providers(local_master_key)
key_vault_namespace = "encryption.__keyVault"

client = connect_mongo(uri)
database_name = "employee_data"
collection_name = "employee_salary"
coll = client[database_name][collection_name]
coll.drop()
key_vault_coll = client["encryption"]["__keyVault"]
ensure_key_vault_index(key_vault_coll)
```
### Ensuring Key Vault index
We ensure the keyAltNames index exists in the Key Vault, allowing alternate keys to be unique and filtered.
```python
def ensure_key_vault_index(key_vault_coll):
    index_name = "keyAltNames_1"
    indexes = key_vault_coll.index_information()
    if index_name not in indexes:
        key_vault_coll.create_index(
            [("keyAltNames", 1)], unique=True, partialFilterExpression={"keyAltNames": {"$exists": True}}
        )
        print("Index created in key vault.")
    else:
        print("Index already exists in key vault.")
```
### Generating and reusing data key
The ensure_data_key function checks if a specific data key already exists. If it doesn’t, it creates a new one.
```python
def ensure_data_key(client_encryption, key_vault_coll, key_alt_name):
    existing_key = key_vault_coll.find_one({"keyAltNames": key_alt_name})
    if existing_key:
        print("Data key already exists. Reusing existing key.")
        return existing_key["_id"]

    print("Creating new data key.")
    data_key_id = client_encryption.create_data_key("local", key_alt_names=[key_alt_name])
    return data_key_id
```
### Encrypting the salary
The encrypt_salary function converts the salary to cents, prepares the value for encryption, and encrypts it using the configured MongoDB encryption algorithm.
```python
def encrypt_salary(client_encryption, data_key_id, salary):
    salary_in_cents = int(salary * 100)
    encrypted_salary = client_encryption.encrypt(
        value=salary_in_cents,
        algorithm="AEAD_AES_256_CBC_HMAC_SHA_512-Deterministic",
        key_id=data_key_id
    )
    return encrypted_salary
```
![Creating the KeyVault collection][5]

![Creating the __safeContent___1 index][6]

### Inserting encrypted documents
After encrypting the salary, we insert each employee document into the collection. The document includes fields such as Name, Position, Company, Currency, Start Date, and the encrypted salary.
```python
def insert_employee_doc(coll, name, position, company, salary_encrypted, currency, start_date):
    employee_doc = {
        "name": name,
        "position": position,
        "company": company,
        "salary": salary_encrypted,
        "currency": currency,
        "startDate": start_date
    }
    coll.insert_one(employee_doc)
```
```python
employees = [
	{"name": "Alice Johnson", "position": "Software Engineer", "company": "MongoDB", "salary": 50000, "currency": "USD", "start_date": datetime(2007, 2, 3)},
	{"name": "Bob Smith", "position": "Product Manager", "company": "MongoDB", "salary": 70000, "currency": "USD", "start_date": datetime(2009, 3, 14)},
]
for emp in employees:
	encrypted_salary = encrypt_salary(client_encryption, data_key_id, emp["salary"])
	insert_employee_doc(coll, emp["name"], emp["position"], emp["company"], encrypted_salary, emp["currency"], emp["start_date"])
```

![Inserting documents with encrypted fields][7]

Without the key, we can’t see the data. It worked.
### Querying and decrypting documents
To query and view the encrypted salary field, we use find_all_and_decrypt_salaries, which retrieves all documents, decrypts the salary, and displays the data.

```python
def find_all_and_decrypt_salaries(coll, client_encryption):
    for doc in coll.find():
        encrypted_salary = doc["salary"]
        decrypted_salary = client_encryption.decrypt(encrypted_salary)
        salary_in_dollars = decrypted_salary / 100.0
        print(f"Employee: {doc['name']}, Salary: {salary_in_dollars} USD")
```
For comparison, the find_all_without_decryption function retrieves the same documents but without decrypting the salaries.
```python
def find_all_without_decryption(coll):
    for doc in coll.find():
        print(f"Employee: {doc['name']}, Encrypted Salary: {doc['salary']}")
```
The result is:

![Script output decrypting results][8]

Let’s zoom in and see the difference between two objects, one encrypted and the other not.

![Script output without decrypting results][9]

As we can see in this image, the real value of the object that is used without encryption comes in a format that is unreadable to us without the encryption key.
You can check the entire code in the [GitHub repository](https://github.com/SamuelMolling/mongodb-lab/blob/main/csfle-py/main.py) and more information in the MongoDB documentation.

## Conclusion

CSFLE and Queryable Encryption are advanced encryption solutions in MongoDB, providing distinct methods for protecting sensitive data and enabling secure queries. CSFLE is ideal for cases where client-side control and equality queries are sufficient, while Queryable Encryption is effective for scenarios requiring range queries, with future support for more advanced query types, like substring searches, in development. With MongoDB 8.0’s range query support, Queryable Encryption becomes even more powerful and flexible for securing sensitive data.

Using Python, you can easily configure and use these encryption solutions to enhance MongoDB application security, meeting strict data compliance and security requirements.

For more MongoDB resources and tools, visit the [MongoDB Developer Center](https://www.mongodb.com/developer/) to explore additional articles.





  [1]: https://images.contentstack.io/v3/assets/blt39790b633ee0d5a7/blt0c0464ccf1629bbc/6765a01cc24a552455fd5dbc/encryption-key-vault.png
  [2]: https://images.contentstack.io/v3/assets/blt39790b633ee0d5a7/blta02f4625d035f595/6765a0423c253807bf75704b/employee-salary.png
  [3]: https://images.contentstack.io/v3/assets/blt39790b633ee0d5a7/blt9503129b3ec9577a/6765a094ccead14ed83f3960/data-salary.png
  [4]: https://images.contentstack.io/v3/assets/blt39790b633ee0d5a7/blt29ef19a71b19aea7/6765a0bf558615384a6a298e/python-script.png
  [5]: https://images.contentstack.io/v3/assets/blt39790b633ee0d5a7/blt2914d0e4359a9d08/6765a107ccead1b9983f3968/encryption-key-vault.png
  [6]: https://images.contentstack.io/v3/assets/blt39790b633ee0d5a7/blt4db56150d4654126/6765a134b1ed3c36d0dc4e84/employee-salary.png
  [7]: https://images.contentstack.io/v3/assets/blt39790b633ee0d5a7/bltb177254f20f0231f/6765a15d3e03b23ecb33745e/inserting-documents.png
  [8]: https://images.contentstack.io/v3/assets/blt39790b633ee0d5a7/blt02423caaf3cadf93/6765a17d05ab076b692c313a/script-output.png
  [9]: https://images.contentstack.io/v3/assets/blt39790b633ee0d5a7/blt4f1b92f8f3acd3a2/6765a19a775da775911f2c3e/script-output-results.png

Protecting stored data is only a start. You need a solution that works during processing, allowing meaningful searches. Enter CSFLE and Queryable Encryption!

Encryption in Use: The Magic of MongoDB for Secure Queries

This article is for anyone who intends to use or already uses infrastructure as code (IaC) on the MongoDB Atlas platform or wants to learn more about it.

MongoDB Atlas With Terraform - Cluster and Backup Policies

In this tutorial, I will show how to create a user for the MongoDB database in Atlas using Terraform and how to store this credential securely in HashiCorp Vault. We saw in the previous article, [MongoDB Atlas With Terraform - Cluster and Backup Policies](https://www.mongodb.com/developer/products/atlas/atlas-terraform-cluster-backup-policies/), how to create a cluster with configured backup policies. Now, we will go ahead and create our first user. If you haven't seen the previous articles, I suggest you look to understand how to get started.

This article is for anyone who intends to use or already uses infrastructure as code (IaC) on the MongoDB Atlas platform or wants to learn more about it.

Everything we do here is contained in the provider/resource documentation: 

 - [mongodbatlas_database_user](https://registry.terraform.io/providers/mongodb/mongodbatlas/latest/docs/resources/database_user)
- [vault_kv_secret_v2](https://registry.terraform.io/providers/hashicorp/vault/latest/docs/resources/kv_secret_v2)

> Note: We will not use a backend file. However, for productive implementations, it is extremely important and safer to store the state file in a remote location such as [S3](https://developer.hashicorp.com/terraform/language/settings/backends/s3), [GCS](https://developer.hashicorp.com/terraform/language/settings/backends/gcs), [Azurerm](https://developer.hashicorp.com/terraform/language/settings/backends/azurerm), etc.

## Creating a User
At this point, we will create our first user using Terraform in MongoDB Atlas and store the URI to connect to my cluster in HashiCorp Vault. For those unfamiliar, [HashiCorp Vault](https://www.hashicorp.com/products/vault) is a secrets management tool that allows you to securely store, access, and manage sensitive credentials such as passwords, API keys, certificates, and more. It is designed to help organizations protect their data and infrastructure in complex, distributed IT environments. In it, we will store the connection URI of the user that will be created with the cluster we created in the [last article](https://www.mongodb.com/developer/products/atlas/atlas-terraform-cluster-backup-policies/).

Before we begin, make sure that all the prerequisites mentioned in the previous article are properly configured: Install [Terraform](https://developer.hashicorp.com/terraform/tutorials/aws-get-started/install-cli), create an API key in MongoDB Atlas, and set up a project and a cluster in Atlas. These steps are essential to ensure the success of creating your database user.

### Configuring HashiCorp Vault to run on Docker
The first step is to run HashiCorp Vault so that we can test our module. It is possible to run Vault on Docker Local. If you don't have Docker installed, you can [download it](https://docs.docker.com/engine/install/). After downloading Docker, we will download the image we want to run — in this case, from Vault. To do this, we will execute a command in the terminal `docker pull vault:1.13.3` or download using Docker Desktop.

![Looking for the image in docker][1]

Now, we will create a container from this image. Click on the image and click on **Run**. After this, a box will open where we only need to map the port from our computer to the container. In this case, I will use port 8200 which is the Vault's default port. Click **Run**.

![Screen to configure docker port][2]

The container will start running. If we go to our browser and enter the URL `localhost:8200/`, the Vault login screen will appear.

![Vault sign in screen][3]

To access the Vault, we will use the Root Token that is generated when we create the container.

![Log screen containing token][4]

Now, we will log in. After opening, we will create a new KV-type engine just to illustrate it a little better. Click **Secrets Engines** -> Enable new Engine -> Generic KV and click **Next**.

![Secrets engine selection screen][5]

In Path, put `kv/my_app` and click on **Enable Engine**. Now, we have our Vault configured and working.

## Terraform provider configuration for MongoDB Atlas and HashiCorp Vault
The next step is to configure the Terraform provider. This will allow Terraform to communicate with the MongoDB Atlas and Vault API to manage resources. Add the following block of code to your providers.tf file:

```
provider "mongodbatlas" {}
provider "vault" {
  address = "http://localhost:8200"
  token = "hvs.brmNeZd31NwEmyky1uYI2wvY"
  skip_child_token = true
}
```

In the previous article, we configured the Terraform provider by placing our public and private keys in environment variables. We will continue in this way. We will add a new provider, the Vault. In it, we will configure the Vault address, the authentication token, and the skip_child_token parameter so that we can authenticate to the Vault.
> Note: It is not advisable to specify the authentication token in a production environment. Use one of the authentication methods recommended by HashiCorp, such as app_role. You can evaluate the options [in Terraform’s docs](https://registry.terraform.io/providers/hashicorp/vault/latest/docs)

## Creating the Terraform version file
The version file continues to have the same purpose, as mentioned in other articles, but we will add the version of the Vault provider as something new.

```
terraform {
    required_version = ">= 0.12"
    required_providers {
        mongodbatlas = {
            source = "mongodb/mongodbatlas"
            version = "1.14.0"
        }
        vault = {
            source = "hashicorp/vault"
            version = "4.0.0"
        }
    }
}
```
### Defining the database user and vault resource
After configuring the version file and establishing the Terraform and provider versions, the next step is to define the user resource in MongoDB Atlas. This is done by creating a .tf file — for example, main.tf — where we will create our module. As we are going to make a module that will be reusable, we will use variables and default values so that other calls can create users with different permissions, without having to write a new module.

```
# ------------------------------------------------------------------------------
# RANDOM PASSWORD
# ------------------------------------------------------------------------------
resource "random_password" "default" {
    length = var.password_length
    special = false
}

# ------------------------------------------------------------------------------
# DATABASE USER
# ------------------------------------------------------------------------------
resource "mongodbatlas_database_user" "default" {
    project_id = data.mongodbatlas_project.default.id
    username = var.username
    password = random_password.default.result
    auth_database_name = var.auth_database_name

    dynamic "roles" {
        for_each = var.roles
        content {
            role_name = try(roles.value["role_name"], null)
            database_name = try(roles.value["database_name"], null)
            collection_name = try(roles.value["collection_name"], null)
        }
    }

    dynamic "scopes" {
        for_each = var.scope
        content {
            name = scopes.value["name"]
            type = scopes.value["type"]
        }
    }


    dynamic "labels" {
        for_each = local.tags
        content {
            key = labels.key
            value = labels.value
        }
    }
}

resource "vault_kv_secret_v2" "default" {
    mount = var.vault_mount
    name = var.secret_name
    data_json = jsonencode(local.secret)
}
```

At the beginning of the file, we have the random_password resource that is used to generate a random password for our user. In the mongodbatlas_database_user resource, we will specify our user details. We are placing some values as variables as done in other articles, such as name and auth_database_name with a default value of admin. Below, we create three dynamic blocks: roles, scopes, and labels. For roles, it is a list of maps that can contain the name of the role (read, readWrite, or some other), the database_name, and the collection_name. These values can be optional if you create a user with atlasAdmin permission, as in this case, it does not. It is necessary to specify a database or collection, or if you wanted, to specify only the database and not a specific collection. We will do an example. For the scopes block, the type is a DATA_LAKE or a CLUSTER. In our case, we will specify a cluster, which is the name of our created cluster, the demo cluster. And the labels serve as tags for our user.

Finally, we define the vault_kv_secret_v2 resource that will create a secret in our Vault. It receives the mount where it will be created and the name of the secret. The data_json is the value of the secret; we are creating it in the locals.tf file that we will evaluate below. It is a JSON value — that is why we are encoding it.

In the variable.tf file, we create variables with default values:
```
variable "project_name" {
    description = "The name of the Atlas project"
    type = string
}

variable "cluster_name" {
    description = "The name of the Atlas cluster"
    type = string
}

variable "password_length" {
    description = "The length of the password"
    type = number
    default = 20
}

variable "username" {
    description = "The username of the database user"
    type = string
}

variable "auth_database_name" {
    description = "The name of the database in which the user is created"
    type = string
    default = "admin"
}

variable "roles" {
    description = <<HEREDOC
    Required - One or more user roles blocks.
    HEREDOC
    type = list(map(string))
}

variable "scope" {
    description = "The scopes to assign to the user"
    type = list(object({
        name = string
        type = string
    }))
    default = []
}

variable "labels" {
    type = map(any)
    default = null
    description = "A JSON containing additional labels"
}

variable "uri_options" {
    type = string
    default = "retryWrites=true&w=majority&readPreference=secondaryPreferred"
    description = "A string containing additional URI configs"
}

variable "vault_mount" {
    description = "The mount point for the Vault secret"
    type = string
}

variable "secret_name" {
    description = "The name of the Vault secret"
    type = string
}

variable "application" {
    description = <<HEREDOC
    Optional - Key-value pairs that tag and categorize the cluster for billing and organizational purposes.
    HEREDOC
    type = string
}

variable "environment" {
    description = <<HEREDOC
    Optional - Key-value pairs that tag and categorize the cluster for billing and organizational purposes.
    HEREDOC
    type = string
}
```

We configured a file called **locals.tf** with the values for our Vault and the tags that were created, like the last article. The interesting thing here is that we are defining how our user's connection string will be assembled and saved in the Vault. We could only save the username and password too, but I personally prefer to save the URI. This way, I can specify some good practices such as defining connection tags, such as readPreference, without depending on the developer to place it in the application. In the code below, there are some text treatments so that the URI is correct. At the end, I create a variable called **secret** that has a URI key and receives the value of the created URI.

```terraform
locals {
    private_connection_srv = data.mongodbatlas_advanced_cluster.default.connection_strings.0.standard_srv
    cluster_uri = trimprefix(local.private_connection_srv, "mongodb+srv://")
    private_connection_string = "mongodb+srv://${mongodbatlas_database_user.default.username}:${random_password.default.result}@${local.cluster_uri}/${var.auth_database_name}?${var.uri_options}"

    secret = { "URI" = local.private_connection_string }

    tags = {
    name = var.application
    environment = var.environment
    }
}
```
In this article, we adopt the use of data sources in Terraform to establish a dynamic connection with existing resources, such as our MongoDB Atlas project and our cluster. Specifically, in the data.tf file, we define a data source, mongodbatlas_project and mongodbatlas_advanced_cluster, to access information about an existing project and cluster based on its name:

```terraform
data "mongodbatlas_project" "default" {
    name = var.project_name
}


data "mongodbatlas_advanced_cluster" "default" {
    project_id = data.mongodbatlas_project.default.id
    name = var.cluster_name
}
```

Finally, we define our variables file, terraform.tfvars:

```terraform
project_name = "project-test"
username = "usr_myapp"
application = "teste-cluster"
environment = "dev"
cluster_name = "cluster-demo"

roles = [{
        "role_name" = "readWrite",
        "database_name" = "db1",
        "collection_name" = "collection1"
    }, {
        "role_name" : "read",
        "database_name" : "db2"
}]

scope = [{
    name = "cluster-demo",
    type = "CLUSTER"
}]

secret_name = "MY_MONGODB_SECRET"
vault_mount = "kv/my_app"
```
These values defined in terraform.tfvars are used by Terraform to populate corresponding variables in your configuration. In it, we are specifying the user's scope, values for the Vault, and our user's roles. The user will have readWrite permission on db1 in collection1 and read permission on db2 in all collections for the demo cluster.

The file structure is as follows:

 - main.tf: In this file, we will define the main resource, the mongodbatlas_database_user and vault_kv_secret_v2, along with a random password generation. Here, you have configured the cluster and backup routines. 
 - provider.tf: This file is where we define the provider we are using, in our case, mongodbatlas and Vault.
 - terraform.tfvars: This file contains the variables that will be used in our module — for example, the user name and Vault information, among others.
 - variable.tf: Here, we define the variables mentioned in the terraform.tfvars file, specifying the type and, optionally, a default value.
 - version.tf: This file is used to specify the version of Terraform and the providers we are using.
 - data.tf: Here, we specify a datasource that will bring us information about our project and created cluster. We will search for its name and for our module, it will give us the project ID and cluster information such as its connection string.
 - locals.tf: We specify example tags to use in our user and treatments to create the URI in the Vault.

Now is the time to apply. =D

We run a Terraform init in the terminal in the folder where the files are located so that it downloads the providers, modules, etc…

> Note: Remember to export the environment variables with the public and private key.

```terraform
export MONGODB_ATLAS_PUBLIC_KEY="your_public_key"
export MONGODB_ATLAS_PRIVATE_KEY=your_private_key"
```

Now, we run init and then plan, as in previous articles.

We assess that our plan is exactly what we expect and run the apply to create it.

When running the `terraform apply` command, you will be prompted for approval with `yes` or `no`. Type `yes`.

Now, let's look in Atlas to see if the user was created successfully...

![User displayed in database access][6]

![Access permissions displayed][7]

Let's also look in the Vault to see if our secret was created.

![MongoDB secret URI][8]

It was created successfully! Now, let's test if the URI is working perfectly.

This is the format of the URI that is generated:
`mongosh "mongodb+srv://usr_myapp:<password>@<clusterEndpoint>/admin?retryWrites=true&majority&readPreference=secondaryPreferred"`

![Mongosh login ][9]

We connect and will make an insertion to evaluate whether the permissions are adequate — initially, in db1 in collection1.

![Command to insert to db and acknowledged][10]

Success! Now, in db3, make sure it will not have permission in another database.

![Access denied to unauthroized collection][11]
Excellent — permission denied, as expected.

We have reached the end of this series of articles about MongoDB. I hope they were enlightening and useful for you!

To learn more about MongoDB and various tools, I invite you to visit the [Developer Center](https://www.mongodb.com/developer/) to read the other articles.


  [1]: https://images.contentstack.io/v3/assets/blt39790b633ee0d5a7/blt3adf134a1cc654f8/661cefe94c473591d2ee4ca7/image2.png
  [2]: https://images.contentstack.io/v3/assets/blt39790b633ee0d5a7/blt01b4534800d306c0/661cefe912f2752a7aeff578/image8.png
  [3]: https://images.contentstack.io/v3/assets/blt39790b633ee0d5a7/bltabb003cbf7efb6fa/661cefe936f462858244ec50/image1.png
  [4]: https://images.contentstack.io/v3/assets/blt39790b633ee0d5a7/blt2c34530c41490c28/661cefe90aca6b12ed3273b3/image7.png
  [5]: https://images.contentstack.io/v3/assets/blt39790b633ee0d5a7/blt867d41655e363848/661cefe931ff3a1d35a41344/image9.png
  [6]: https://images.contentstack.io/v3/assets/blt39790b633ee0d5a7/bltcdaf7406e85f79d5/661cefe936f462543444ec54/image3.png
  [7]: https://images.contentstack.io/v3/assets/blt39790b633ee0d5a7/bltbb0d0b37cd3e7e23/661cefe91c390d5d3c98ec3d/image10.png
  [8]: https://images.contentstack.io/v3/assets/blt39790b633ee0d5a7/blt0dc4c9ad575c4118/661cefe9ba18470cf69b8c14/image6.png
  [9]: https://images.contentstack.io/v3/assets/blt39790b633ee0d5a7/bltc6c61799f656701f/661cf85d4c4735186bee4ce7/image5.png
  [10]: https://images.contentstack.io/v3/assets/blt39790b633ee0d5a7/bltdd01eaae2a3d9d24/661cefe936f462254644ec58/image11.png
  [11]: https://images.contentstack.io/v3/assets/blt39790b633ee0d5a7/blt05fe248cb479b18a/661cf85d4c47359b89ee4ce5/image4.png

Learn how to create a user for MongoDB and secure their credentials securely in Hashicorp Vault.

MongoDB Atlas With Terraform: Database Users and Vault

In this tutorial, I will show you how to start using MongoDB Atlas with Terraform and create some simple resources. This first part is simpler and more introductory, but in the next article, I will explore more complex items and how to connect the creation of several resources into a single module. The tutorial is aimed at people who want to maintain their infrastructure as code (IaC) in a standardized and simple way. If you already use or want to use IaC on the MongoDB Atlas platform, this article is for you.

What are modules?

They are code containers for multiple resources that are used together. They serve several important purposes in building and managing infrastructure as code, such as:

 1. Code reuse.
 2. Organization.
 3. Encapsulation.
 4. Version management.
 5. Ease of maintenance and scalability.
 6. Sharing in the community.

Everything we do here is contained in the [provider/resource documentation](https://registry.terraform.io/providers/mongodb/mongodbatlas/latest/docs/resources/project).

> Note: We will not use a backend file. However, for productive implementations, it is extremely important and safer to store the state file in a remote location such as an [S3](https://developer.hashicorp.com/terraform/language/settings/backends/s3), [GCS](https://developer.hashicorp.com/terraform/language/settings/backends/gcs), [Azurerm](https://developer.hashicorp.com/terraform/language/settings/backends/azurerm), etc…

## Creating a project

In this first step, we will dive into the process of creating a project using Terraform. Terraform is a powerful infrastructure-as-code tool that allows you to manage and provision IT resources in an efficient and predictable way. By using it in conjunction with MongoDB Atlas, you can automate the creation and management of database resources in the cloud, ensuring a consistent and reliable infrastructure.

To get started, you'll need to install [Terraform](https://developer.hashicorp.com/terraform/tutorials/aws-get-started/install-cli) in your development environment. This step is crucial as it is the basis for running all the scripts and infrastructure definitions we will create. After installation, the next step is to configure Terraform to work with MongoDB Atlas. You will need an API key that has permission to create a project at this time.

To create an API key, you must:
 1. Select **Access Manager** at the top of the page, and click **Organization Access**. 
 2. Click **Create API Key**.
    ![Organization Access Manager for your organization][1]
 3. Enter a brief description of the API key and the necessary permission. In this case, I put it as Organization Owner. After that, click **Next**.
    ![Screen to create your API key][2]
 4. Your API key will be displayed on the screen.
    ![Screen with information about your API key][3]
 5. Release IP in the Access List (optional): If you have enabled your organization to use API keys, the requestor's IP must be released in the Access List; you must include your IP in this list. To validate whether it is enabled or not, go to **Organization Settings -> Require IP Access List** for the Atlas Administration API. In my case, it is disabled, as it is just a demonstration, but in case you are using this in an organization, I strongly advise you to enable it.
    ![Validate whether the IP Require Access List for APIs is enabled in Organization Settings][4]

After creating an API key, let's start working with Terraform. You can use the IDE of your choice; I will be using VS Code. Create the files within a folder. The files we will need at this point are:
 - main.tf: In this file, we will define the main resource, `mongodbatlas_project`. Here, you will configure the project name and organization ID, as well as other specific settings, such as teams, limits, and alert settings.
 - provider.tf: This file is where we define the provider we are using — in our case, `mongodbatlas`. Here, you will also include the access credentials, such as the API key.
 - terraform.tfvars: This file contains the variables that will be used in our project — for example, the project name, team information, and limits, among others.
 - variable.tf: Here, we define the variables mentioned in the terraform.tfvars file, specifying the type and, optionally, a default value.
 - version.tf: This file is used to specify the version of Terraform and the providers we are using.

The main.tf file is the heart of our Terraform project. In it, you start with the data source declaration `mongodbatlas_roles_org_id` to obtain the `org_id`, which is essential for creating the project.
Next, you define the `mongodbatlas_project` resource with several settings. Here are some examples:
 - `name` and `org_id` are basic settings for the project name and organization ID.
 - Dynamic blocks are used to dynamically configure teams and limits, allowing flexibility and code reuse.
 - Other settings, like `with_default_alerts_settings` and `is_data_explorer_enabled`, are options for customizing the behavior of your MongoDB Atlas project.

In the main.tf file, we will then add our project resource, called `mongodbatlas_project`.

```tf
data "mongodbatlas_roles_org_id" "org" {}

resource "mongodbatlas_project" "default" {
 name   = var.name
 org_id = data.mongodbatlas_roles_org_id.org.org_id

 dynamic "teams" {
   for_each = var.teams
   content {
     team_id    = teams.value.team_id
     role_names = teams.value.role_names
   }
 }

 dynamic "limits" {
   for_each = var.limits
   content {
     name  = limits.value.name
     value = limits.value.value
   }
 }

 with_default_alerts_settings                     = var.with_default_alerts_settings
 is_collect_database_specifics_statistics_enabled = var.is_collect_database_specifics_statistics_enabled
 is_data_explorer_enabled                         = var.is_data_explorer_enabled
 is_extended_storage_sizes_enabled                = var.is_extended_storage_sizes_enabled
 is_performance_advisor_enabled                   = var.is_performance_advisor_enabled
 is_realtime_performance_panel_enabled            = var.is_realtime_performance_panel_enabled
 is_schema_advisor_enabled                        = var.is_schema_advisor_enabled
}
```

In the provider file, we will define the provider we are using and the API key that will be used. As we are just testing, I will specify the API key as a variable that we will input into our code. However, when you are using it in production, you will not want to pass the API key in the code in exposed text, so it is possible to pass it through environment variables or even AWS secret manager.
```tf
provider "mongodbatlas" {
   public_key  = var.atlas_public_key
   private_key = var.atlas_private_key
}
```

In the variable.tf file, we will specify the variables that we are waiting for a user to pass. As I mentioned earlier, the API key is an example.
```tf
variable "name" {
 description = <<HEREDOC
   The name of the project you want to create.
   HEREDOC
 type        = string
}

variable "teams" {
 description = <<HEREDOC
   The list of teams that belong to the project.
   The roles can be:
   Organization:
       ORG_OWNER
       ORG_MEMBER
       ORG_GROUP_CREATOR
       ORG_BILLING_ADMIN
       ORG_READ_ONLY
   Project:
       GROUP_CLUSTER_MANAGER
       GROUP_DATA_ACCESS_ADMIN
       GROUP_DATA_ACCESS_READ_ONLY
       GROUP_DATA_ACCESS_READ_WRITE
       GROUP_OWNER
       GROUP_READ_ONLY
   HEREDOC
 type = list(object({
   team_id    = string
   role_names = list(string)
 }))
 default = []
}

variable "is_collect_database_specifics_statistics_enabled" {
 description = <<HEREDOC
   If true, Atlas collects and stores database-specific statistics for the specified project.
   HEREDOC
 type        = bool
 default     = true
}

variable "is_data_explorer_enabled" {
 description = <<HEREDOC
   If true, Atlas enables Data Explorer for the specified project.
   HEREDOC
 type        = bool
 default     = false
}

variable "is_extended_storage_sizes_enabled" {
 description = <<HEREDOC
   If true, Atlas enables extended storage sizes for the specified project.
   HEREDOC
 type        = bool
 default     = true
}

variable "is_performance_advisor_enabled" {
 description = <<HEREDOC
   If true, Atlas enables Performance Advisor for the specified project.
   HEREDOC
 type        = bool
 default     = true
}


variable "is_realtime_performance_panel_enabled" {
 description = <<HEREDOC
   If true, Atlas enables the Real Time Performance Panel for the specified project.
   HEREDOC
 type        = bool
 default     = true
}


variable "is_schema_advisor_enabled" {
 description = <<HEREDOC
   If true, Atlas enables Schema Advisor for the specified project.
   HEREDOC
 type        = bool
 default     = true
}


variable "with_default_alerts_settings" {
 description = <<HEREDOC
   If true, Atlas enables default alerts settings for the specified project.
   HEREDOC
 type        = bool
 default     = true
}


variable "limits" {
 description = <<HEREDOC
   Allows one to configure a variety of limits to a Project. The limits attribute is optional.
   https://www.mongodb.com/docs/atlas/reference/api-resources-spec/v2/#tag/Projects/operation/setProjectLimit
   HEREDOC
 type = list(object({
   name  = string
   value = string
 }))


 default = []
}


variable "atlas_public_key" {
 description = <<HEREDOC
   The public key of the Atlas user you want to use to create the project.
   HEREDOC
 type        = string
}


variable "atlas_private_key" {
 description = <<HEREDOC
   The private key of the Atlas user you want to use to create the project.
   HEREDOC
 type        = string
}
```

As you can see, we are placing several variables. The idea is that we can reuse the module to create different projects that have different specifications. For example, one needs the data from the databases to be visible on the Atlas Platform, so we put the `is_data_explorer_enabled` variable as `True`. On the other hand, if, for some security reason, the company does not want users to use the platform for data visualization, it is possible to specify it as `False`, and the Collections button that we have when we enter the cluster within Atlas will disappear.

In the versions.tf file, we specify the version of Terraform we use and the necessary providers with their respective versions. This configuration is crucial so that the code always runs in the same version, avoiding inconsistencies and incompatibilities that may arise with updates.

Here is what our providers file will look like:
```tf
terraform {
   required_version = ">= 0.12"
   required_providers {
       mongodbatlas = {
           source = "mongodb/mongodbatlas"
           version = "1.14.0"
       }
   }
}
```
 - `required_version = ">= 0.12"`: This line specifies that your Terraform project requires, at a minimum, Terraform version 0.12. By using >=, you indicate that any version of Terraform from 0.12 onward is compatible with your project. This offers some flexibility by allowing team members and automation systems to use newer versions of Terraform as long as they are not older than 0.12.
 - `required_providers`: This section lists the providers required for your Terraform project. In your case, you are specifying the mongodbatlas provider.
 - `source = "mongodb/mongodbatlas"`: This defines the source of the mongodbatlas provider. Here, mongodb/mongodbatlas is the official identifier of the MongoDB Atlas provider in the Terraform Registry.
 - `version = "1.14.0":` This line specifies the exact version of the mongodbatlas provider that your project will use, which is version 1.14.0. Unlike Terraform configuration, where we specify a minimum version, here you are defining a provider-specific version. This ensures that everyone using your code will work with the same version of the provider, avoiding discrepancies and issues related to version differences.

Finally, we have the variable file that will be included in our code, .tfvars.
```tf
name = "project-test"
atlas_public_key = "YOUR PUBLIC KEY"
atlas_private_key = "YOUR PRIVATE KEY"
```

We are specifying the value of the name variable, which is the name of the project and the public/private key of our provider. You may wonder, "Where are the other variables that we specified in the main.tf and variable.tf files?" The answer is: These variables were specified with a default value within the variable.tf file — for example, the limits value:
```tf
variable "limits" {
 description = <<HEREDOC
   Allows one to configure a variety of limits to a Project. The limits attribute is optional.
   https://www.mongodb.com/docs/atlas/reference/api-resources-spec/v2/#tag/Projects/operation/setProjectLimit
   HEREDOC
 type = list(object({
   name  = string
   value = string
 }))
 default = []
}
```

We are saying that if no information is passed in .tfvars, the default value is empty. This means that it will not create any limit rules for our project. If we want to specify a limit, we just put the following variable in .tfvars:
```tf
name              = "project-test"
atlas_public_key  = "YOUR PUBLIC KEY"
atlas_private_key = "YOUR PRIVATE KEY"
limits = [{
 "name" : "atlas.project.deployment.clusters",
 "value" : 26
 }, {
 "name" : "atlas.project.deployment.nodesPerPrivateLinkRegion",
 "value" : 51
}]
```

Now is the time to apply. =D

We run a `terraform init` in the terminal in the folder where the files are located so that it downloads the providers, modules, etc…

Now that init has worked, let's run the plan and evaluate what will happen. You can run the plan, like this `terraform plan`:
```console
samuelmolling@Samuels-MacBook-Pro project % terraform plan
data.mongodbatlas_roles_org_id.org: Reading...
data.mongodbatlas_roles_org_id.org: Read complete after 0s [id=5d7072c9014b769c4bd89f60]

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
   + create

Terraform will perform the following actions:

   # mongodbatlas_project.default will be created
   + resource "mongodbatlas_project" "default" {
       + cluster_count = (known after apply)
       + created = (known after apply)
       + id = (known after apply)
       + is_collect_database_specifics_statistics_enabled = true
       + is_data_explorer_enabled = false
       + is_extended_storage_sizes_enabled = true
       + is_performance_advisor_enabled = true
       + is_realtime_performance_panel_enabled = true
       + is_schema_advisor_enabled = true
       + name = "project-test"
       + org_id = "5d7072c9014b769c4bd89f60"
       + with_default_alerts_settings = true

       + limits {
           + current_usage = (known after apply)
           + default_limit = (known after apply)
           + maximum_limit = (known after apply)
           + name = "atlas.project.deployment.clusters"
           + value = 26
         }
       + limits {
           + current_usage = (known after apply)
           + default_limit = (known after apply)
           + maximum_limit = (known after apply)
           + name = "atlas.project.deployment.nodesPerPrivateLinkRegion"
           + value = 51
         }
     }

Plan: 1 to add, 0 to change, 0 to destroy.

───────────────────────────────────────── ───────── ───────────────────────────────────────── ───────── ───────────────────────────────────────── ───────── ───────────────────────────────────────── ────

Note: You didn't use the -out option to save this plan, so Terraform can't guarantee to take exactly these actions if you run `terraform apply` now.
```

Show! It was exactly the output we expected to see: the creation of a project resource, with some limits and `is_data_explorer_enabled` set to False. Let's apply this!

When you run the command `terraform apply`, you will be asked for approval with `yes` or `no`. Type `yes`.
```console
samuelmolling@Samuels-MacBook-Pro project % terraform apply
data.mongodbatlas_roles_org_id.org: Reading...
data.mongodbatlas_roles_org_id.org: Read complete after 0s [id=5d7072c9014b769c4bd89f60]

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
   + create

Terraform will perform the following actions:

   # mongodbatlas_project.default will be created
   + resource "mongodbatlas_project" "default" {
       + cluster_count = (known after apply)
       + created = (known after apply)
       + id = (known after apply)
       + is_collect_database_specifics_statistics_enabled = true
       + is_data_explorer_enabled = false
       + is_extended_storage_sizes_enabled = true
       + is_performance_advisor_enabled = true
       + is_realtime_performance_panel_enabled = true
       + is_schema_advisor_enabled = true
       + name = "project-test"
       + org_id = "5d7072c9014b769c4bd89f60"
       + with_default_alerts_settings = true

       + limits {
           + current_usage = (known after apply)
           + default_limit = (known after apply)
           + maximum_limit = (known after apply)
           + name = "atlas.project.deployment.clusters"
           + value = 26
         }
       + limits {
           + current_usage = (known after apply)
           + default_limit = (known after apply)
           + maximum_limit = (known after apply)
           + name = "atlas.project.deployment.nodesPerPrivateLinkRegion"
           + value = 51
         }
     }

Plan: 1 to add, 0 to change, 0 to destroy.

Do you want to perform these actions?
   Terraform will perform the actions described above.
   Only 'yes' will be accepted to approve.

   Enter a value: yes

mongodbatlas_project.default: Creating...
mongodbatlas_project.default: Creation complete after 9s [id=659ed54eb3343935e840ce1f]

Apply complete! Resources: 1 added, 0 changed, 0 destroyed.
```

Now, let's look at Atlas to see if the project was created successfully...

![MongoDB Atlas view of our project][5]

It worked!

In this tutorial, we saw how to create our first API key. We created a project using Terraform and our first module. In an upcoming article, we’ll look at how to create a cluster and user using Terraform and Atlas.

To learn more about MongoDB and various tools, I invite you to enter the [Developer Center](https://www.mongodb.com/developer/) to read the other articles.

  [1]: https://images.contentstack.io/v3/assets/blt39790b633ee0d5a7/blt4febcb777b3987da/65af88cec3bf069197fe083c/image1.png
  [2]: https://images.contentstack.io/v3/assets/blt39790b633ee0d5a7/bltb484eb33048284e2/65af88cec3bf061522fe0840/image3.png
  [3]: https://images.contentstack.io/v3/assets/blt39790b633ee0d5a7/blt322b2cb5a1505843/65af88cedf8733067dcdc1f3/image5.png
  [4]: https://images.contentstack.io/v3/assets/blt39790b633ee0d5a7/bltf51cdf161ede323e/65af88ce18af000fd20a9124/image2.png
  [5]: https://images.contentstack.io/v3/assets/blt39790b633ee0d5a7/blt7ff9636ae3df9340/65af88cec7bb161d62a2a593/image4.png

Learn how to get started with organising your MongoDB deployment with Terraform, using code to build and maintain your infrastructure.

Samuel Molling

Content by Samuel Molling

Encryption in Use: The Magic of MongoDB for Secure Queries

MongoDB Atlas With Terraform - Cluster and Backup Policies

MongoDB Atlas With Terraform: Database Users and Vault

MongoDB Atlas with Terraform