Use Explicit Encryption

On this page

Overview

Before You Get Started
Procedure
Create a Customer Master Key
Create a Unique Index on your Key Vault collection
Create your Data Encryption Keys and Encrypted Collection
Configure your MongoClient for Encrypted Reads and Writes
Insert a Document with Encrypted Fields
Retrieve Your Document with Encrypted Fields
Learn More

Queryable Encryption with equality queries is generally available (GA) in MongoDB 7.0 and later. The Queryable Encryption Public Preview, released in version 6.0, is no longer supported. Data encrypted using the Public Preview is incompatible with the feature release. For more information, see Compatibility Changes in MongoDB 7.0.

Overview

This guide shows you how to encrypt a document with explicit encryption and a MongoDB driver.

After completing this guide, you should be able to configure a driver to encrypt fields in a document using explicit encryption. With this knowledge, you should be able to create a client application that uses explicit encryption. with automatic decryption.

Important

Do Not Use this Sample Application In Production

Because the instructions in this tutorial include storing an encryption key in an insecure environment, you should not use an unmodified version of this application in production. Using this application in production risks unauthorized access to the encryption key or loss of the key needed to decrypt your data. The purpose of this tutorial is to demonstrate how to use Queryable Encryption without needing to set up a Key Management System.

You can use a Key Management System to securely store your encryption key in a production environment. A KMS is a remote service that securely stores and manages your encryption keys. To learn how to set up a Queryable Encryption enabled application that uses a KMS, see the Queryable Encryption Tutorials.

Before You Get Started

To complete and run the code in this guide, you need to set up your development environment as shown in the Installation Requirements page.

Tip

See: Full Application

To see the complete code for the application you make in this guide, select the tab corresponding to your preferred MongoDB driver and follow the provided link:

Complete Node.js Application

Complete Python Application

Complete Java Application

Complete Go Application

Complete C# Application

Procedure

Create a Customer Master Key

You must create a Customer Master Key (CMK) to perform Queryable Encryption.

Create a 96-byte Customer Master Key and save it to the file master-key.txt:

using (var randomNumberGenerator = System.Security.Cryptography.RandomNumberGenerator.Create())
{
    var bytes = new byte[96];
    randomNumberGenerator.GetBytes(bytes);
    var localMasterKeyBase64Write = Convert.ToBase64String(bytes);
    File.WriteAllText("master-key.txt", localMasterKeyBase64Write);
}

func localMasterKey() []byte {
	key := make([]byte, 96)
	if _, err := rand.Read(key); err != nil {
		log.Fatalf("Unable to create a random 96 byte data key: %v", err)
	}
	if err := ioutil.WriteFile("master-key.txt", key, 0644); err != nil {
		log.Fatalf("Unable to write key to file: %v", err)
	}
	return key
}

byte[] localMasterKeyWrite = new byte[96];
new SecureRandom().nextBytes(localMasterKeyWrite);
try (FileOutputStream stream = new FileOutputStream("master-key.txt")) {
    stream.write(localMasterKeyWrite);
}

const fs = require("fs");
const crypto = require("crypto");
try {
  fs.writeFileSync("master-key.txt", crypto.randomBytes(96));
} catch (err) {
  console.error(err);
}

path = "master-key.txt"
file_bytes = os.urandom(96)
with open(path, "wb") as f:
    f.write(file_bytes)

Warning

Do Not Use a Local Key File in Production

A local key file in your filesystem is insecure and is not recommended for production. Instead, you should store your Customer Master Keys in a remote Key Management System (KMS).

To learn how to use a remote KMS in your Queryable Encryption implementation, see the Tutorials guide.

Tip

Generate a CMK from the Command Line

Use the following command to generate a CMK from a Unix shell or PowerShell:

Unix shell:
```
echo $(head -c 96 /dev/urandom | base64 | tr -d '\n')
```
PowerShell:
```
$r=[byte[]]::new(64);$g=[System.Security.Cryptography.RandomNumberGenerator]::Create();$g.GetBytes($r);[Convert]::ToBase64String($r)
```

Save the output of the preceding command to a file named customer-master-key.txt.

Tip

See: Complete Code

To view the complete code for making a Customer Master Key, see the Queryable Encryption sample application repository.

Create a Unique Index on your Key Vault collection

Create a unique index on the keyAltNames field in your encryption.__keyVault namespace.

Select the tab corresponding to your preferred MongoDB driver:

var connectionString = "<Your MongoDB URI>";
var keyVaultNamespace = CollectionNamespace.FromFullName("encryption.__keyVault");
var keyVaultClient = new MongoClient(connectionString);
var indexOptions = new CreateIndexOptions<BsonDocument>
{
    Unique = true,
    PartialFilterExpression = new BsonDocument
        {{"keyAltNames", new BsonDocument {{"$exists", new BsonBoolean(true)}}}}
};
var builder = Builders<BsonDocument>.IndexKeys;
var indexKeysDocument = builder.Ascending("keyAltNames");
var indexModel = new CreateIndexModel<BsonDocument>(indexKeysDocument, indexOptions);
var keyVaultDatabase = keyVaultClient.GetDatabase(keyVaultNamespace.DatabaseNamespace.DatabaseName);
// Drop the Key Vault Collection in case you created this collection
// in a previous run of this application.
keyVaultDatabase.DropCollection(keyVaultNamespace.CollectionName);
var keyVaultCollection = keyVaultDatabase.GetCollection<BsonDocument>(keyVaultNamespace.CollectionName);
keyVaultCollection.Indexes.CreateOne(indexModel);

Important

When building or running the Golang code in this guide using go build or go run, always include the cse build constraint to enable Queryable Encryption. See the following shell command for an example of including the build constraint:

go run -tags cse make-data-key.go

uri := "<Your MongoDB URI>"
keyVaultClient, err := mongo.Connect(context.TODO(), options.Client().ApplyURI(uri))
if err != nil {
	return fmt.Errorf("Connect error for regular client: %v", err)
}
defer func() {
	_ = keyVaultClient.Disconnect(context.TODO())
}()
keyVaultDb := "encryption"
keyVaultColl := "__keyVault"
keyVaultNamespace := keyVaultDb + "." + keyVaultColl
keyVaultIndex := mongo.IndexModel{
	Keys: bson.D{{"keyAltNames", 1}},
	Options: options.Index().
		SetUnique(true).
		SetPartialFilterExpression(bson.D{
			{"keyAltNames", bson.D{
				{"$exists", true},
			}},
		}),
}
// Drop the Key Vault Collection in case you created this collection
// in a previous run of this application.
if err = keyVaultClient.Database(keyVaultDb).Collection(keyVaultColl).Drop(context.TODO()); err != nil {
	log.Fatalf("Collection.Drop error: %v", err)
}
_, err = keyVaultClient.Database(keyVaultDb).Collection(keyVaultColl).Indexes().CreateOne(context.TODO(), keyVaultIndex)
if err != nil {
	panic(err)
}

String keyVaultDb = "encryption";
String keyVaultColl = "__keyVault";
MongoClient keyVaultClient = MongoClients.create(connectionString);
String encryptedDbName = "medicalRecords";
String encryptedCollName = "patients";
// Drop the Key Vault Collection in case you created this collection
// in a previous run of this application.
keyVaultClient.getDatabase(keyVaultDb).getCollection(keyVaultColl).drop();
MongoCollection keyVaultCollection = keyVaultClient.getDatabase(keyVaultDb).getCollection(keyVaultColl);
IndexOptions indexOpts = new IndexOptions().partialFilterExpression(new BsonDocument("keyAltNames", new BsonDocument("$exists", new BsonBoolean(true) ))).unique(true);
keyVaultCollection.createIndex(new BsonDocument("keyAltNames", new BsonInt32(1)), indexOpts);
keyVaultClient.close();

const uri = "<Your Connection String>";
const keyVaultClient = new MongoClient(uri);
await keyVaultClient.connect();
const keyVaultDB = keyVaultClient.db(keyVaultDatabase);
// Drop the Key Vault Collection in case you created this collection
// in a previous run of this application.
await keyVaultDB.dropDatabase();
const keyVaultColl = keyVaultDB.collection(keyVaultCollection);
await keyVaultColl.createIndex(
  { keyAltNames: 1 },
  {
    unique: true,
    partialFilterExpression: { keyAltNames: { $exists: true } },
  }
);

connection_string = "<your connection string here>"
key_vault_coll = "__keyVault"
key_vault_db = "encryption"
key_vault_namespace = f"{key_vault_db}.{key_vault_coll}"
key_vault_client = MongoClient(connection_string)
# Drop the Key Vault Collection in case you created this collection
# in a previous run of this application.
key_vault_client.drop_database(key_vault_db)
key_vault_client[key_vault_db][key_vault_coll].create_index(
    [("keyAltNames", ASCENDING)],
    unique=True,
    partialFilterExpression={"keyAltNames": {"$exists": True}},
)

Create your Data Encryption Keys and Encrypted Collection

Read the Customer Master Key and Specify KMS Provider Settings

Retrieve the contents of the Customer Master Key file that you generated in the Create a Customer Master Key step of this guide.

Pass the CMK value to your KMS provider settings. The client uses these settings to discover the CMK. Set the provider name to local to inform the driver you are using a Local Key Provider.

Select the tab corresponding to your preferred MongoDB driver:

var kmsProviders = new Dictionary<string, IReadOnlyDictionary<string, object>>();
const string provider = "local";
var localMasterKeyBase64Read = File.ReadAllText("master-key.txt");
var localMasterKeyBytes = Convert.FromBase64String(localMasterKeyBase64Read);
var localOptions = new Dictionary<string, object>
{
    {"key", localMasterKeyBytes}
};
kmsProviders.Add(provider, localOptions);

key, err := ioutil.ReadFile("master-key.txt")
if err != nil {
	log.Fatalf("Could not read the key from master-key.txt: %v", err)
}
provider := "local"
kmsProviders := map[string]map[string]interface{}{"local": {"key": key}}

String kmsProvider = "local";
String path = "master-key.txt";
byte[] localMasterKeyRead = new byte[96];
try (FileInputStream fis = new FileInputStream(path)) {
    if (fis.read(localMasterKeyRead) < 96)
        throw new Exception("Expected to read 96 bytes from file");
}
Map<String, Object> keyMap = new HashMap<String, Object>();
keyMap.put("key", localMasterKeyRead);
Map<String, Map<String, Object>> kmsProviders = new HashMap<String, Map<String, Object>>();
kmsProviders.put("local", keyMap);

const provider = "local";
const path = "./master-key.txt";
// WARNING: Do not use a local key file in a production application
const localMasterKey = fs.readFileSync(path);
const kmsProviders = {
  local: {
    key: localMasterKey,
  },
};

provider = "local"
path = "./master-key.txt"
# WARNING: Do not use a local key file in a production application
with open(path, "rb") as f:
    local_master_key = f.read()
kms_providers = {
    "local": {
        "key": local_master_key  # local_master_key variable from the previous step
    },
}

Create your Data Encryption Keys

Construct a client with your MongoDB connection string and Key Vault collection namespace, and create the Data Encryption Keys:

Note

Key Vault Collection Namespace Permissions

To complete this tutorial, the database user your application uses to connect to MongoDB must have dbAdmin permissions on the following namespaces:

encryption.__keyVault
medicalRecords database

var clientEncryptionOptions = new ClientEncryptionOptions(
    keyVaultClient,
    keyVaultNamespace,
    kmsProviders: kmsProviders
    );
var clientEncryption = new ClientEncryption(clientEncryptionOptions);
var dataKeyOptions1 = new DataKeyOptions(alternateKeyNames: new List<string> { "dataKey1" });
var dataKeyOptions2 = new DataKeyOptions(alternateKeyNames: new List<string> { "dataKey2" });
BsonBinaryData CreateKeyGetID(DataKeyOptions options)
{
    var dateKeyGuid = clientEncryption.CreateDataKey(provider, options, CancellationToken.None);
    return new BsonBinaryData(dateKeyGuid, GuidRepresentation.Standard);
}
var dataKeyId1 = CreateKeyGetID(dataKeyOptions1);
var dataKeyId2 = CreateKeyGetID(dataKeyOptions2);
var dataKeyId3 = CreateKeyGetID(dataKeyOptions3);
var dataKeyId4 = CreateKeyGetID(dataKeyOptions4);

clientEncryptionOpts := options.ClientEncryption().SetKeyVaultNamespace(keyVaultNamespace).
	SetKmsProviders(kmsProviders)
clientEnc, err := mongo.NewClientEncryption(keyVaultClient, clientEncryptionOpts)
if err != nil {
	return fmt.Errorf("NewClientEncryption error %v", err)
}
defer func() {
	_ = clientEnc.Close(context.TODO())
}()
dataKeyOpts1 := options.DataKey().
	SetKeyAltNames([]string{"demoDataKey1"})
dataKeyID1, err := clientEnc.CreateDataKey(context.TODO(), provider, dataKeyOpts1)
if err != nil {
	return fmt.Errorf("create data key error %v", err)
}
dataKeyOpts2 := options.DataKey().
	SetKeyAltNames([]string{"demoDataKey2"})
dataKeyID2, err := clientEnc.CreateDataKey(context.TODO(), provider, dataKeyOpts2)
if err != nil {
	return fmt.Errorf("create data key error %v", err)
}

String keyVaultNamespace = keyVaultDb + "." + keyVaultColl;
ClientEncryptionSettings clientEncryptionSettings = ClientEncryptionSettings.builder()
        .keyVaultMongoClientSettings(MongoClientSettings.builder()
                .applyConnectionString(new ConnectionString(connectionString))
                .build())
        .keyVaultNamespace(keyVaultNamespace)
        .kmsProviders(kmsProviders)
        .build();
ClientEncryption clientEncryption = ClientEncryptions.create(clientEncryptionSettings);
List<String> keyAlts1 = new ArrayList<String>();
keyAlts1.add("dataKey1");
BsonBinary dataKeyId1 = clientEncryption.createDataKey(kmsProvider, new DataKeyOptions()
        .keyAltNames(keyAlts1));
List<String> keyAlts2 = new ArrayList<String>();
keyAlts2.add("dataKey2");
BsonBinary dataKeyId2 = clientEncryption.createDataKey(kmsProvider, new DataKeyOptions()
        .keyAltNames(keyAlts2));

const clientEnc = new ClientEncryption(keyVaultClient, {
  keyVaultNamespace: keyVaultNamespace,
  kmsProviders: kmsProviders,
});
const dek1 = await clientEnc.createDataKey(provider, {
  keyAltNames: ["dataKey1"],
});
const dek2 = await clientEnc.createDataKey(provider, {
  keyAltNames: ["dataKey2"],
});

client = MongoClient(connection_string)
client_encryption = ClientEncryption(
    kms_providers,  # pass in the kms_providers variable from the previous step
    key_vault_namespace,
    client,
    CodecOptions(uuid_representation=STANDARD),
)
data_key_id_1 = client_encryption.create_data_key(provider, key_alt_names=["dataKey1"])
data_key_id_2 = client_encryption.create_data_key(provider, key_alt_names=["dataKey2"])

Create Your Encrypted Collection

Use a Queryable Encryption enabled MongoClient instance to specify what fields you must encrypt and create your encrypted collection:

var encryptedCollectionNamespace = CollectionNamespace.FromFullName("medicalRecords.patients");
var encryptedFieldsMap = new Dictionary<string, BsonDocument>
{
    {
        encryptedCollectionNamespace.FullName, new BsonDocument
        {
            {
                "fields", new BsonArray
                {
                    new BsonDocument
                    {
                        {"keyId", dataKeyId1},
                        {"path", new BsonString("patientId")},
                        {"bsonType", new BsonString("int")},
                        {
                            "queries", new BsonDocument
                            {
                                {"queryType", new BsonString("equality")}
                            }
                        }
                    },
                    new BsonDocument
                    {
                        {"keyId", dataKeyId2},
                        {"path", new BsonString("medications")},
                        {"bsonType", new BsonString("array")},
                    },
                }
            }
        }
    }
};
var extraOptions = new Dictionary<string, object>()
{
   { "cryptSharedLibPath", "<path to crypt_shared library>" },
};
var autoEncryptionOptions = new AutoEncryptionOptions(
    keyVaultNamespace,
    kmsProviders,
    encryptedFieldsMap: encryptedFieldsMap,
    extraOptions: extraOptions);
var clientSettings = MongoClientSettings.FromConnectionString(connectionString);
clientSettings.AutoEncryptionOptions = autoEncryptionOptions;
var secureClient = new MongoClient(clientSettings);
var encryptedDatabase = secureClient.GetDatabase(encryptedCollectionNamespace.DatabaseNamespace.DatabaseName);
// Drop the encrypted collection in case you created this collection
// in a previous run of this application.
encryptedDatabase.DropCollection(encryptedCollectionNamespace.CollectionName);
encryptedDatabase.CreateCollection(encryptedCollectionNamespace.CollectionName);
Console.WriteLine("Created encrypted collection!");

dbName := "medicalRecords"
collName := "patients"
encNamespace := (dbName + "." + collName)
encryptedFieldsMap := bson.M{
	encNamespace: bson.M{
		"fields": []bson.M{
			{
				"path":     "patientId",
				"bsonType": "int",
				"keyId":    dataKeyID1,
				"queries": []bson.M{
					{
						"queryType": "equality",
					},
				},
			},
			{
				"path":     "medications",
				"bsonType": "array",
				"keyId":    dataKeyID2,
			},
		},
	},
}
extraOptions := map[string]interface{}{
	"cryptSharedLibPath": "<Your Crypt Shared lib Path>",
}
autoEncryptionOpts := options.AutoEncryption().
	SetKmsProviders(kmsProviders).
	SetKeyVaultNamespace(keyVaultNamespace).
	SetEncryptedFieldsMap(encryptedFieldsMap).
	SetExtraOptions(extraOptions)
secureClient, err := mongo.Connect(context.TODO(), options.Client().ApplyURI(uri).SetAutoEncryptionOptions(autoEncryptionOpts))
if err != nil {
	return fmt.Errorf("Connect error for encrypted client: %v", err)
}
defer func() {
	_ = secureClient.Disconnect(context.TODO())
}()
// Drop the encrypted collection in case you created this collection
// in a previous run of this application.
if err = secureClient.Database(dbName).Collection(collName).Drop(context.TODO()); err != nil {
	log.Fatalf("Collection.Drop error: %v", err)
}
err = secureClient.Database(dbName).CreateCollection(context.TODO(), collName)
if err != nil {
	return fmt.Errorf("Error creating collection: %v", err)
}

String encryptedNameSpace = encryptedDbName + "." + encryptedCollName;
BsonDocument encFields = new BsonDocument().append("fields",
        new BsonArray(Arrays.asList(
                new BsonDocument().append("keyId", dataKeyId1)
                        .append("path", new BsonString("patientId"))
                        .append("bsonType", new BsonString("int"))
                        .append("queries", new BsonDocument().append("queryType", new BsonString("equality"))),
                new BsonDocument().append("keyId", dataKeyId2)
                        .append("path", new BsonString("medications"))
                        .append("bsonType", new BsonString("array")),
                )));
Map<String, BsonDocument> encryptedFieldsMap = new HashMap<String, BsonDocument>();
encryptedFieldsMap.put(encryptedNameSpace, encFields);
Map<String, Object> extraOptions = new HashMap<String, Object>();
extraOptions.put("cryptSharedLibPath", "<path to crypt_shared>");
MongoClientSettings clientSettings = MongoClientSettings.builder()
        .applyConnectionString(new ConnectionString(connectionString))
        .autoEncryptionSettings(AutoEncryptionSettings.builder()
                .keyVaultNamespace(keyVaultNamespace)
                .kmsProviders(kmsProviders)
                .encryptedFieldsMap(encryptedFieldsMap)
                .extraOptions(extraOptions)
                .build())
        .build();
MongoClient mongoClientSecure = MongoClients.create(clientSettings);
MongoDatabase encDb = mongoClientSecure.getDatabase(encryptedDbName);
// Drop the encrypted collection in case you created this collection
// in a previous run of this application.
encDb.getCollection(encryptedCollName).drop();
encDb.createCollection(encryptedCollName);

const encryptedFieldsMap = {
  [`${secretDB}.${secretCollection}`]: {
    fields: [
      {
        keyId: dek1,
        path: "patientId",
        bsonType: "int",
        queries: { queryType: "equality" },
      },
      {
        keyId: dek2,
        path: "medications",
        bsonType: "array",
      },
    ],
  },
};
const extraOptions = {
  cryptSharedLibPath: "<path to FLE Shared Library>",
};
const encClient = new MongoClient(uri, {
  autoEncryption: {
    keyVaultNamespace,
    kmsProviders,
    extraOptions,
    encryptedFieldsMap,
  },
});
await encClient.connect();
const newEncDB = encClient.db(secretDB);
// Drop the encrypted collection in case you created this collection
// in a previous run of this application.
await newEncDB.dropDatabase();
await newEncDB.createCollection(secretCollection);
console.log("Created encrypted collection!");

encrypted_db_name = "medicalRecords"
encrypted_coll_name = "patients"
encrypted_fields_map = {
    f"{encrypted_db_name}.{encrypted_coll_name}": {
        "fields": [
            {
                "keyId": data_key_id_1,
                "path": "patientId",
                "bsonType": "int",
                "queries": {"queryType": "equality"},
            },
            {
                "keyId": data_key_id_2,
                "path": "medications",
                "bsonType": "array",
            },
        ],
    },
}
key_vault_namespace = "encryption.__keyVault"
auto_encryption = AutoEncryptionOpts(
    kms_providers,
    key_vault_namespace,
    encrypted_fields_map=encrypted_fields_map,
    crypt_shared_lib_path="<path to FLE Shared Library>",
)
secure_client = MongoClient(connection_string, auto_encryption_opts=auto_encryption)
# Drop the encrypted collection in case you created this collection
# in a previous run of this application.
secure_client.drop_database(encrypted_db_name)
encrypted_db = secure_client[encrypted_db_name]
encrypted_db.create_collection(encrypted_coll_name)
print("Created encrypted collection!")

The output from the code in this section should resemble the following:

Created encrypted collection!

Tip

See: Complete Code

To view the complete code for making a Data Encryption Key, see the Queryable Encryption sample application repository.

Configure your MongoClient for Encrypted Reads and Writes

Specify the Key Vault Collection Namespace

Specify encryption.__keyVault as the Key Vault collection namespace.

var connectionString = "<Your MongoDB URI>";
var keyVaultNamespace = CollectionNamespace.FromFullName("encryption.__keyVault");
var coll = "patients";
var db = "medicalRecords";

keyVaultColl := "__keyVault"
keyVaultDb := "encryption"
keyVaultNamespace := keyVaultDb + "." + keyVaultColl
dbName := "medicalRecords"
collName := "patients"

String db = "medicalRecords";
String coll = "patients";
String keyVaultDb = "encryption";
String keyVaultColl = "__keyVault";
String keyVaultNamespace = String.format("%1$s.%2$s", keyVaultDb, keyVaultColl);
String connectionString = "<Your MongoDB URI>";

const eDB = "encryption";
const eKV = "__keyVault";
const keyVaultNamespace = `${eDB}.${eKV}`;
const secretDB = "medicalRecords";
const secretCollection = "patients";

key_vault_namespace = "encryption.__keyVault"
key_vault_db_name, key_vault_coll_name = key_vault_namespace.split(".", 1)

Specify the Customer Master Key

Specify the KMS provider and specify your Customer Master Key inline:

var kmsProviders = new Dictionary<string, IReadOnlyDictionary<string, object>>();
const string provider = "local";
const string localMasterKeyPath = "master-key.txt";
var localMasterKeyBase64Read = File.ReadAllText(localMasterKeyPath);
var localMasterKeyBytes = Convert.FromBase64String(localMasterKeyBase64Read);
var localOptions = new Dictionary<string, object>
{
    {"key", localMasterKeyBytes}
};
kmsProviders.Add(provider, localOptions);

key, err := ioutil.ReadFile("master-key.txt")
if err != nil {
	log.Fatalf("Could not read the key from master-key.txt: %v", err)
}
kmsProviders := map[string]map[string]interface{}{"local": {"key": key}}

String kmsProvider = "local";
String path = "master-key.txt";
byte[] localMasterKeyRead = new byte[96];
try (FileInputStream fis = new FileInputStream(path)) {
    if (fis.read(localMasterKeyRead) < 96)
        throw new Exception("Expected to read 96 bytes from file");
}
Map<String, Object> keyMap = new HashMap<>();
keyMap.put("key", localMasterKeyRead);
Map<String, Map<String, Object>> kmsProviders = new HashMap<>();
kmsProviders.put(kmsProvider, keyMap);

const fs = require("fs");
const path = "./master-key.txt";
// WARNING: Do not use a local key file in a production application
const localMasterKey = fs.readFileSync(path);
const kmsProviders = {
  local: {
    key: localMasterKey,
  },
};

path = "./master-key.txt"
with open(path, "rb") as f:
    local_master_key = f.read()
kms_providers = {
    "local": {
        "key": local_master_key  # local_master_key variable from the previous step
    },
}

Retrieve Data Encryption Keys

Retrieve the Data Encryption Keys created in the Create a Data Encryption Key step of this guide:

var regularClient = new MongoClient(connectionString);
var keyVaultCollection = regularClient.GetDatabase(keyVaultNamespace.DatabaseNamespace.DatabaseName)
    .GetCollection<BsonDocument>(keyVaultNamespace.CollectionName);
Guid GetKeyId(string altName)
{
    var filter = Builders<BsonDocument>.Filter.Eq<BsonString>("keyAltNames", altName);
    return keyVaultCollection.Find(filter).First<BsonDocument>()["_id"].AsGuid;
}
var dataKeyId1 = GetKeyId("dataKey1");
var dataKeyId2 = GetKeyId("dataKey2");

uri := "<Your MongoDB URI>"
regularClient, err := mongo.Connect(context.TODO(), options.Client().ApplyURI(uri))
if err != nil {
	panic(fmt.Errorf("Client connect error %v", err))
}
var foundDoc1 bson.M
err = regularClient.Database(keyVaultDb).Collection(keyVaultColl).FindOne(context.TODO(), bson.D{{"keyAltNames", "demoDataKey1"}}).Decode(&foundDoc1)
if err != nil {
	panic(err)
}
var dataKeyID1 = foundDoc1["_id"].(primitive.Binary)
var foundDoc2 bson.M
err = regularClient.Database(keyVaultDb).Collection(keyVaultColl).FindOne(context.TODO(), bson.D{{"keyAltNames", "demoDataKey2"}}).Decode(&foundDoc2)
if err != nil {
	panic(err)
}
var dataKeyID2 = foundDoc2["_id"].(primitive.Binary)

MongoClient client = MongoClients.create(connectionString);
MongoCollection<Document> keyVaultClient = client.getDatabase(keyVaultDb).getCollection(keyVaultColl);
BsonBinary dataKeyId1 = new BsonBinary(BsonBinarySubType.UUID_STANDARD, keyVaultClient.find(eq("keyAltNames", "dataKey1")).first().get("_id", Binary.class).getData());
BsonBinary dataKeyId2 = new BsonBinary(BsonBinarySubType.UUID_STANDARD, keyVaultClient.find(eq("keyAltNames", "dataKey2")).first().get("_id", Binary.class).getData());

const uri = "<Your MongoDB URI>";
const unencryptedClient = new MongoClient(uri);
await unencryptedClient.connect();
const keyVaultClient = unencryptedClient.db(eDB).collection(eKV);
const dek1 = await keyVaultClient.findOne({ keyAltNames: "dataKey1" });
const dek2 = await keyVaultClient.findOne({ keyAltNames: "dataKey2" });

connection_string = "<your connection string here>"
client = MongoClient(connection_string)
key_vault = client[key_vault_db_name][key_vault_coll_name]
data_key_id_1 = key_vault.find_one({"keyAltNames": "dataKey1"})["_id"]
data_key_id_2 = key_vault.find_one({"keyAltNames": "dataKey2"})["_id"]

Specify the Path of the Automatic Encryption Shared Library

var extraOptions = new Dictionary<string, object>()
{
    {"cryptSharedLibPath", "<path to crypt_shared library>"},
};

extraOptions := map[string]interface{}{
	"cryptSharedLibPath": "<path to crypt_shared library>",
}

Map<String, Object> extraOptions = new HashMap<>();
extraOptions.put("cryptSharedLibPath", "<path to crypt_shared library>");

const extraOptions = {
  cryptSharedLibPath: "<path to crypt_shared library>",
};

opts = AutoEncryptionOpts(
    kms_providers,
    key_vault.full_name,
    bypass_query_analysis=True,
    key_vault_client=client,
    crypt_shared_lib_path="<path to FLE Shared Library>",
)

Tip

Learn More

To learn more about the library referenced by this path, see the Automatic Encryption Shared Library for Queryable Encryption page.

Create a MongoClient Object

Instantiate a MongoClient object with the following automatic encryption settings:

var clientSettings = MongoClientSettings.FromConnectionString(connectionString);
var autoEncryptionOptions = new AutoEncryptionOptions(
    keyVaultNamespace,
    kmsProviders,
    bypassQueryAnalysis: true,
    extraOptions: extraOptions);
clientSettings.AutoEncryptionOptions = autoEncryptionOptions;
var secureClient = new MongoClient(clientSettings);
var collection = secureClient.GetDatabase(db).GetCollection<BsonDocument>(coll);

autoEncryptionOpts := options.AutoEncryption().
	SetKmsProviders(kmsProviders).
	SetKeyVaultNamespace(keyVaultNamespace).
	SetExtraOptions(extraOptions).
	SetBypassQueryAnalysis(true)
secureClient, err := mongo.Connect(context.TODO(), options.Client().ApplyURI(uri).SetAutoEncryptionOptions(autoEncryptionOpts))
if err != nil {
	return fmt.Errorf("Connect error for encrypted client: %v", err)
}
defer func() {
	_ = secureClient.Disconnect(context.TODO())
}()
var coll = secureClient.Database(dbName).Collection(collName)

MongoClientSettings clientSettings = MongoClientSettings.builder()
        .applyConnectionString(new ConnectionString(connectionString))
        .autoEncryptionSettings(AutoEncryptionSettings.builder()
                .keyVaultNamespace(keyVaultNamespace)
                .kmsProviders(kmsProviders)
                .extraOptions(extraOptions)
                .bypassQueryAnalysis(true)
                .build())
        .build();
MongoClient mongoClientSecure = MongoClients.create(clientSettings);

const encryptedClient = new MongoClient(uri, {
  autoEncryption: {
    kmsProviders: kmsProviders,
    keyVaultNamespace: keyVaultNamespace,
    bypassQueryAnalysis: true,
    keyVaultClient: unencryptedClient,
    extraOptions: extraOptions,
  },
});
await encryptedClient.connect();

encrypted_client = MongoClient(connection_string, auto_encryption_opts=opts)
db = encrypted_client.medicalRecords
coll = db.patients

Note

Automatic Decryption

We use a MongoClient instance with automatic encryption enabled to perform automatic decryption.

To learn more about explicit encryption with automatic decryption, see the Fundamentals section.

Create a ClientEncryption Object

Instantiate a ClientEncryption object as follows:

var clientEncryptionOptions = new ClientEncryptionOptions(
    keyVaultClient: regularClient,
    keyVaultNamespace: keyVaultNamespace,
    kmsProviders: kmsProviders
    );
var clientEncryption = new ClientEncryption(clientEncryptionOptions);

clientEncryptionOpts := options.ClientEncryption().SetKeyVaultNamespace(keyVaultNamespace).SetKmsProviders(kmsProviders)
clientEnc, err := mongo.NewClientEncryption(regularClient, clientEncryptionOpts)
if err != nil {
	panic(fmt.Errorf("NewClientEncryption error %v", err))
}
defer func() {
	_ = clientEnc.Close(context.TODO())
}()

ClientEncryptionSettings clientEncryptionSettings = ClientEncryptionSettings.builder()
        .keyVaultMongoClientSettings(MongoClientSettings.builder()
                .applyConnectionString(new ConnectionString(connectionString))
                .build())
        .keyVaultNamespace(keyVaultNamespace)
        .kmsProviders(kmsProviders)
        .build();
ClientEncryption clientEncryption = ClientEncryptions.create(clientEncryptionSettings);

const encryption = new ClientEncryption(unencryptedClient, {
  keyVaultNamespace,
  kmsProviders,
});

client_encryption = ClientEncryption(
    kms_providers, key_vault_namespace, client, client.codec_options
)

Insert a Document with Encrypted Fields

Use your Queryable Encryption enabled MongoClient instance to insert a document with encrypted fields into the medicalRecords.patients namespace using the following code snippet:

var patientId = 12345678;
var medications = new BsonArray
        {
            new BsonString("Atorvastatin"),
            new BsonString("Levothyroxine")
        };
var indexedEncrypted = clientEncryption.Encrypt(
    patientId,
    new EncryptOptions(algorithm: "Indexed", keyId: dataKeyId1, contentionFactor: 1),
    CancellationToken.None);
var unindexedEncrypted = clientEncryption.Encrypt(
    medications,
    new EncryptOptions(algorithm: "Unindexed", keyId: dataKeyId2),
    CancellationToken.None);
collection.InsertOne(new BsonDocument { { "firstName", "Jon" }, { "patientId", indexedEncrypted }, { "medications", unindexedEncrypted } });

patientIdRawValueType, patientIdRawValueData, err := bson.MarshalValue(12345678)
if err != nil {
	panic(err)
}
patientIdRawValue := bson.RawValue{Type: patientIdRawValueType, Value: patientIdRawValueData}
patientIdEncryptionOpts := options.Encrypt().
	SetAlgorithm("Indexed").
	SetKeyID(dataKeyID1).
	SetContentionFactor(1)
patientIdEncryptedField, err := clientEnc.Encrypt(
	context.TODO(),
	patientIdRawValue,
	patientIdEncryptionOpts)
if err != nil {
	panic(err)
}
medicationsRawValueType, medicationsRawValueData, err := bson.MarshalValue([]string{"Atorvastatin", "Levothyroxine"})
if err != nil {
	panic(err)
}
medicationsRawValue := bson.RawValue{Type: medicationsRawValueType, Value: medicationsRawValueData}
medicationsEncryptionOpts := options.Encrypt().
	SetAlgorithm("Unindexed").
	SetKeyID(dataKeyID2)
medicationsEncryptedField, err := clientEnc.Encrypt(
	context.TODO(),
	medicationsRawValue,
	medicationsEncryptionOpts)
if err != nil {
	panic(err)
}
_, err = coll.InsertOne(
	context.TODO(),
	bson.D{{"firstName", "Jon"}, {"patientId", patientIdEncryptedField}, {"medications", medicationsEncryptedField}})
if err != nil {
	panic(err)
}

BsonInt32 patientId = new BsonInt32(12345678);
ArrayList<BsonString> medications = new ArrayList<>();
medications.add(new BsonString("Atorvastatin"));
medications.add(new BsonString("Levothyroxine"));
BsonBinary indexedEncrypted = clientEncryption.encrypt(patientId, new EncryptOptions("Indexed").keyId(dataKeyId1).contentionFactor(1L));
BsonBinary unindexedEncrypted = clientEncryption.encrypt(new BsonArray(medications), new EncryptOptions("Unindexed").keyId(dataKeyId2));
MongoCollection<BsonDocument> collection = mongoClientSecure.getDatabase(db).getCollection(coll, BsonDocument.class);
collection.insertOne(new BsonDocument("firstName", new BsonString("Jon")).append("patientId", indexedEncrypted).append("medications", unindexedEncrypted));

const patientId = 12345678;
const medications = ["Atorvastatin", "Levothyroxine"];
const indexedInsertPayload = await encryption.encrypt(patientId, {
  algorithm: "Indexed",
  keyId: dek1._id,
  contentionFactor: 1,
});
const unindexedInsertPayload = await encryption.encrypt(medications, {
  algorithm: "Unindexed",
  keyId: dek2._id,
});
const encryptedColl = encryptedClient
  .db(secretDB)
  .collection(secretCollection);
await encryptedColl.insertOne({
  firstName: "Jon",
  patientId: indexedInsertPayload,
  medications: unindexedInsertPayload,
});

patientId = 12345678
medications = ["Atorvastatin", "Levothyroxine"]
indexed_insert_payload = client_encryption.encrypt(
    patientId, Algorithm.INDEXED, data_key_id_1, contention_factor=1
)
unindexed_insert_payload = client_encryption.encrypt(
    medications, Algorithm.UNINDEXED, data_key_id_2
)
coll.insert_one(
    {
        "firstName": "Jon",
        "patientId": indexed_insert_payload,
        "medications": unindexed_insert_payload,
    }
)

When you insert a document, your Queryable Encryption enabled client encrypts the fields of your document such that it resembles the following:

{
  "_id": {
    "$oid": "6303e36053cc7ec2e6a630bd"
  },
  "firstName": "Jon",
  "patientId": {
    "$binary": {
      "base64": "BxLJUBmg703civqMz8ASsD4QEYeSneOGiiYHfLE77ELEkp1EC/fXPrKCNRQl2mAFddszqDJ0P3znKrq0DVMEvJoU6wa0Ra+U+JjNVr8NtJE+TpTLCannY5Av6iGfLAaiHbM/E8Ftz1YCQsArQwuNp3wIV/GJPLa2662xsyk0wz7F6IRGC3FlnxpN4UIFaHE1M7Y6kEnx3tEy5uJBvU4Sex7I2H0kqHthClH77Q6xHIHc8H9d6upvgnEbkKBCnmc24A2pSG/xZ7LBsV3j5aOboPISuN/lvg==",
      "subType": "06"
    }
  },
  "medications": {
    "$binary": {
      "base64": "BvOsveapfUxiuQxCMSM2fYIEyRlQaSqR+0NxlMarwurBflvoMz1FrSjSGgCVCpK8X+YrilP6Bac99kkaUmRJfjo4savxcjpOfEnUj5bHciPyfQBYmYF4PMLDtTTzGZpPilb9d5KgpIMBXxHi+dIcog==",
      "subType": "06"
    }
  },
  "__safeContent__": [
    {
      "$binary": {
        "base64": "ZLPIpgxzXpHUGrvdIHetwmMagR+mqvuUj5nzXNGf/WM=",
        "subType": "00"
      }
    }
  ]
}

Warning

Do not Modify the safeContent Field

The __safeContent__ field is essential to Queryable Encryption. Do not modify the contents of this field.

Tip

See: Complete Code

To view the complete code to insert a document encrypted with explicit encryption, see the Queryable Encryption sample application repository.

Retrieve Your Document with Encrypted Fields

Retrieve the document with encrypted fields you inserted in the Insert a Document with Encrypted Fields step of this guide through a query on an encrypted field:

var findPayload = clientEncryption.Encrypt(
    patientId,
    new EncryptOptions(algorithm: "Indexed", keyId: dataKeyId1, queryType: "equality", contentionFactor: 1),
    CancellationToken.None);
var doc = collection.Find(new BsonDocument { { "patientId", findPayload } }).Single();
Console.WriteLine($"Encrypted document: {doc}");

findPayloadRawValueType, findPayloadRawValueData, err := bson.MarshalValue(12345678)
if err != nil {
	panic(err)
}
findPayloadRawValue := bson.RawValue{Type: findPayloadRawValueType, Value: findPayloadRawValueData}
findPayloadEncryptionOpts := options.Encrypt().
	SetAlgorithm("Indexed").
	SetKeyID(dataKeyID1).
	SetQueryType("equality").
	SetContentionFactor(1)
findPayloadEncryptedField, err := clientEnc.Encrypt(
	context.TODO(),
	findPayloadRawValue,
	findPayloadEncryptionOpts)
if err != nil {
	panic(err)
}
var resultSecure bson.M
coll.FindOne(context.TODO(), bson.D{{"firstName", findPayloadEncryptedField}}).Decode(&resultSecure)
if err != nil {
	panic(err)
}
outputSecure, err := json.MarshalIndent(resultSecure, "", "    ")
if err != nil {
	panic(err)
}
fmt.Printf("\nFound document searching on explicitly encrypted field:\n%s\n", outputSecure)

BsonBinary findPayloadEncrypted = clientEncryption.encrypt(patientId, new EncryptOptions("Indexed").keyId(dataKeyId1).queryType("equality").contentionFactor(1L));
BsonDocument result = collection.find(eq("patientId", findPayloadEncrypted)).first();
System.out.println("Finding a document with manually encrypted field: " + result.toJson());

const findPayload = await encryption.encrypt(patientId, {
  algorithm: "Indexed",
  keyId: dek1._id,
  queryType: "equality",
  contentionFactor: 1,
});
console.log("Finding a document with manually encrypted field:");
console.log(await encryptedColl.findOne({ patientId: findPayload }));

find_payload = client_encryption.encrypt(
    patientId,
    Algorithm.INDEXED,
    data_key_id_1,
    query_type=QueryType.EQUALITY,
    contention_factor=1,
)
doc = coll.find_one({"encryptedIndexed": find_payload})
print("\nReturned document:\n")
pprint.pprint(doc)

The output of the preceding code snippet should contain the following document:

{
  "__safeContent__": [
    {
      "Subtype": 0,
      "Data": "LfaIuWm9o30MIGrK7GGUoStJMSNOjRgbxy5q2TPiDes="
    }
  ],
  "_id": "6303a770857952ca5e363fd2",
  "firstName": "Jon",
  "medications": ["Atorvastatin", "Levothyroxine"],
  "patientId": 12345678
}

Tip

See: Complete Code

To view the code to retrieve your document with encrypted fields, see the Queryable Encryption sample application repository.

Learn More

To view a tutorial on using Queryable Encryption with a remote KMS, see Tutorials.

To learn how Queryable Encryption works, see Explicit Encryption.

To learn more about the topics mentioned in this guide, see the following links:

Back

Use KMIP

Reference

Overview

Important

Do Not Use this Sample Application In Production

Before You Get Started

Tip

See: Full Application

Procedure

Create a Customer Master Key

Warning

Do Not Use a Local Key File in Production

Tip

Generate a CMK from the Command Line

Tip

See: Complete Code

Create a Unique Index on your Key Vault collection

Important

Create your Data Encryption Keys and Encrypted Collection

Read the Customer Master Key and Specify KMS Provider Settings

Create your Data Encryption Keys

Note

Key Vault Collection Namespace Permissions

Create Your Encrypted Collection

Tip

See: Complete Code

Configure your MongoClient for Encrypted Reads and Writes

Specify the Key Vault Collection Namespace

Specify the Customer Master Key

Retrieve Data Encryption Keys

Specify the Path of the Automatic Encryption Shared Library

Tip

Learn More

Create a MongoClient Object

Note

Automatic Decryption

Create a ClientEncryption Object

Insert a Document with Encrypted Fields

Warning

Do not Modify the __safeContent__ Field

Tip

See: Complete Code

Retrieve Your Document with Encrypted Fields

Tip

See: Complete Code

Learn More

Do not Modify the safeContent Field