Deploy Replica Sets in a Multi-Kubernetes Cluster

On this page

Deploy a MongoDBMultiCluster Resource

Use this procedure to create a new replica set in a member Kubernetes cluster in a multi-Kubernetes cluster MongoDB deployment.

This procedure allows you to set different settings for the replica set resource, such as overrides for statefulSet configuration. As an alternative to using this procedure, you can use the Multi-Kubernetes-Cluster Quick Start, which creates a multi-Kubernetes cluster MongoDB deployment with default settings.

Before you begin:

Learn about multi-Kubernetes-cluster deployments.
Review the list of multi-Kubernetes-cluster services and tools.
Install the Kubernetes Operator in a multi-Kubernetes cluster MongoDB deployment. See Multi-Kubernetes-Cluster Quick Start.
Complete the prerequisites.

Deploy a `MongoDBMultiCluster` Resource

You can use the following procedures in this TLS-Encrypted Connections tab:

Deploy a MongoDBMultiCluster resource
Renew TLS Certificates for a MongoDBMultiCluster resource

These procedures establish TLS-encrypted connections between MongoDB hosts in a replica set, and between client applications and MongoDB deployments.

Before you begin, you must have valid certificates for TLS encryption.

Deploy a `MongoDBMultiCluster` Resource

Create the secret for the TLS certificate of your `MongoDBMultiCluster` resource.

Run the kubectl command to create a new secret that stores the MongoDBMultiCluster resource certificate:

kubectl --context $MDB_CENTRAL_CLUSTER_FULL_NAME \
  --namespace=<metadata.namespace> \
  create secret tls <prefix>-<metadata.name>-cert \
  --cert=<resource-tls-cert> \
  --key=<resource-tls-key>

Note

You must prefix your secrets with <prefix>-<metadata.name>.

For example, if you call your deployment my-deployment and you set the prefix to mdb, you must name the TLS secret for the client TLS communications mdb-my-deployment-cert. Also, you must name the TLS secret for internal cluster authentication (if enabled) mdb-my-deployment-clusterfile.

Create the ConfigMap to link your CA with your `MongoDBMultiCluster` resource.

Run the kubectl command to link your CA to your MongoDBMultiCluster resource. Specify the CA certificate file that you must always name ca-pem for the MongoDBMultiCluster resource:

kubectl --context $MDB_CENTRAL_CLUSTER_FULL_NAME \
  --namespace=<metadata.namespace> \
  create configmap custom-ca -from-file=ca-pem=<your-custom-ca-file>

Configure `kubectl` to use the central cluster's namespace.

If you have not done so already, run the following commands to run all kubectl commands on the central cluster in the default namespace.

kubectl config use-context $MDB_CENTRAL_CLUSTER_FULL_NAME
kubectl config set-context $(kubectl config current-context) \
  --namespace=mongodb

Copy and paste the sample resource.

Copy the sample replica set YAML file and paste it into a new text file.
Change the file's settings to match your desired replica set configuration.

1 # This example provides statefulSet overrides per cluster.
2 
3 apiVersion: mongodb.com/v1
4 kind: MongoDBMultiCluster
5 metadata:
6   name: multi-replica-set
7 spec:
8   version: 6.0.0-ent
9   type: ReplicaSet
10   duplicateServiceObjects: false
11   credentials: my-credentials
12   opsManager:
13     configMapRef:
14       name: my-project
15   clusterSpecList:
16     - clusterName: cluster1.example.com
17       members: 2
18       statefulSet:
19         spec:
20           template:
21             spec:
22               containers:
23                 # Example of custom sidecar containers. Remove it before using the file in production.
24                 - name: sidecar1
25                   image: busybox
26                   command: [ "sleep" ]
27                   args: [ "infinity" ]
28           # Use the following settings to override the default storage size of the "data" Persistent Volume.
29           volumeClaimTemplates:
30             - metadata:
31                 name: data
32               spec:
33                 resources:
34                   requests:
35                     storage: 1Gi
36     - clusterName: cluster2.example.com
37       members: 1
38       statefulSet:
39         spec:
40           template:
41             spec:
42               containers:
43                 # Example of custom sidecar containers. Remove it before using the file in production.
44                 - name: sidecar2
45                   image: busybox
46                   command: [ "sleep" ]
47                   args: [ "infinity" ]
48           volumeClaimTemplates:
49             - metadata:
50                 name: data
51               spec:
52                 resources:
53                   requests:
54                     storage: 1Gi
55     - clusterName: cluster3.example.com
56       members: 1
57       statefulSet:
58         spec:
59           template:
60             spec:
61               containers:
62                 # Example of custom sidecar containers. Remove it before using the file in production.
63                 - name: sidecar3
64                   image: busybox
65                   command: [ "sleep" ]
66                   args: [ "infinity" ]
67           volumeClaimTemplates:
68             - metadata:
69                 name: data
70               spec:
71                 resources:
72                   requests:
73                     storage: 1Gi
74 
75 ...

Change the settings to your preferred values.

Key	Type	Description	Example
`metadata.name`	string	Label for the `MongoDBMultiCluster` resource. Resource names must be 44 characters or less. See also `metadata.name` and names in the Kubernetes documentation.	`multi-replica-set`
`spec.version`	string	Version of MongoDB that this `MongoDBMultiCluster` resource should run. The format should be `X.Y.Z` for the Community edition and `X.Y.Z-ent` for the Enterprise edition. IMPORTANT: Ensure that you choose a compatible MongoDB Server version. Compatible versions differ depending on the base image that the MongoDB database resource uses. To learn more about MongoDB versioning, see MongoDB Versioning in the MongoDB Manual.	`6.0.0-ent`
`spec` `.opsManager` `.configMapRef` `.name`	string	Name of the ConfigMap with the Ops Manager connection configuration. The `spec.cloudManager.configMapRef.name` setting is an alias for this setting and can be used in its place. This value must exist on the same namespace as the resource you want to create. IMPORTANT: The Kubernetes Operator tracks any changes to the ConfigMap and reconciles the state of the `MongoDB` resource.	`<my-project>`
`spec` `.clusterSpecList` `.clusterName`	string	Name of the cluster in the `MongoDBMultiCluster` resource.	`cluster1.example.com`
`spec` `.clusterSpecList` `.members`	integer	The number of members in this cluster.	`2`
`spec` `.clusterSpecList` `.statefulSet` `.spec`	collection	Optional. Provides the configuration for the StatefulSet override for each of the cluster's StatefulSets in a multi-Kubernetes cluster MongoDB deployment. If specified at an individual cluster level under `clusterSpecList`, overrides the global configuration for the StatefulSet for the entire multi-Kubernetes cluster MongoDB deployment. See Multi-Kubernetes-Cluster Resource Specification and StatefulSet v1 apps Kubernetes documentation.	See the example.
`spec` `.clusterSpecList` `.statefulSet` `.spec` `.volumeClaimTemplates` `.spec`	collection	Optional. If specified, provides a per-cluster override for the default storage size of the volumeClaimtemplates, for the persistent volume that stores the data.	See the example.
`spec.credentials`	string	Name of the secret you created as Ops Manager API authentication credentials for the Kubernetes Operator to communicate with Ops Manager. The Ops Manager Kubernetes Secret object holding the Credentials must exist on the same Namespace as the resource you want to create. IMPORTANT: The Kubernetes Operator tracks any changes to the Secret and reconciles the state of the `MongoDB` resource.	`<mycredentials>`
`spec.type`	string	Type of `MongoDB` resource to create. The only supported value for this field is `ReplicaSet`. See Limitations.	`ReplicaSet`

Add any additional accepted settings for a `MongoDBMultiCluster` resource deployment.

You can also add any optional settings to the object specification. See Multi-Kubernetes-Cluster Resource Specification.

Save this replica set config file with a `.yaml` extension.

Start your replica set deployment.

In any directory, invoke the following Kubernetes command to create your replica set:

kubectl apply -f <replica-set-conf>.yaml

Track the status of your multi-Kubernetes cluster MongoDB deployment.

To check the status of your MongoDBMultiCluster resource, use the following command on the central cluster:

kubectl get mdbmc <resource-name> -o yaml -w

With the -w (watch) flag set, when the configuration changes, the output refreshes immediately until the status phase achieves the Running state. To learn more about resource deployment statuses, see Troubleshoot the Kubernetes Operator.

Renew TLS Certificates for a `MongoDBMultiCluster` Resource

Renew your TLS certificates periodically using the following procedure.

Renew the secret for a `MongoDBMultiCluster` resource.

Run this kubectl command to renew an existing secret that stores the certificates for the MongoDBMultiCluster resource:

kubectl --context $MDB_CENTRAL_CLUSTER_FULL_NAME \
--namespace=<metadata.namespace> \
create secret tls <prefix>-<metadata.name>-cert \
--cert=<resource-tls-cert> \
--key=<resource-tls-key> \
--dry-run=client \
-o yaml |
kubectl apply -f -

This procedure doesn't encrypt connections between MongoDB hosts in a replica set, and between client applications and MongoDB deployments.

Deploy a `MongoDBMultiCluster` Resource

Configure `kubectl` to use the central cluster's namespace.

If you have not done so already, run the following commands to run all kubectl commands on the central cluster in the default namespace.

kubectl config use-context $MDB_CENTRAL_CLUSTER_FULL_NAME
kubectl config set-context $(kubectl config current-context) \
  --namespace=mongodb

Copy and paste the sample resource.

Copy the sample replica set YAML file and paste it into a new text file.
Change the file's settings to match your desired replica set configuration.

1 # This example provides statefulSet overrides per cluster.
2 
3 apiVersion: mongodb.com/v1
4 kind: MongoDBMultiCluster
5 metadata:
6   name: multi-replica-set
7 spec:
8   version: 6.0.0-ent
9   type: ReplicaSet
10   duplicateServiceObjects: false
11   credentials: my-credentials
12   opsManager:
13     configMapRef:
14       name: my-project
15   clusterSpecList:
16     - clusterName: cluster1.example.com
17       members: 2
18       statefulSet:
19         spec:
20           template:
21             spec:
22               containers:
23                 # Example of custom sidecar containers. Remove it before using the file in production.
24                 - name: sidecar1
25                   image: busybox
26                   command: [ "sleep" ]
27                   args: [ "infinity" ]
28           # Use the following settings to override the default storage size of the "data" Persistent Volume.
29           volumeClaimTemplates:
30             - metadata:
31                 name: data
32               spec:
33                 resources:
34                   requests:
35                     storage: 1Gi
36     - clusterName: cluster2.example.com
37       members: 1
38       statefulSet:
39         spec:
40           template:
41             spec:
42               containers:
43                 # Example of custom sidecar containers. Remove it before using the file in production.
44                 - name: sidecar2
45                   image: busybox
46                   command: [ "sleep" ]
47                   args: [ "infinity" ]
48           volumeClaimTemplates:
49             - metadata:
50                 name: data
51               spec:
52                 resources:
53                   requests:
54                     storage: 1Gi
55     - clusterName: cluster3.example.com
56       members: 1
57       statefulSet:
58         spec:
59           template:
60             spec:
61               containers:
62                 # Example of custom sidecar containers. Remove it before using the file in production.
63                 - name: sidecar3
64                   image: busybox
65                   command: [ "sleep" ]
66                   args: [ "infinity" ]
67           volumeClaimTemplates:
68             - metadata:
69                 name: data
70               spec:
71                 resources:
72                   requests:
73                     storage: 1Gi
74 
75 ...

Change the settings to your preferred values.

Key	Type	Description	Example
`metadata.name`	string	Label for the `MongoDBMultiCluster` resource. Resource names must be 44 characters or less. See also `metadata.name` and names in the Kubernetes documentation.	`multi-replica-set`
`spec.version`	string	Version of MongoDB that this `MongoDBMultiCluster` resource should run. The format should be `X.Y.Z` for the Community edition and `X.Y.Z-ent` for the Enterprise edition. IMPORTANT: Ensure that you choose a compatible MongoDB Server version. Compatible versions differ depending on the base image that the MongoDB database resource uses. To learn more about MongoDB versioning, see MongoDB Versioning in the MongoDB Manual.	`6.0.0-ent`
`spec` `.opsManager` `.configMapRef` `.name`	string	Name of the ConfigMap with the Ops Manager connection configuration. The `spec.cloudManager.configMapRef.name` setting is an alias for this setting and can be used in its place. This value must exist on the same namespace as the resource you want to create. IMPORTANT: The Kubernetes Operator tracks any changes to the ConfigMap and reconciles the state of the `MongoDB` resource.	`<my-project>`
`spec` `.clusterSpecList` `.clusterName`	string	Name of the cluster in the `MongoDBMultiCluster` resource.	`cluster1.example.com`
`spec` `.clusterSpecList` `.members`	integer	The number of members in this cluster.	`2`
`spec` `.clusterSpecList` `.statefulSet` `.spec`	collection	Optional. Provides the configuration for the StatefulSet override for each of the cluster's StatefulSets in a multi-Kubernetes cluster MongoDB deployment. If specified at an individual cluster level under `clusterSpecList`, overrides the global configuration for the StatefulSet for the entire multi-Kubernetes cluster MongoDB deployment. See Multi-Kubernetes-Cluster Resource Specification and StatefulSet v1 apps Kubernetes documentation.	See the example.
`spec` `.clusterSpecList` `.statefulSet` `.spec` `.volumeClaimTemplates` `.spec`	collection	Optional. If specified, provides a per-cluster override for the default storage size of the volumeClaimtemplates, for the persistent volume that stores the data.	See the example.
`spec.credentials`	string	Name of the secret you created as Ops Manager API authentication credentials for the Kubernetes Operator to communicate with Ops Manager. The Ops Manager Kubernetes Secret object holding the Credentials must exist on the same Namespace as the resource you want to create. IMPORTANT: The Kubernetes Operator tracks any changes to the Secret and reconciles the state of the `MongoDB` resource.	`<mycredentials>`
`spec.type`	string	Type of `MongoDB` resource to create. The only supported value for this field is `ReplicaSet`. See Limitations.	`ReplicaSet`

Add any additional accepted settings for a `MongoDBMultiCluster` resource deployment.

You can also add any optional settings to the object specification. See Multi-Kubernetes-Cluster Resource Specification.

Save this replica set config file with a `.yaml` extension.

Start your replica set deployment.

In any directory, invoke the following Kubernetes command to create your replica set:

kubectl apply -f <replica-set-conf>.yaml

Track the status of your multi-Kubernetes cluster MongoDB deployment.

To check the status of your MongoDBMultiCluster resource, use the following command on the central cluster:

kubectl get mdbmc <resource-name> -o yaml -w

Back

Quick Start

Deploy Replica Sets in a Multi-Kubernetes Cluster without a Service Mesh

1	# This example provides statefulSet overrides per cluster.
2
3	apiVersion: mongodb.com/v1
4	kind: MongoDBMultiCluster
5	metadata:
6	name: multi-replica-set
7	spec:
8	version: 6.0.0-ent
9	type: ReplicaSet
10	duplicateServiceObjects: false
11	credentials: my-credentials
12	opsManager:
13	configMapRef:
14	name: my-project
15	clusterSpecList:
16	- clusterName: cluster1.example.com
17	members: 2
18	statefulSet:
19	spec:
20	template:
21	spec:
22	containers:
23	# Example of custom sidecar containers. Remove it before using the file in production.
24	- name: sidecar1
25	image: busybox
26	command: [ "sleep" ]
27	args: [ "infinity" ]
28	# Use the following settings to override the default storage size of the "data" Persistent Volume.
29	volumeClaimTemplates:
30	- metadata:
31	name: data
32	spec:
33	resources:
34	requests:
35	storage: 1Gi
36	- clusterName: cluster2.example.com
37	members: 1
38	statefulSet:
39	spec:
40	template:
41	spec:
42	containers:
43	# Example of custom sidecar containers. Remove it before using the file in production.
44	- name: sidecar2
45	image: busybox
46	command: [ "sleep" ]
47	args: [ "infinity" ]
48	volumeClaimTemplates:
49	- metadata:
50	name: data
51	spec:
52	resources:
53	requests:
54	storage: 1Gi
55	- clusterName: cluster3.example.com
56	members: 1
57	statefulSet:
58	spec:
59	template:
60	spec:
61	containers:
62	# Example of custom sidecar containers. Remove it before using the file in production.
63	- name: sidecar3
64	image: busybox
65	command: [ "sleep" ]
66	args: [ "infinity" ]
67	volumeClaimTemplates:
68	- metadata:
69	name: data
70	spec:
71	resources:
72	requests:
73	storage: 1Gi
74
75	...

Deploy a MongoDBMultiCluster Resource

Create the secret for the TLS certificate of your MongoDBMultiCluster resource.

Note

Create the ConfigMap to link your CA with your MongoDBMultiCluster resource.

Configure kubectl to use the central cluster's namespace.

Copy and paste the sample resource.

Change the settings to your preferred values.

Add any additional accepted settings for a MongoDBMultiCluster resource deployment.

Save this replica set config file with a .yaml extension.

Start your replica set deployment.

Track the status of your multi-Kubernetes cluster MongoDB deployment.

Renew TLS Certificates for a MongoDBMultiCluster Resource

Renew the secret for a MongoDBMultiCluster resource.

Deploy a MongoDBMultiCluster Resource

Configure kubectl to use the central cluster's namespace.

Copy and paste the sample resource.

Change the settings to your preferred values.

Add any additional accepted settings for a MongoDBMultiCluster resource deployment.

Save this replica set config file with a .yaml extension.

Start your replica set deployment.

Track the status of your multi-Kubernetes cluster MongoDB deployment.

Deploy a `MongoDBMultiCluster` Resource

Create the secret for the TLS certificate of your `MongoDBMultiCluster` resource.

Create the ConfigMap to link your CA with your `MongoDBMultiCluster` resource.

Configure `kubectl` to use the central cluster's namespace.

Add any additional accepted settings for a `MongoDBMultiCluster` resource deployment.

Save this replica set config file with a `.yaml` extension.

Renew TLS Certificates for a `MongoDBMultiCluster` Resource

Renew the secret for a `MongoDBMultiCluster` resource.

Deploy a `MongoDBMultiCluster` Resource

Configure `kubectl` to use the central cluster's namespace.

Add any additional accepted settings for a `MongoDBMultiCluster` resource deployment.

Save this replica set config file with a `.yaml` extension.