Prerequisites

Review Supported Hardware Architectures

See supported hardware architectures.

Clone the MongoDB Enterprise Kubernetes Operator Repository

Clone the MongoDB Enterprise Kubernetes Operator repository:

git clone https://github.com/mongodb/mongodb-enterprise-kubernetes.git

Set Environment Variables

Set the environment variables with cluster names where you deploy the clusters, as in this example:

export MDB_CENTRAL_CLUSTER_FULL_NAME="mdb-central"
export MDB_CLUSTER_1_FULL_NAME="mdb-1"
export MDB_CLUSTER_2_FULL_NAME="mdb-2"
export MDB_CLUSTER_3_FULL_NAME="mdb-3"

Install Go and Helm

Install the following tools:

1. Install Go v1.17 or later.

2. Install Helm.

Understand Kubernetes Roles and Role Bindings

To use a multi-Kubernetes cluster MongoDB deployment, you must have a specific set of Kubernetes Roles, ClusterRoles, RoleBindings, ClusterRoleBindings, and ServiceAccounts, which you can configure in any of the following ways:

• Follow the Multi-Kubernetes-Cluster Quick Start, which tells you how to use the MongoDB Plugin to automatically create the required objects and apply them to the appropriate clusters within your multi-Kubernetes cluster MongoDB deployment.

• Use Helm to configure the required Kubernetes Roles and service accounts for each member cluster:

  helm template --show-only \
    templates/database-roles.yaml \
    mongodb/enterprise-operator \
    --set namespace=mongodb | \
  kubectl apply -f - \
    --context=$MDB_CLUSTER_1_FULL_NAME \
    --namespace mongodb

  helm template --show-only \
    templates/database-roles.yaml \
    mongodb/enterprise-operator \
    --set namespace=mongodb | \
  kubectl apply -f - \
    --context=$MDB_CLUSTER_2_FULL_NAME \
    --namespace mongodb

  helm template --show-only \
    templates/database-roles.yaml \
    mongodb/enterprise-operator \
    --set namespace=mongodb | \
  kubectl apply -f - \
    --context=$MDB_CLUSTER_3_FULL_NAME \
    --namespace mongodb

• Manually create Kubernetes object .yaml files and add the required Kubernetes Roles and service accounts to your multi-Kubernetes cluster MongoDB deployment with the kubectl apply command. This may be necessary for certain highly automated workflows. MongoDB provides sample configuration files.

  For custom resources scoped to a subset of namespaces:

  • Roles, Role Bindings, and Service Accounts for your Central Cluster

  • Roles, Role Bindings, and Service Accounts for your Member Clusters

  For custom resources scoped to a cluster-wide namespace:

  • ClusterRoles, ClusterRoleBindings, and ServiceAccounts for your Central Cluster

  • ClusterRoles, ClusterRoleBindings, and ServiceAccounts for your Member Cluster

  Each file defines multiple resources. To support your deployment, you must replace the placeholder values in the following fields:

  • subjects.namespace in each RoleBinding or ClusterRoleBinding resource

  • metadata.namespace in each ServiceAccount resource

  After modifying the definitions, apply them by running the following command for each file:

  kubectl apply -f <fileName>

Set the Deployment's Scope

By default, the multi-cluster Kubernetes Operator is scoped to the namespace in which you install it. The Kubernetes Operator reconciles the MongoDBMultiCluster resource deployed in the same namespace as the Kubernetes Operator.

When you run the MongoDB kubectl plugin as part of the multi-cluster quick start, and don't modify the kubectl mongodb plugin settings, the plugin:

• Creates a default ConfigMap named mongodb-enterprise-operator-member-list that contains all the member clusters of the multi-Kubernetes cluster MongoDB deployment. This name is hard-coded and you can't change it. See Known Issues.

• Creates ServiceAccounts, Roles, ClusterRoles, RoleBindings and ClusterRoleBindings in the central cluster and each member cluster.

• Applies the correct permissions for service accounts.

• Uses the preceding settings to create your multi-Kubernetes cluster MongoDB deployment.

Once the Kubernetes Operator creates the multi-Kubernetes cluster MongoDB deployment, the Kubernetes Operator starts watching MongoDB resources in the mongodb namespace.

To configure the Kubernetes Operator with the correct permissions to deploy in a subset or all namespaces, run the following command and specify the namespaces that you would like the Kubernetes Operator to watch.

kubectl mongodb multicluster setup \
  --central-cluster="${MDB_CENTRAL_CLUSTER_FULL_NAME}" \
  --member-clusters="${MDB_CLUSTER_1_FULL_NAME},${MDB_CLUSTER_2_FULL_NAME},${MDB_CLUSTER_3_FULL_NAME}" \
  --member-cluster-namespace="mongodb2" \
  --central-cluster-namespace="mongodb2" \
  --create-service-account-secrets \
  --cluster-scoped="true"

When you install the multi-Kubernetes cluster MongoDB deployment to multiple or all namespaces, you can configure the Kubernetes Operator to:

• Watch Resources in Multiple Namespaces

• Watch Resources in All Namespaces

WATCH_NAMESPACE: "$namespace1,$namespace2,$namespace3"

helm upgrade \
  --install \
  mongodb-enterprise-operator-multi-cluster \
  mongodb/enterprise-operator \
  --namespace mongodb \
  --set namespace=mongodb \
  --version <mongodb-kubernetes-operator-version>\
  --set operator.name=mongodb-enterprise-operator-multi-cluster \
  --set operator.createOperatorServiceAccount=false \
  --set operator.createResourcesServiceAccountsAndRoles=false \
  --set "multiCluster.clusters={$MDB_CLUSTER_1_FULL_NAME,$MDB_CLUSTER_2_FULL_NAME,$MDB_CLUSTER_3_FULL_NAME}" \
  --set operator.watchNamespace="$namespace1,$namespace2,$namespace3"

WATCH_NAMESPACE: "*"

helm upgrade \
  --install \
  mongodb-enterprise-operator-multi-cluster \
  mongodb/enterprise-operator \
  --namespace mongodb \
  --set namespace=mongodb \
  --version <mongodb-kubernetes-operator-version>\
  --set operator.name=mongodb-enterprise-operator-multi-cluster \
  --set operator.createOperatorServiceAccount=false \
  --set operator.createResourcesServiceAccountsAndRoles=false \
  --set "multiCluster.clusters={$MDB_CLUSTER_1_FULL_NAME,$MDB_CLUSTER_2_FULL_NAME,$MDB_CLUSTER_3_FULL_NAME}" \
  --set operator.watchNamespace="*"

Plan for External Connectivity: Should You Use a Service Mesh?

A service mesh enables inter-cluster communication between the replica set members deployed in different Kubernetes clusters. Using a service mesh greatly simplifies creating multi-Kubernetes cluster MongoDB deployments and is the recommended way of deploying MongoDB across multiple Kubernetes clusters. However, if your IT organization doesn't use a service mesh, you can deploy a replica set in a multi-Kubernetes cluster MongoDB deployment without it.

Depending on your environment, do the following:

• If you can use a service mesh, install Istio.

• If you can't use a service mesh:

  1. Review the section How Does the Kubernetes Operator Establish Connectivity?

  2. Enable external connectivity through external domains and DNS zones.

  3. Deploy a multi-Kubernetes cluster without a service mesh.

Check Connectivity Across Clusters

Follow the steps in this procedure to verify that service FQDNs are reachable across Kubernetes clusters.

In this example, you deploy a sample application defined in sample-service.yaml across two Kubernetes clusters.

kubectl create --context="${CTX_CLUSTER_1}" namespace sample
kubectl create --context="${CTX_CLUSTER_2}" namespace sample

kubectl apply --context="${CTX_CLUSTER_1}" \
   -f sample-service.yaml \
   -l service=helloworld1 \
   -n sample
kubectl apply --context="${CTX_CLUSTER_2}" \
   -f sample-service.yaml \
   -l service=helloworld2 \
   -n sample

kubectl apply --context="${CTX_CLUSTER_1}" \
   -f sample-service.yaml \
   -l version=v1 \
   -n sample

kubectl get pod --context="${CTX_CLUSTER_1}" \
   -n sample \
   -l app=helloworld

kubectl apply --context="${CTX_CLUSTER_2}" \
   -f sample-service.yaml \
   -l version=v2 \
   -n sample

kubectl get pod --context="${CTX_CLUSTER_2}" \
   -n sample \
   -l app=helloworld

kubectl run  --context="${CTX_CLUSTER_1}" \
   -n sample \
   curl --image=radial/busyboxplus:curl \
   -i --tty \
   curl -sS helloworld2.sample:5000/hello

Hello version: v2, instance: helloworld-v2-758dd55874-6x4t8

kubectl run --context="${CTX_CLUSTER_2}" \
   -n sample \
   curl --image=radial/busyboxplus:curl \
   -i --tty \
   curl -sS helloworld1.sample:5000/hello

Hello version: v1, instance: helloworld-v1-758dd55874-6x4t8

Review the Requirements for Deploying Ops Manager

As part of the Quick Start, you deploy an Ops Manager resource on the central cluster.

Prepare for TLS-Encrypted Connections

If you plan to secure your multi-Kubernetes cluster MongoDB deployment using TLS encryption, complete the following tasks to enable internal cluster authentication and generate TLS certificates for member clusters and the MongoDB Agent:

kubectl --context $MDB_CENTRAL_CLUSTER_FULL_NAME \
--namespace=<metadata.namespace> \

curl https://raw.githubusercontent.com/mon  mongodb-enterprise-kubernetes/master/tools/multicluster/setup_tl  -o  setup_tls.sh

Choose GitOps or the kubectl MongoDB Plugin

You can choose to create and maintain the resource files needed for the MongoDBMultiCluster resources deployment in a GitOps environment.

If you use a GitOps workflow, you can't use the kubectl mongodb plugin, which automatically configures role-based access control (RBAC) and creates the kubeconfig file that allows the central cluster to communicate with its member clusters. Instead, you must manually configure or build your own automation for configuring the RBAC and kubeconfig files based on the procedure and examples in Configure Resources for GitOps.

The following prerequisite sections describe how to install the kubectl MongoDB plugin if you don't use GitOps or configure resources for GitOps if you do.

Install the kubectl MongoDB Plugin

Use the kubectl mongodb plugin to:

• Set up multi-Kubernetes-cluster Deployments

• Run automatic and manual disaster recovery

To install the kubectl mongodb plugin:

tar -zxvf kubectl-mongodb_<version>_darwin_amd64.tar.gz

mv kubectl-mongodb /usr/local/bin/kubectl-mongodb

kubectl mongodb multicluster setup
kubectl mongodb multicluster recover

Configure Resources for GitOps

If you use a GitOps workflow, you won't be able to use the kubectl mongodb plugin to automatically configure role-based access control (RBAC) or the kubeconfig file that allows the central cluster to communicate with its member clusters. Instead, you must manually configure and apply the following resource files or build your own automation based on the information below.

To configure RBAC and the kubeconfig for GitOps:

apiVersion: v1
kind: ConfigMap
data:
  cluster1: ""
  cluster2: ""
metadata:
  namespace: <namespace>
  name: mongodb-enterprise-operator-member-list
  labels:
    multi-cluster: "true"