Set Up Data Federation

On this page

Prerequisites

Procedure
Create the AWS IAM role in Atlas
Modify your AWS IAM role trust policy.
Create the AtlasDataFederation custom resource.
Check the status of your federated database instance.
Take the Next Steps

This tutorial demonstrates how to create a federated database instance in Atlas from Kubernetes configuration files with Atlas Kubernetes Operator. The federated database instance in this tutorial connects an AWS S3 bucket and an Atlas cluster.

Prerequisites

This tutorial requires:

A running Kubernetes cluster with Atlas Kubernetes Operator deployed, including an AtlasProject Custom Resource.
An AWS user account with permissions to modify IAM roles.
The Atlas CLI.

Procedure

Important

Custom Resources No Longer Delete Objects by Default

Atlas Kubernetes Operator uses custom resource configuration files to manage your Atlas configuration, but as of Atlas Kubernetes Operator 2.0, custom resources you delete in Kubernetes are no longer (by default) deleted in Atlas. Instead, Atlas Kubernetes Operator simply stops managing those resources in Atlas. For example, if you delete an AtlasProject Custom Resource in Kubernetes, by default the Atlas Kubernetes Operator no longer automatically deletes the corresponding project from Atlas. This change in behavior is intended to help prevent accidental or unexpected deletions. To learn more, including how to revert this behavior to the default used prior to Atlas Kubernetes Operator 2.0, see New Default: Deletion Protection in Atlas Kubernetes Operator 2.0.
Similarly, Atlas Kubernetes Operator does not delete teams from Atlas if you remove them from an Atlas project in Kubernetes with the Atlas Kubernetes Operator.

Explicitly define your desired configuration details in order to avoid implicitly using default Atlas configuration values. In some cases, inheriting Atlas defaults may result in a reconciliation loop which can prevent your custom resource from achieving a READY state. For example, explicitly defining your desired autoscaling behavior in your AtlasDeployment custom resource, as shown in the included example, ensures that a static instance size in your custom resource is not being repeatedly applied to an Atlas deployment which has autoscaling enabled.

autoScaling:
  diskGB:
    enabled: true
  compute:
    enabled: true
    scaleDownEnabled: true
    minInstanceSize: M30
    maxInstanceSize: M40

Create the AWS IAM role in Atlas

Run the following Atlas CLI command to create the new AWS IAM role in Atlas. Replace the following placeholder with your value:

Placeholder	Description
`PROJECT-ID`	Unique 24-character hexadecimal string that identifies the Atlas project to use.

atlas cloudProviders accessRoles aws create --projectId <PROJECT-ID>

Note the returned field values RoleID, Atlas AWS Account ARN, and Unique External ID:
```
AWS IAM role '<RoleID>' successfully created.
Atlas AWS Account ARN: <AtlasAWSAccountARN>
Unique External ID: <AtlasAssumedRoleExternalID>
```

Modify your AWS IAM role trust policy.

Log in to your AWS Management Console.
Navigate to the Identity and Access Management (IAM) service.
Select Roles from the left-side navigation.
Click on the existing IAM role you wish to use for Atlas access from the list of roles.
Select the Trust Relationships tab.
Click the Edit trust relationship button.

Edit the Policy Document. Add a new Statement object with the following content.

Note

Replace the highlighted lines with values returned in the previous step.

{
   "Version":"2012-10-17",
   "Statement":[
      {
         "Effect":"Allow",
         "Principal":{
            "AWS":"<atlasAWSAccountArn>"
         },
         "Action":"sts:AssumeRole",
         "Condition":{
            "StringEquals":{
               "sts:ExternalId":"<atlasAssumedRoleExternalId>"
            }
         }
      }
   ]
}

Click the Update Trust Policy button.

Create the `AtlasDataFederation` custom resource.

Run the following command to create the AtlasDataFederation Custom Resource. The roleId should match the value returned for RoleID in the previous step and the spec.projectRef.name should match the name of your AtlasProject Custom Resource:

cat <<EOF | kubectl apply -f -
apiVersion: atlas.mongodb.com/v1
kind: AtlasDataFederation
metadata:
  name: my-federated-deployment
spec:
  projectRef:
    name: my-project
    namespace: default
  cloudProviderConfig:
    aws:
      roleId: 12345678
      testS3Bucket: my-bucket
  dataProcessRegion:
    cloudProvider: AWS
    region: OREGON_USA
  name: my-fdi
  storage:
    databases:
    - collections:
      - dataSources:
        - allowInsecure: false
          collection: my-collection
          collectionRegex:
          database: my-database
          databaseRegex:
          defaultFormat: ".avro"
          path: /
          provenanceFieldName: string
          storeName: my-data-store
          urls:
          - string:
        name: my-collection-mdb
      maxWildcardCollections: 100
      name: my-database-mdb
      views:
      - name: my-view
        pipeline:
        source: my-source-collection
    stores:
    - name: my-store
      provider: S3
      additionalStorageClasses:
      - STANDARD
      bucket: my-bucket
      delimiter: /
      includeTags: false
      prefix: data-
      public: false
      region: US_WEST_1
EOF

Check the status of your federated database instance.

Run the following command until you recieve a True response, which indicates the database user is ready:

kubectl get atlasdatafederation my-federated-deployment -o=jsonpath='{.status.conditions[?(@.type=="Ready")].status}'

Take the Next Steps

To configure private endpoints for your federated database instance, see Manage Private Endpoints.

Back

Import Projects

Atlas Search