Live Migrate (Push) a Replica Set Monitored by Ops Manager or Cloud Manager (MongoDB Before 6.0.13)
On this page
Atlas can facilitate a live migration where if Cloud Manager or Ops Manager monitors a source replica set cluster, the migration service pushes this cluster to an Atlas cluster. Atlas keeps the destination cluster in sync with the source cluster until you cut your applications over to the destination cluster in Atlas.
Once you reach the cutover step in the following procedure, stop the writes to the source cluster. Stop your application instances, point them to the Atlas cluster, and restart them.
Restrictions
You can't select an
M0
(Free Tier) orM2/M5
shared cluster as the destination for live migration. To migrate data from anM0
(Free Tier) orM2/M5
shared cluster to a paid cluster, see Modify a Cluster.You can't live migrate using this migration procedure to an Atlas destination cluster that has BI Connector for Atlas enabled.
You can't live migrate to an Atlas destination cluster if you use Ops Manager in local mode.
During live migration, Atlas disables host alerts.
If your source and destination clusters are running MongoDB 6.0.13 or later, you can live migrate to Atlas using this procedure.
To live migrate a sharded cluster, see Live Migrate (Push) a Sharded Cluster Monitored by Ops Manager or Cloud Manager (MongoDB Before 6.0.13).
Support for VPC Peering and Private Endpoints
The following table lists the current support status for VPC peering and private endpoints for source and destination replica set clusters that you live migrate to Atlas.
Cloud Provider | VPC Peering | Private Endpoints |
---|---|---|
Azure | ||
AWS | ||
Google Cloud |
To enable VPC peering with live migration on Azure, AWS, or Google Cloud:
Configure a VPC peering connection between the migration host and the destination Atlas cluster.
To enable private endpoints with live migration on Azure, AWS, or Google Cloud:
Configure a private endpoint between the migration host and the destination Atlas cluster.
Note
Private endpoints are supported ONLY for live migration of replica sets deployed within a single cloud provider and single region. Sharded clusters, multi-region clusters, and multi-cloud clusters don't support live migration through private endpoints.
Required Access
To live migrate your data, you must have Organization Owner
access
to Atlas.
Prerequisites
Before you begin the live migration from Cloud Manager or Ops Manager to Atlas:
While the supported migration paths allow you to migrate from a source cluster on MongoDB 4.0 or 4.2 to a cluster in Atlas, we highly recommend that before you use this procedure, you upgrade the source cluster to MongoDB 5.0 or later.
Create an Atlas organization and then create a project in this organization.
Deploy your cluster in this project.
Connect to your cluster from all client servers where your applications run.
If migrating from Ops Manager, upgrade Ops Manager to version 5.0.
When you migrate from MongoDB 4.4 or earlier to an Atlas cluster that runs MongoDB 5.0 or later, drop any geoHaystack indexes from your collections.
During the live migration process, Atlas validates that it can collect MongoDB database statistics using dbStats. Before you live migrate to an Atlas cluster, review the project settings for the source cluster in Cloud Manager or review the project settings for the source cluster in Ops Manager and ensure that the option Collect Database Specific Statistics is enabled. This option is enabled by default in Cloud Manager or Ops Manager, and it should remain enabled so that the migration process passes validation.
On your source cluster in Cloud Manager or Ops Manager, prepare the following items:
Provision a migration host in Ops Manager, or provision a migration host in Cloud Manager.
Obtain the following external IP addresses from your Cloud Manager or Ops Manager administrator:
If migrating from Ops Manager, the external IP addresses or CIDR blocks of the Ops Manager instances. If migrating from Cloud Manager, Atlas automatically obtains these addresses.
The external IP addresses or CIDR blocks of the provisioned migration hosts in Cloud Manager or Ops Manager.
The username and password used to connect to the source cluster.
If the source cluster uses TLS/SSL with a Custom Root Certificate Authority, to ensure the hosts can read the certificate, add the source cluster's CA file to the migration hosts.
Consider configuring a VPC peering connection or a private endpoint between each migration host and the destination Atlas cluster on the same cloud provider and in the same region as the destination cluster.
Note
If you choose not to use VPC peering or private endpoints, the live migration process runs over public IP addresses that you add to the Atlas project's IP access list as part of the live migration procedure.
Live Migration Workflow
This section outlines the workflow. For detailed steps, see the procedure for migrating a replica set from Ops Manager or Cloud Manager to Atlas.
!["To live migrate your deployment to Atlas, generate a link-token,
provision a migration host, and start live migration."](/docs/atlas/static/cbd9ffbc59f3c6468423161d6c851d0c/047e5/live-migration-stages.webp)
The stages in the live migration workflow are:
Stage 1: Link with Atlas. Perform this step in Atlas, after you have created your Atlas account, organization, and project; deployed your dedicated cluster in this project; and can connect to it.
In Atlas, go to the Organization Settings page.
If it's not already displayed, select your desired organization from the Organizations menu in the navigation bar.
Click the Organization Settings icon next to the Organizations menu.
The Organization Settings page displays.
Click Live Migration in the sidebar.
The Live Migration to Atlas page displays.
Select Migrate from Ops Manager or Cloud Manager and start the live migration wizard.
If you are migrating from MongoDB Community using Ops Manager, accept the Ops Manager Migration Agreement.
If you are migrating from Ops Manager, enter the external IP addresses of your Ops Manager instances to the Atlas access list. If you are migrating from Cloud Manager, skip this step.
Stage 2: Provision Migration Host.
Provision a migration host in Ops Manager, or provision a migration host in Cloud Manager. A migration host runs a dedicated MongoDB Agent that orchestrates the live migration process from Cloud Manager or Ops Manager to Atlas.
Note
If you are migrating a source MongoDB deployment that hasn't used Ops Manager or Cloud Manager before, add existing MongoDB processes to Ops Manager or add existing MongoDB processes to Cloud Manager.
In the Live Migration: Connect to Atlas section of your Cloud Manager or Ops Manager organization's Settings page, select Connect to Atlas and paste the link-token that you created in Atlas. To learn more, see Connect to Atlas for Live Migration in Ops Manager, or Connect to Atlas for Live Migration in Cloud Manager.
Stage 3: Start the Migration. In Atlas, follow the steps in the wizard to start the live migration process.
Migration Path and Supported Platforms
Live Migration from Cloud Manager or Ops Manager to Atlas is supported for all platforms on which you can provision a migration host. For a full list of supported platforms on which you can provision a migration host, see Cloud Manager Prerequisites in Cloud Manager or Ops Manager Prerequisites in Ops Manager.
If you want to live migrate your data from a Windows- or macOS-based deployment to Atlas, you must provision your migration host on one of the supported platforms.
Atlas live migration (push) supports the following migration paths:
Source Replica Set MongoDB Version | Destination Atlas Replica Set MongoDB Version |
---|---|
4.2 | 5.0 |
4.4 | 5.0 |
5.0 | 5.0 |
Network Access for Live Migrating from Ops Manager or Cloud Manager
Contact your Cloud Manager or Ops Manager service administrator and obtain external IP addresses for the following components:
External IP Address of the Source Cluster in Ops Manager
If you are migrating from Ops Manager, the live migration service in Atlas requires an external IP address or an external CIDR block of the Ops Manager instance in the source cluster deployment. This address could be from a single Ops Manager instance, or, if the source deployment uses multiple Ops Manager instances, from the gateway the Ops Manager instances will use to reach Atlas.
The live migration service uses this external IP address when generating a link-token. A link-token is a string that contains the information necessary to connect from Cloud Manager or Ops Manager to Atlas during a live migration from a Cloud Manager or Ops Manager deployment to a cluster in Atlas.
External IP Address of the Migration Host in Ops Manager or Cloud Manager
Before you begin the live migration procedure, add the IP addresses or CIDR blocks of your migration hosts to the project IP access list. Atlas allows connections to the destination cluster only from hosts with entries in the project's access list.
Pre-Migration Validation
Before starting the live migration procedure, Atlas runs validation checks on the source and destination clusters.
The source cluster is a replica set.
If the source cluster is a standalone, convert the standalone MongoDB node in Ops Manager, or convert the standalone MongoDB node in Cloud Manager to a single-node replica set and then live migrate it to Atlas using the procedure in this section.
The source cluster enables collecting database statistics for its project in Cloud Manager or Ops Manager. This allows Atlas to collect MongoDB database statistics during the live migration process. To confirm that the option Collect Database Specific Statistics is enabled, review the project settings for the source cluster in Cloud Manager or review the project settings for the source cluster in Ops Manager.
The destination Atlas cluster is a replica set and it doesn't have BI Connector for Atlas enabled.
Security Requirements for the Migration Host
For push-type live migrations, you're responsible for provisioning, securing, and running the migration host. The migration host only encrypts outbound communication with Atlas clusters.
To learn more about Atlas security, see the Atlas Security whitepaper.
Index Key Limits
If your MongoDB deployment contains indexes with keys which exceed the Index Key Limit, before you start the live migration procedure, modify indexes so that they do not contain oversized keys.
Considerations
Network Encryption
During push live migrations, if the source cluster does not use TLS encryption for its data, the traffic from the source cluster to the migration host is not encrypted, but the traffic from the migration host to Atlas is encrypted. Determine if this is acceptable before you start a push live migration procedure.
Database Users and Roles
If the source cluster doesn't use authentication, you must create a user in Atlas because Atlas doesn't support running without authentication.
Atlas doesn't migrate any user or role data to the destination cluster.
If the source cluster enforced authentication, before you migrate you must re-create the appropriate authentication mechanism used by your applications on the destination Atlas cluster. The following table lists authentication mechanisms and how to configure them in Atlas.
Authentication Mechanism | Configuration Method |
---|---|
SCRAM | |
LDAP | |
AWS KMS, Azure Key Vault, Google Cloud KMS |
Destination Cluster Configuration
For the destination cluster, consider the following:
The destination cluster in Atlas must match or exceed the source deployment in terms of RAM, CPU, and storage. Provision a destination cluster of an adequate size so that it can accommodate both the migration process and the expected workload, or scale up the destination cluster to a tier with more processing power, bandwidth or disk IO.
To maximize migration performance, use at least an M40 cluster for the destination cluster. When migrating large data sets, use an M80 cluster with 6000 IOPS disks or higher.
You can also choose to temporarily increase the destination Atlas cluster's size for the duration of the migration process.
After you migrate your application's workload to a cluster in Atlas, contact support for assistance with further performance tuning and sizing of your destination cluster to minimize costs.
To avoid unexpected sizing changes, disable auto-scaling on the destination cluster. To learn more, see Manage Clusters.
To prevent unbounded oplog collection growth, and to ensure that the live migration's lag window stays within the bounds of the oplog replication lag window, set an oplog size to a large enough fixed value for the duration of the live migration process.
To learn more, see:
oplog Sizing in the Cluster-to-Cluster Sync documentation.
If you are observing performance issues even after you've followed these recommendations, contact support.
The destination Atlas cluster must be a replica set.
You can't select an
M0
(Free Tier) orM2/M5
shared-tier cluster as the destination for live migration.Do not change the
featureCompatibilityVersion
flag while Atlas live migration is running.
Avoid Workloads on the Destination Cluster
Avoid running any workloads, including those that might be running on namespaces that don't overlap with the live migration process, on the destination cluster. This action avoids potential locking conflicts and performance degradation during the live migration process.
Don't run multiple migrations to the same destination cluster at the same time.
Don't start the cutover process for your applications to the destination cluster while the live migration process is syncing.
Avoid Cloud Backups
Atlas stops taking on-demand cloud backup snapshots of the target cluster during live migration. Once you complete the cutover step in the live migration procedure on this page, Atlas resumes taking cloud backup snapshots based on your backup policy.
Avoid Namespace Changes
Don't make any namespace changes during the migration
process, such as using the
renameCollection
command
or executing an aggregation pipeline that includes the
$out
aggregation stage.
Avoid Elections
The live migration process makes a best attempt to continue a migration during temporary network interruptions and elections on the source or destination clusters. However, these events might cause the live migration process to fail. If the live migration process can't recover automatically, restart it from the beginning.
Staging and Production Environments
Consider running the following procedure twice. Perform a partial migration that stops at the Perform the Cutover step first. This creates an up-to-date Atlas-backed staging cluster to test application behavior and performance using the latest driver version that supports the MongoDB version of the Atlas cluster.
After you test your application, run the full migration procedure using a separate Atlas cluster to create your Atlas-backed production environment.
Migrating from MongoDB Community
If you are migrating to Atlas from MongoDB Community using Ops Manager, you must accept the Ops Manager Migration Agreement as the first step in the Live Migration procedure.
Migrate Your Cluster
Important
Avoid making changes to the source cluster configuration while the
live migration procedure runs, such as removing replica set members
or modifying mongod
runtime settings, such as
featureCompatibilityVersion
.
Procedure
In Atlas, go to the Organization Settings page.
If it's not already displayed, select your desired organization from the Organizations menu in the navigation bar.
Click the Organization Settings icon next to the Organizations menu.
The Organization Settings page displays.
Go to the Live Migration to Atlas page.
Click Live Migration in the sidebar.
The Live Migration to Atlas page displays.
Start the migration process.
Click Migrate from Ops Manager or Cloud Manager.
Click I'm Ready to Start and, if you are migrating from MongoDB Community using Ops Manager, click Ops Manager License Agreement to read and accept it. To read the agreement outside of the Live Migration Wizard, see the Ops Manager Migration Agreement.
Atlas displays a Live Migration wizard with instructions on how to proceed with the process. The process pushes the data from your source cluster to the new destination cluster. After you complete the wizard steps, you can point your application to the new cluster.
Link with Atlas.
Click Generate Link-Token. Atlas displays the page for generating a link-token.
If you are migrating from Ops Manager, enter the external IP addresses or a CIDR block of your Ops Manager instances to add them to the access list of the API Key embedded in the link-token. This allows Ops Manager to send project and cluster information to Atlas. Ops Manager does not send any data stored in your MongoDB databases in this step. For example, enter
23.248.95.14
.If you are migrating from Cloud Manager, skip this step.
Click Next to see a page that contains the generated link-token.
Copy the link-token and store it in a secure location. Atlas never displays the contents of the link-token. Atlas also does not display the link-token after generating it. Do not share it publicly.
Note
Use one unique link-token for live migrating all projects in one Cloud Manager or Ops Manager organization to Atlas.
Click Done.
Paste the link-token into Ops Manager or Cloud Manager.
Access the organization in Cloud Manager or Ops Manager:
Open Ops Manager or Cloud Manager, and navigate to the organization whose project's cluster you are live migrating to Atlas.
Click Settings in the left navigation panel.
In the Live Migration: Connect to Atlas section, click Connect to Atlas. The Connect to Atlas dialog box opens.
Paste the link-token you generated in the previous step of the Live Migration wizard and click Connect to Atlas. Cloud Manager or Ops Manager establishes the connection to Atlas. Use the Refresh button to send an update to Atlas, if needed. To learn more, see Connect to Atlas for Live Migration in Ops Manager, or Connect to Atlas for Live Migration in Cloud Manager.
Create the destination Atlas cluster.
If you haven't already, create a destination cluster in Atlas. See Prerequisites.
Initiate the migration from the destination cluster.
Click Select Target Cluster from Projects.
Go to your destination Atlas cluster's project and find your destination cluster.
Click and select Migrate Data to this Cluster from the dropdown list to start the migration. The Migrate Data to This Cluster page opens.
Click Migrate from Ops Manager or Cloud Manager and fill in the fields as follows:
Select the source project in Cloud Manager or Ops Manager, if it's not already selected.
Select a migration host for handling the migration.
Review the IP address access list and check that the migration host's external IP address is included in this list. If it's not added, add it now:
Click Set Network Access for Host
Click + Add IP Address
Return to the Live Migration wizard. Select the source cluster from the dropdown and choose Migrate data to this cluster under .
Select the source cluster from the drop-down.
If you suspend the source cluster from automation in Cloud Manager or Ops Manager, but continue to monitor the source cluster with the Monitoring Agent, the Username and Password display. If your deployment requires user authentication, provide the user name and password in these fields. The database user whose credentials you provide must have at least the backup role on the admin database and must be authenticated using both SCRAM-SHA-1 and SCRAM-SHA-256.
If the source cluster uses TLS/SSL, toggle the Is encryption in transit enabled? button.
If the source cluster uses TLS/SSL with a custom Root Certificate Authority (CA), copy the path to the CA file from your migration host and paste this path into the provided text box. The file must be present on the migration host to ensure the migration host can read the certificate. Atlas checks that the certificate is present and readable.
If your replica set destination cluster has data, and you want to preserve it, keep the Clear any existing data on your destination cluster option unchecked. The live migration service warns you if it finds duplicate namespaces. If you want to delete the existing data, check this option.
Choose a connection to connect to the cluster. The Standard connection always shows as available in the UI. However, other connection options are enabled only if you have previously configured a VPC peering connection or a private endpoint. If Atlas detects that you don't have VPC connections or private endpoints configured, these options are grayed out.
If you aren't using VPC peering or a private endpoint, click Standard connection and proceed to the Validation stage of this step.
If you configured a VPC peering connection between the migration host and the Atlas cluster, the VPC Peering option is active. Click VPC Peering to connect using VPC peering for live migration. If the VPC Peering option is grayed out, configure a VPC peering connection before starting this procedure. To learn more, see Support for VPC Peering and Private Endpoints.
If you are migrating a replica set and configured a private endpoint between the migration host and the Atlas cluster, the Private Endpoint option is active. Click Private Endpoint to connect with a private endpoint, and then select a previously-configured private endpoint from the dropdown. Only private endpoints that are in
AVAILABLE
state are valid. If the Private Endpoint option is grayed out, configure a private endpoint before starting this procedure. To learn more, see Support for VPC Peering and Private Endpoints.Note
Private endpoints are supported ONLY for live migration of replica sets deployed within a single cloud provider and single region. Sharded clusters, multi-region clusters, and multi-cloud clusters don't support live migration through private endpoints.
Click Validate. The validation process verifies that your migration host is reachable, and performs the following validation checks to ensure that you can start live migration to Atlas.
To take advantage of the following validation checks, upgrade the MongoDB Agent in Ops Manager, or upgrade the MongoDB Agent in Cloud Manager to the latest version. If the live migration process detects that the source cluster is running a version of MongoDB Agent earlier than 12.0.11.7606 for Ops Manager, or 12.5.0.7713 for Cloud Manager, it displays a banner warning suggesting that you upgrade the MongoDB Agent.
The following validation checks run during the live migration:
The migration host can connect to the destination cluster.
If the source cluster uses TLS/SSL with a custom Root Certificate Authority (CA), the migration host can access the source cluster using TLS/SSL.
The database user credentials are valid. This validation check runs only if you suspend the source cluster from automation in Cloud Manager or Ops Manager, but continue to monitor the source cluster with the Monitoring Agent.
If you are migrating from Ops Manager before version 5.0.9, the live migration process validates that the destination cluster has enough disk space based on the data size. If you are migrating from Cloud Manager or Ops Manager 5.0.9 or later, the migration process validates that the destination cluster has enough disk space based on the storage size of the compressed data. To learn more about data and storage sizes, see dbStats.
If validation fails, check the migration host, the validity of your external IP addresses or CIDR block, and the link-token. Also check the database user credentials, your TLS/SSL certificates, and the amount of disk storage size on the destination cluster.
If validation succeeds, click Next. The Migrate from Ops Manager or Cloud Manager page displays.
Prepare to Cut Over.
A lag time value dispays during the final oplog tailing phase that represents the current lag between the source and destination clusters. This lag time may fluctuate depending on the rate of oplog generation on the source cluster, but should decrease over time as the live migration process copies the oplog entries to the destination cluster.
When the lag timer and the Prepare to Cutover button turn green, click it to proceed to the next step.
Perform the cutover.
When Atlas detects that the source and destination clusters are nearly in sync, it starts an extendable 120 hour (5 day) timer to begin the cutover stage of the live migration procedure. If the 120 hour period passes, Atlas stops synchronizing with the source cluster. You can extend the time remaining by 24 hours by clicking Extend time below the <time> left to cut over timer.
To finish migrating your MongoDB data to your Atlas cluster, complete the following steps on the Your migration is almost complete! page:
Prepare to point to your Atlas cluster. Copy the new connection string so that you can update it and point your application to the destination Atlas cluster.
Stop your application. This action ensures that no more writes occur on the source cluster.
Wait for the optime gap to reach zero. When the counter reaches zero, the source and destination clusters are in sync.
Wait for additional time until Atlas configures your destination cluster and it is ready to use. For smaller clusters, this time is 3-5 minutes. For larger clusters, this time can extend up to 10 minutes or longer, depending on the cluster size and configuration.
Restart your application using the new Atlas connection string and confirm that your application is working with the Atlas cluster.
Click Cut Over to complete the migration process.
If you click Cancel on the live migration progress bar, Atlas stops synchronizing writes from the source cluster. All migrated data remains on your Atlas cluster.
You can click Cut Over again to allow Atlas to complete the migration process.
When you are ready to redirect writes to the destination cluster in Atlas:
Use the destination cluster's connection string to connect to your application.
Confirm that your application is working with the destination Atlas cluster.
Verify your data on the destination cluster.
Resume writes to the destination cluster.
Atlas performs these actions to complete the process:
Removes the MongoDB live migration server subnets from the IP access list on the destination cluster.
Removes the database user that live migration used to import data to the destination cluster.
Marks the migration process as complete.
Push Live Migration APIs
To run tasks associated with the live migration procedure, see Push Live Migration API.
Migration Support
If you have any questions regarding migration support beyond what is covered in this documentation, or if you encounter an error during migration, please request support through the Atlas UI.
To file a support ticket:
In Atlas, go to the Project Support page.
If it's not already displayed, select the organization that contains your desired project from the Organizations menu in the navigation bar.
If it's not already displayed, select your desired project from the Projects menu in the navigation bar.
Next to the Projects menu, expand the Options menu, then click Project Support.
The Project Support page displays.
Request support.
Click Request Support.
For Issue Category, select
Help with live migration
.For Priority, select the appropriate priority. For questions, please select
Medium Priority
. If there was a failure in migration, please selectHigh Priority
.For Request Summary, please include
Live Migration
in your summary.For More details, please include any other relevant details to your question or migration error.
Click the Request Support button to submit the form.
Push Live Migration CLI Commands
To migrate a cluster using the Atlas CLI, you can perform the following steps:
Create or delete a link-token
Create or view a validation job
Create or view a migration job
Perform the cutover
For other steps in the live migration procedure, you must use the Cloud Manager UI or the Atlas UI. To learn more, see the live migration workflow.
Before you migrate a cluster using the Atlas CLI, complete the pre-migration validation.
Note
Before you run any Atlas CLI commands, you must:
Create or Delete a Link-Token
To create a new link-token using the Atlas CLI, run the following command:
atlas liveMigrations link create [options]
To delete the link-token you specify using the Atlas CLI, run the following command:
atlas liveMigrations link delete [options]
To learn more about the syntax and parameters for the previous commands, see the Atlas CLI documentation for atlas liveMigrations link create and atlas liveMigrations link delete.
If you are migrating from Ops Manager, request an external IP address and specify it in the link-token. To learn more, see Request an External IP Address in the Ops Manager documentation.
Create and View a Validation Job
To create a new validation request using the Atlas CLI, run the following command:
atlas liveMigrations validation create [options]
To return the details for the validation request you specify using the Atlas CLI, run the following command:
atlas liveMigrations validation describe [options]
To learn more about the syntax and parameters for the previous commands, see the Atlas CLI documentation for atlas liveMigrations validation create and atlas liveMigrations validation describe.
To learn what Atlas validates, see the Validate bullet in the Migrate Your Cluster section on this page.
Create and View a Migration Job
To create one new migration job using the Atlas CLI, run the following command:
atlas liveMigrations create [options]
To return the details of the migration job you specify using the Atlas CLI, run the following command:
atlas liveMigrations describe [options]
To learn more about the syntax and parameters for the previous commands, see the Atlas CLI documentation for atlas liveMigrations create and atlas liveMigrations describe.
Perform the Cutover
To start the cutover for live migration using the Atlas CLI, run the following command:
atlas liveMigrations cutover [options]
To learn more about the command syntax and parameters, see the Atlas CLI documentation for atlas liveMigrations cutover.
When the cutover completes, Atlas completes the live migration process and stops synchronizing with the source cluster. To learn more, see the Migrate Your Cluster section on this page.