Verify Data Transfer

Starting in 1.9, mongosync includes an embedded verifier, which runs a series of verification checks on the source and destination clusters to confirm that the migration was successful. This is the preferred verification method for deployments that meet the requirements.

When the mongosync process starts, it prompts the user with a disclaimer about the embedded verifier. You must accept the disclaimer, or if you have already read and acknowledged it, start mongosync with the --acceptDisclaimer option to use the verifier.

The verifier doesn't check every aspect of a migration:

• To verify sync on unsupported namespaces and sharded clusters, use a different verification method.

• To verify index sync, use the Index Comparison method.

• To verify metdata sync, use the Metadata Comparison method.

The most basic method of verification is to compare the number of documents in each synced collection on the source cluster to the number on the destination cluster.

Before you can verify data transfer with this method, mongosync must be in the COMMITTED state.

This method only verifies a successful sync when run against clusters with insert-only workloads.

You can verify sync by comparing MD5 hashes of collections synced from the source cluster to the destination cluster.

Before you can verify data transfer with this method, mongosync must be in the COMMITTED state.

While hash comparison ensures that the destination cluster has received all changes from the source, the dbHash command locks the cluster, preventing additional writes until it completes.

Hash comparison is not possible with sharded clusters. It also does not work with standalone servers and replica sets that use MongoDB 4.4 or earlier releases, since the document field order can vary.

You can verify sync by comparing documents on the source and destination clusters.

Write a script that queries collections on the source cluster and then checks that the correct documents, indexes, collections, metadata, and views exist with the same values on the destination cluster.

Before you can verify data transfer with this method, mongosync must be in the COMMITTED state.

To verify the transfer of indexes, run the db.collection.getIndexes() method on the source and destination clusters and compare the results.

To verify the transfer of metadata, run the db.getCollectionInfos() method on the source and destination clusters and compare the results.

To verify the transfer of shard keys to a synced collection, run a query on the config.collections collection to find a document whose _id value is the namespace of the target collection. Compare the key value of this document in the source and destination clusters.

For example, for a collection named pets in the records database, you can verify the shard key using the following query in mongosh:

db.getSiblingDB("config").collections.find({ _id : "records.pets" })

{
    "_id" : "records.pets",
    "lastmod" : ISODate("2021-07-21T15:48:15.193Z"),
    "timestamp": Timestamp(1626882495, 1),
    "key" : {
          "_id" : 1
    },
    "unique" : false,
    "lastmodEpoch" : ObjectId("5078407bd58b175c5c225fdc"),
    "uuid" :  UUID("f8669e52-5c1b-4ea2-bbdc-a00189b341da")
}

Migration Verifier connects to the source and destination clusters and performs a series of verification checks, comparing documents, views, and indexes to confirm the sync was successful.

Migration Verifier is an experimental and unsupported tool.