Atlas Search Index Performance

Index Size and Configuration

When you create an Atlas Search index, field mapping defaults to dynamic, which means that Atlas Search dynamically indexes all the datatypes that can be dynamically indexed in your collection. Other options such as enabling highlights can also result in your index taking up more disk space. You can reduce the size and performance footprint of your Atlas Search index by:

• Specifying a custom index definition to narrow the amount and type of data that is indexed.

• Setting the store option to false when specifying a string type in an index definition.

{
  "_id": 0,
  "name": "Springfield High",
  "mascot": "Pumas",
  "teachers": [
    {
      "first": "Jane",
      "last": "Smith",
      "classes": [
        {
          "subject": "art of science",
          "grade": "12th"
        },
        ... // 2 more embedded documents
      ]
    },
    ... // 1 more embedded document
  ],
  "clubs": {
    "stem": [
      {
        "club_name": "chess",
        "description": "provides students opportunity to play the board game of chess informally and competitively in tournaments."
      },
      ... // 1 more embedded document
    ],
    ... // 1 more embedded document
  }
}

Creating and Updating an Atlas Search Index

Creating an Atlas Search index is resource-intensive. The performance of your Atlas cluster may be impacted while the index builds.

Atlas replicates all writes on the collection. This means that for each collection with Atlas Search indexes, the writes are amplified to the amount of Atlas Search indexes defined for that collection.

In some instances, your Atlas Search index must be rebuilt. Rebuilding the Atlas Search index also consumes resources and may affect database performance. Atlas Search automatically rebuilds the index only in the event of:

• Changes to the index definition

• Atlas Search version updates that include breaking changes

• Hardware-related problems such as index corruption

Eventual Consistency and Indexing Latency

Atlas Search supports eventual consistency and does not provide any stronger consistency guarantees. This means that data inserted into a MongoDB collection and indexed by Atlas Search will not be available immediately for $search queries.

Atlas Search reads data from MongoDB change streams and indexes that data in an asynchronous process. This process is typically very fast, but might sometimes be impacted by replication latency, system resource availability, and index definition complexity. A large number of Atlas Search indexes might also contribute to replication lag and latency for Atlas Search indexes.

Document Mapping Explosions

Mapping explosions occur when Atlas Search indexes a document with arbitrary keys and you have a dynamic mapping. The mongot process might consume increasing amounts of memory and could crash. If you add too many fields to an index, mapping explosions can occur. To address this issue, you can upgrade your cluster or use a static mapping that does not index all fields in your data.

When searching over fields using a wildcard path, design your search to use a tuple-like schema. If you perform a wildcard path search that uses a key-value schema, Atlas Search indexes each key as its own field, which can cause mapping explosions.

ruleBuilder: {
  ruleName1: <data>,
  ruleName2: <data>,
  .....
  ruleName1025: <data>
}

{
  ruleBuilder: [
    {name: ruleName1, data: <data>},
    {name: ruleName2, data: <data>},
    ...
    {name: ruleName1025, data: <data>}
  ]
}

Storing Source Fields

You can configure fields to store on Atlas Search and improve performance of subsequent aggregation pipeline stages like $sort, $match, $group, and $skip. Use this optimization if your original documents and matched dataset are so large that a full data lookup is inefficient. To learn more about storing specific fields on Atlas Search and returning those stored fields only, see Define Stored Source Fields in Your Atlas Search Index and Return Stored Source Fields.

We recommend storing only the minimum number of fields required for subsequent stages. If necessary, you can use $lookup at the end of the pipeline stage to retrieve entire documents as shown in the Examples. Storing unnecessary fields increases disk utilization and could negatively impact performance during indexing and querying.

Atlas Search Upgrade

Atlas Search is deployed on your Atlas cluster. When a new version of Atlas Search is deployed, your Atlas cluster might experience brief network failures in returning query results. To mitigate issues during deployment and minimize impact to your application, consider the following:

• Implement retry logic in your application.

• Configure Atlas maintenance windows.

  Note

  Atlas Search upgrades start only during the maintenance window and might continue after the maintenance window.

To learn more about the changes in each release, see Atlas Search Changelog.

Scaling Up Indexing Performance

You can scale up your initial sync and steady state indexing for an Atlas Search index by upgrading your cluster to a higher tier with more cores. Atlas Search uses a percentage of all available cores to run both initial sync and steady state indexing and performance improves as new cores are made available by upgrading your cluster.

Atlas Cluster Configuration Change

If you reconfigure your deployment to use the local NVMe storage type or upscale an NVMe-based cluster, Atlas Search performs an initial sync of all configured Atlas Search indexes after each node completes its underlying configuration or upscale action. If the Atlas Search index initial syncs take longer than the time it took to complete the cluster configuration change, you can't run $search queries until the initial sync completes on all the nodes in your Atlas cluster.

We recommend deploying dedicated Search Nodes to scale your Atlas cluster and $search workloads independently. Dedicated Search Nodes run only the mongot process and therefore improve the availability, performance, and workload balancing of the mongot process.