Docs Menu

Configure Index Partition

Important

The numPartitions option is available as a Preview feature.

For indexing, Atlas Search counts each document as a single index object when it isn't nested inside another document. For embedded documents, Atlas Search counts each embedded document as additional index objects depending on the number of levels of nesting. Atlas Search stops replicating changes for indexes larger than 2,100,000,000 index objects.

If you deployed Atlas Search on separate search nodes, you can increase the number of Atlas Search index objects by partitioning your index objects in to sub-indexes. By default, Atlas Search supports one partition per shard. Each partition supports up to 2 billion index objects. You can create up to sixty-four (64) sub-indexes by using the numPartitions option.

When you configure partitions for your index, Atlas Search automatically distributes the index objects between the sub-indexes in an optimal way. When you run queries against a collection with sub-indexes, Atlas Search scatters the queries to all the sub-indexes and gathers the search results and metadata to sort, merge, and return the results.

We recommend partitioning your index when:

  • Your index objects reach 50% of the total limit.

  • The number of documents in your collection reaches two billion.

  • Your index is in the STALE state because Atlas Search stopped replication.

When you configure sub-indexes or modify the number of sub-indexes, Atlas Search triggers a rebuild of your index.

If you have more than one sub-index in your cluster, you can't remove all the search nodes and migrate to a deployment model where both the mongod and mongot processes run on the same node.

{
"name": "<index-name>",
"analyzer": "<analyzer-for-index>",
"searchAnalyzer": "<analyzer-for-query>",
"mappings": {
"dynamic": <boolean>,
"fields": { <field-definition> }
},
"numPartitions": <integer>,
...
}

The Atlas Search numPartitions option takes the following values:

  • 1 - to create a single index, with no additional sub-indexes. This is the default value.

  • 2 - to create up to two sub-indexes.

  • 4 - to create up to four sub-indexes.

  • 8 - to create up to eight sub-indexes.

  • 16 - to create up to sixteen sub-indexes.

  • 32 - to create up to thirty-two sub-indexes.

  • 64 - to create up to sixty-four sub-indexes.

The following index example uses the sample_mflix.movies collection to demonstrate how to configure up to 4 sub-indexes for the data in the collection. You can use the Visual Editor or the JSON Editor in the Atlas UI and other supported clients to create the index.


Use the Select your language drop-down menu to set the client of the example in this section.


curl --user "{PUBLIC-KEY}:{PRIVATE-KEY}" --digest \
--header "Accept: application/json" \
--header "Content-Type: application/json" \
--include \
--request POST "https://cloud.mongodb.com/api/atlas/v2/groups/{groupId}/clusters/{clusterName}/search/indexes" \
--data '
{
"collectionName": "movies",
"database": "sample_mflix",
"name": "partitioned_index",
"type": "search",
"definition": {
"analyzer": "lucene.standard",
"mappings": {
"dynamic": true,
},
"numPartitions": 4,
"searchAnalyzer": "lucene.standard"
}
}'
  1. Create a file named indexDef.json similar to the following:

    {
    "collectionName": "movies",
    "database": "sample_mflix",
    "definition": {
    "mappings": {
    "dynamic": true
    },
    },
    "name": "partitioned_index",
    "numPartitions": 4
    }
  2. Run the following command to create the index.

    atlas deployments search indexes create --file indexDef.json
1
  1. If it's not already displayed, select the organization that contains your desired project from the Organizations menu in the navigation bar.

  2. If it's not already displayed, select your desired project from the Projects menu in the navigation bar.

  3. If it's not already displayed, click Clusters in the sidebar.

    The Clusters page displays.

2

You can go the Atlas Search page from the sidebar, the Data Explorer, or your cluster details page.

  1. In the sidebar, click Atlas Search under the Services heading.

    Note

    If you have no clusters, click Create cluster to create one. To learn more, see Create a Cluster.

  2. From the Select data source dropdown, select your cluster and click Go to Atlas Search.

    The Atlas Search page displays.

  1. Click the Browse Collections button for your cluster.

  2. Expand the database and select the collection.

  3. Click the Search Indexes tab for the collection.

    The Atlas Search page displays.

  1. Click the cluster's name.

  2. Click the Atlas Search tab.

    The Atlas Search page displays.

3
4

Make the following selections on the page and then click Next.

Search Type

Select the Atlas Search index type.

Index Name and Data Source

Specify the following information:

  • Index Name: partitioned_index

  • Database and Collection:

    • sample_mflix database

    • movies collection

Configuration Method

For a guided experience, select Visual Editor.

To edit the raw index definition, select JSON Editor.
5
  1. Click Refine Your Index.

  2. Toggle Index Partitions to enable it.

  3. Select 4 from the Number of partitions dropdown and click Save Changes.

  1. Replace the default index definition with the following:

    {
    "mappings": {
    "dynamic": true
    },
    "numPartitions": 4
    }
  2. Click Next.

6

Atlas displays a Toast (brief, non-interactive notification) to let you know your index is building.

7

The newly created index appears on the Atlas Search tab. While the index is building, the Status field reads Build in Progress. When the index is finished building, the Status field reads Active.

Note

Larger collections take longer to index. You will receive an email notification when your index is finished building.

db.movies.createSearchIndex(
"search-index",
{ mappings: { dynamic: true }, "numPartitions": 4 }
)
using MongoDB.Bson;
using MongoDB.Driver;
// connect to your Atlas deployment
var uri = "<connection-string>";
var client = new MongoClient(uri);
var db = client.GetDatabase("sample_mflix");
var collection = db.GetCollection<BsonDocument>("movies");
// define your Atlas Search index
var index = new BsonDocument
{
{ "mappings", new BsonDocument
{
{ "dynamic", true }
}
},
{ "numPartitions", 4 }
};
var result = collection.SearchIndexes.CreateOne(index, "partitioned_index");
Console.WriteLine(result);
import com.mongodb.client.MongoClient;
import com.mongodb.client.MongoClients;
import com.mongodb.client.MongoCollection;
import com.mongodb.client.MongoDatabase;
import org.bson.Document;
public class CreateIndex {
public static void main(String[] args) {
// connect to your Atlas cluster
String uri = "<connection-string>";
try (MongoClient mongoClient = MongoClients.create(uri)) {
// set namespace
MongoDatabase database = mongoClient.getDatabase("sample_mflix");
MongoCollection<Document> collection = database.getCollection("movies");
Document index = new Document()
.append("mappings", new Document()
.append("dynamic", true)
)
.append("numPartitions", 4);
collection.createSearchIndex("partitioned_index", index);
}
}
}
import { MongoClient } from "mongodb";
// connect to your Atlas deployment
const uri = "<connection-string>";
const client = new MongoClient(uri);
async function run() {
try {
const database = client.db("sample_mflix");
const collection = database.collection("movies");
// define your Atlas Search index
const index = {
name: "partitioned_index",
definition: {
/* search index definition fields */
"mappings": {
"dynamic": true
},
"numPartitions": 4
}
}
// run the helper method
const result = await collection.createSearchIndex(index);
console.log(result);
} finally {
await client.close();
}
}
run().catch(console.dir);
from pymongo.mongo_client import MongoClient
from pymongo.operations import SearchIndexModel
def create_index():
# Connect to your Atlas deployment
uri = "<connectionString>"
client = MongoClient(uri)
# Access your database and collection
database = client["sample_mflix"]
collection = database["movies"]
# Create your index model, then create the search index
search_index_model = SearchIndexModel(
definition={
"mappings": {
"dynamic": True
},
"numPartitions": 4
},
name="partitioned_index",
)
result = collection.create_search_index(model=search_index_model)
print(result)