Configure Index Partition
On this page
Important
The numPartitions
option is available as a Preview feature.
For indexing, Atlas Search counts each document as a single index object when it isn't nested inside another document. For embedded documents, Atlas Search counts each embedded document as additional index objects depending on the number of levels of nesting. Atlas Search stops replicating changes for indexes larger than 2,100,000,000 index objects.
If you deployed Atlas Search on separate search nodes, you can increase the
number of Atlas Search index objects by partitioning your index objects in to
sub-indexes. By default, Atlas Search supports one partition per shard. Each
partition supports up to 2 billion index objects. You can create up to
sixty-four (64
) sub-indexes by using the numPartitions
option.
When you configure partitions for your index, Atlas Search automatically distributes the index objects between the sub-indexes in an optimal way. When you run queries against a collection with sub-indexes, Atlas Search scatters the queries to all the sub-indexes and gathers the search results and metadata to sort, merge, and return the results.
We recommend partitioning your index when:
Your index objects reach 50% of the total limit.
The number of documents in your collection reaches two billion.
Your index is in the
STALE
state because Atlas Search stopped replication.
When you configure sub-indexes or modify the number of sub-indexes, Atlas Search triggers a rebuild of your index.
If you have more than one sub-index in your cluster, you can't
remove all the search nodes and migrate to a deployment model where both
the mongod
and mongot
processes run on the same node.
Syntax
{ "name": "<index-name>", "analyzer": "<analyzer-for-index>", "searchAnalyzer": "<analyzer-for-query>", "mappings": { "dynamic": <boolean>, "fields": { <field-definition> } }, "numPartitions": <integer>, ... }
Supported Values
The Atlas Search numPartitions
option takes the following values:
1
- to create a single index, with no additional sub-indexes. This is the default value.2
- to create up to two sub-indexes.4
- to create up to four sub-indexes.8
- to create up to eight sub-indexes.16
- to create up to sixteen sub-indexes.32
- to create up to thirty-two sub-indexes.64
- to create up to sixty-four sub-indexes.
Example
The following index example uses the sample_mflix.movies
collection
to demonstrate how to configure up to 4
sub-indexes for the data in
the collection. You can use the Visual Editor or the JSON Editor in
the Atlas UI and other supported clients to create the index.
➤ Use the Select your language drop-down menu to set the client of the example in this section.
curl --user "{PUBLIC-KEY}:{PRIVATE-KEY}" --digest \ --header "Accept: application/json" \ --header "Content-Type: application/json" \ --include \ --request POST "https://cloud.mongodb.com/api/atlas/v2/groups/{groupId}/clusters/{clusterName}/search/indexes" \ --data ' { "collectionName": "movies", "database": "sample_mflix", "name": "partitioned_index", "type": "search", "definition": { "analyzer": "lucene.standard", "mappings": { "dynamic": true, }, "numPartitions": 4, "searchAnalyzer": "lucene.standard" } }'
Create a file named
indexDef.json
similar to the following:{ "collectionName": "movies", "database": "sample_mflix", "definition": { "mappings": { "dynamic": true }, }, "name": "partitioned_index", "numPartitions": 4 } Run the following command to create the index.
atlas deployments search indexes create --file indexDef.json
In Atlas, go to the Clusters page for your project.
If it's not already displayed, select the organization that contains your desired project from the Organizations menu in the navigation bar.
If it's not already displayed, select your desired project from the Projects menu in the navigation bar.
If it's not already displayed, click Clusters in the sidebar.
The Clusters page displays.
Start your index configuration.
Make the following selections on the page and then click Next.
Search Type | Select the Atlas Search index type. |
Index Name and Data Source | Specify the following information:
|
Configuration Method | For a guided experience, select Visual Editor. To edit the raw index definition, select JSON Editor. |
Check the status.
The newly created index appears on the Atlas Search tab. While the index is building, the Status field reads Build in Progress. When the index is finished building, the Status field reads Active.
Note
Larger collections take longer to index. You will receive an email notification when your index is finished building.
db.movies.createSearchIndex( "search-index", { mappings: { dynamic: true }, "numPartitions": 4 } )
using MongoDB.Bson; using MongoDB.Driver; // connect to your Atlas deployment var uri = "<connection-string>"; var client = new MongoClient(uri); var db = client.GetDatabase("sample_mflix"); var collection = db.GetCollection<BsonDocument>("movies"); // define your Atlas Search index var index = new BsonDocument { { "mappings", new BsonDocument { { "dynamic", true } } }, { "numPartitions", 4 } }; var result = collection.SearchIndexes.CreateOne(index, "partitioned_index"); Console.WriteLine(result);
import com.mongodb.client.MongoClient; import com.mongodb.client.MongoClients; import com.mongodb.client.MongoCollection; import com.mongodb.client.MongoDatabase; import org.bson.Document; public class CreateIndex { public static void main(String[] args) { // connect to your Atlas cluster String uri = "<connection-string>"; try (MongoClient mongoClient = MongoClients.create(uri)) { // set namespace MongoDatabase database = mongoClient.getDatabase("sample_mflix"); MongoCollection<Document> collection = database.getCollection("movies"); Document index = new Document() .append("mappings", new Document() .append("dynamic", true) ) .append("numPartitions", 4); collection.createSearchIndex("partitioned_index", index); } } }
import { MongoClient } from "mongodb"; // connect to your Atlas deployment const uri = "<connection-string>"; const client = new MongoClient(uri); async function run() { try { const database = client.db("sample_mflix"); const collection = database.collection("movies"); // define your Atlas Search index const index = { name: "partitioned_index", definition: { /* search index definition fields */ "mappings": { "dynamic": true }, "numPartitions": 4 } } // run the helper method const result = await collection.createSearchIndex(index); console.log(result); } finally { await client.close(); } } run().catch(console.dir);
from pymongo.mongo_client import MongoClient from pymongo.operations import SearchIndexModel def create_index(): # Connect to your Atlas deployment uri = "<connectionString>" client = MongoClient(uri) # Access your database and collection database = client["sample_mflix"] collection = database["movies"] # Create your index model, then create the search index search_index_model = SearchIndexModel( definition={ "mappings": { "dynamic": True }, "numPartitions": 4 }, name="partitioned_index", ) result = collection.create_search_index(model=search_index_model) print(result)