Clustered Collections

Overview

Starting in MongoDB 5.3, you can create a collection with a clustered index. Collections created with a clustered index are called clustered collections.

Benefits

Because clustered collections store documents ordered by the clustered index key value, clustered collections have the following benefits compared to non-clustered collections:

• Faster queries on clustered collections without needing a secondary index, such as queries with range scans and equality comparisons on the clustered index key.

• Clustered collections have a lower storage size, which improves performance for queries and bulk inserts.

• Clustered collections can eliminate the need for a secondary TTL (Time To Live) index.

  • A clustered index is also a TTL index if you specify the expireAfterSeconds field.

  • To be used as a TTL index, the _id field must be a supported date type. See TTL Indexes.

  • If you use a clustered index as a TTL index, it improves document delete performance and reduces the clustered collection storage size.

• Clustered collections have additional performance improvements for inserts, updates, deletes, and queries.

  • All collections have an _id index.

  • A non-clustered collection stores the _id index separately from the documents. This requires two writes for inserts, updates, and deletes, and two reads for queries.

  • A clustered collection stores the index and the documents together in _id value order. This requires one write for inserts, updates, and deletes, and one read for queries.

Behavior

Clustered collections store documents ordered by the clustered index key value.

You can only have one clustered index in a collection because the documents can be stored in only one order. Only collections with a clustered index store the data in sorted order.

You can have a clustered index and add secondary indexes to a clustered collection. Clustered indexes differ from secondary indexes:

• A clustered index can only be created when you create the collection.

• The clustered index keys are stored with the collection. The collection size returned by the collStats command includes the clustered index size.

Starting in MongoDB 6.2, if a usable clustered index exists, the MongoDB query planner evaluates the clustered index against secondary indexes in the query planning process. When a query uses a clustered index, MongoDB performs a bounded collection scan.

Prior to MongoDB 6.2, if a secondary index existed on a clustered collection and the secondary index was usable by your query, the query planner selected the secondary index instead of the clustered index by default. In MongoDB 6.1 and prior, to use the clustered index, you must provide a hint because the query optimizer does not automatically select the clustered index.

Limitations

Clustered collection limitations:

• You cannot transform a non-clustered collection to a clustered collection, or the reverse. Instead, you can:

  • Read documents from one collection and write them to another collection using an aggregation pipeline with an $out stage or a $merge stage.

  • Export collection data with mongodump and import the data into another collection with mongorestore.

• The clustered index key must be on the _id field.

• You cannot hide a clustered index. See Hidden indexes.

• If there are secondary indexes for the clustered collection, the collection has a larger storage size. This is because secondary indexes on a clustered collection with large clustered index keys may have a larger storage size than secondary indexes on a non-clustered collection.

• Clustered collections may not be capped collections.

Set Your Own Clustered Index Key Values

By default, the clustered index key values are the unique document object identifiers.

You can set your own clustered index key values. Your key:

• Must contain unique values.

• Must be immutable.

• Should contain sequentially increasing values. This is not a requirement but improves insert performance.

• Should be as small in size as possible.

  • A clustered index supports keys up to 8 MB in size, but a much smaller clustered index key is best.

  • A large clustered index key causes the clustered collection to increase in size and secondary indexes are also larger. This reduces the performance and storage benefits of the clustered collection.

  • Secondary indexes on clustered collections with large clustered index keys may use more space compared to secondary indexes on non-clustered collections.

Examples

This section shows clustered collection examples.

db.runCommand( {
   create: "products",
   clusteredIndex: { "key": { _id: 1 }, "unique": true, "name": "products clustered key" }
} )

db.createCollection(
   "stocks",
   { clusteredIndex: { "key": { _id: 1 }, "unique": true, "name": "stocks clustered key" } }
)

db.createCollection(
   "orders",
   { clusteredIndex: { "key": { _id: 1 }, "unique": true, "name": "orders clustered key" } }
)

db.orders.insertMany( [
   { _id: ISODate( "2022-03-18T12:45:20Z" ), "quantity": 50, "totalOrderPrice": 500 },
   { _id: ISODate( "2022-03-18T12:47:00Z" ), "quantity": 5, "totalOrderPrice": 50 },
   { _id: ISODate( "2022-03-18T12:50:00Z" ), "quantity": 1, "totalOrderPrice": 10 }
] )

db.orders.find( { _id: { $gt: ISODate( "2022-03-18T12:47:00.000Z" ) } } )

[
   {
      _id: ISODate( "2022-03-18T12:50:00.000Z" ),
      quantity: 1,
      totalOrderPrice: 10
   }
]

db.runCommand( { listCollections: 1 } )

...
name: 'orders',
type: 'collection',
options: {
   clusteredIndex: {
      v: 2,
      key: { _id: 1 },
      name: 'orders clustered key',
      unique: true
   }
},
...