Sharding Configuration

On this page

Overview

Declare Shard keys
Syntax
Specify Associated and Embedded Fields
Sharding Management Rake Tasks
Additional Information
API Documentation

Overview

Sharding is a way to distribute your data across multiple machines. MongoDB uses sharding to support deployments with large data sets and high throughput operations. In this guide, you can learn how to configure sharding in your Mongoid application.

Declare Shard keys

MongoDB uses shard keys to distribute a collection's documents across shards. A shard key is an indexed field, or multiple fields covered by a compound index, that determines the distribution of the collection's documents among the cluster's shards. In your Mongoid application, you can declare a shard key by using the shard_key macro when you create a model.

The following example creates a Person class with a shard key on the ssn field:

class Person
  include Mongoid::Document
  field :ssn
  shard_key ssn: 1
  # The collection must also have an index that starts with the shard key.
  index ssn: 1
end

Note

To shard a collection, the collection must have an index that starts with the shard key. The index can be on only the shard key, or it can be a compound index where the shard key is the prefix. You can use Mongoid's index-management functionality to create the index. To learn more about index management with Mongoid, see the Index Management guide.

If a model declares a shard key, Mongoid expects the sharded collection to use the declared key for sharding. When Mongoid reloads models, it provides the shard key along with the _id field to the find command to improve query performance. If the collection is not sharded with the specified shard key, queries might not return the expected results.

Syntax

You can declare shard keys by using either the full MongoDB syntax or by using a shorthand syntax.

The full syntax follows the format of the mongosh shardCollection() method, and allows you to specify the following types of shard keys:

Ranged keys
Hashed keys
Compound keys

The full syntax also allows you to specify collection and sharding options.

The following example creates each of the preceding type of shard key on the sson field:

# Create a ranged shard key
shard_key ssn: 1
# Create a compound shard key
shard_key ssn: 1, country: 1
# Create a hashed shard key
shard_key ssn: :hashed
# Specify a shard key option
shard_key {ssn: :hashed}, unique: true

The shorthand syntax allows you to declare a shard key by specifying only the field name. This syntax supports only ranged and compound shard keys, and does not allow you to specify collection or sharding options.

The following example creates a ranged and a compound shard key:

# Create a ranged shard key
shard_key :ssn
# Create a compound shard key
shard_key :ssn, :country

Specify Associated and Embedded Fields

You can specify a shard key on a belongs_to association in place of a field name. When doing so, Mongoid creates the shard key on the primary key of the associated collection.

The following example creates a shard key on the belongs_to association in a Person model. Because the associated country collection has a primary key called country_id, Mongoid shards by that field:

class Person
  include Mongoid::Document
  belongs_to :country
  # Shards by country_id
  shard_key country: 1
  # The collection must have an index that starts with the shard key
  index country: 1
end

You can specify a shard key on an embedded document by using dot notation to delimit the field names. The following example creates a shard key on the address.city field:

class Person
  include Mongoid::Document
  field :address
  shard_key "address.city"
end

Note

Because the period (.) character is used to delimit embedded fields, Mongoid does not support creating shard keys on fields with names that contain a period character.

Sharding Management Rake Tasks

You can shard collections in your database according to the shard keys defined in your Mongoid models by running the db:mongoid:shard_collections rake task. To ensure that the collections contain indexes that start with the shard key, you can first run the db:mongoid:create_indexes rake task.

Run the following rake commands to create the indexes and shard the collections based on your model's shard keys:

rake db:mongoid:create_indexes
rake db:mongoid:shard_collections

Index management and sharding rake tasks do not stop when they encounter an error with a particular model class. Instead, they log the error and continue processing the next model. To ensure the rake tasks did not encounter any errors, check the output of the Mongoid logger configured for your application.

Note

When performing schema-related operations in a sharded cluster, nodes might contain out-of-date local configuration-related cache data. To clear the caches, run the flushRouterConfig command on each mongos node.

Additional Information

To learn more about sharding with MongoDB, see the Sharding guide in the MongoDB Server manual.

API Documentation

To learn more about the shard_key macro discussed in this guide, see the shard_key API documentation.

Back

Persistence Targets

Logging