Sharding Configuration

On this page

Declaring Shard Keys

Sharding Management Rake Tasks

Mongoid can assist with setting up collection sharding in sharded environments.

Declaring Shard Keys

Shard keys can be declared on models using the shard_key macro:

class Person
  include Mongoid::Document
  field :ssn
  shard_key ssn: 1
  # The collection must also have an index that starts with the shard key.
  index ssn: 1
end

Note that in order to shard a collection, the collection must have an index that starts with the shard key. Mongoid provides index management functionality, which the examples here take advantage of.

Mongoid supports two syntaxes for declaring shard keys. The standard syntax follows the format of MongoDB shardCollection shell helper and allows specifying ranged and hashed shard keys, compound shard keys and collection sharding options:

shard_key ssn: 1
shard_key ssn: 1, country: 1
shard_key ssn: :hashed
shard_key {ssn: :hashed}, unique: true

The alternative is the shorthand syntax, in which only the keys are given. This syntax only supports ranged shard keys and does not allow options to be specified:

shard_key :ssn
shard_key :ssn, :country

shard_key macro can take the name of a belongs_to association in place of a field name, in which case Mongoid will use the foreign key configured in the association as the field name:

class Person
  include Mongoid::Document
  belongs_to :country
  # Shards by country_id.
  shard_key country: 1
  # The collection must also have an index that starts with the shard key.
  index country: 1
end

The shard key may also reference a field in an embedded document, by using the "." character to delimit the field names:

shard_key "location.x" => 1, "location.y" => 1
shard_key "location.x", "location.y"

Note

Because the "." character is used to delimit fields in embedded documents, Mongoid does not currently support shard key fields that themselves literally contain the "." character.

Note

If a model declares a shard key, Mongoid expects the respective collection to be sharded with the specified shard key. When reloading models, Mongoid will provide the shard key in addition to the id field value to the find command to improve query performance, especially on geographically distributed sharded clusters. If the collection is not sharded with the specified shard key, queries may produce incorrect results.

Sharding Management Rake Tasks

To shard collections in the database according to the shard keys defined in the models, run the db:mongoid:shard_collections Rake task. If necessary, run the db:mongoid:create_indexes Rake task prior to sharding collections:

rake db:mongoid:create_indexes
rake db:mongoid:shard_collections

Note

Like with index management rake tasks, sharding management rake tasks generally do not stop and fail when they encounter the problem with a particular model class. Instead they log the problem (to the configured Mongoid logger) at an appropriate level and continue with the next model. When Mongoid is used in a Rails application, this means the results of the rake task execution will generally be found in the per-environment log file like log/development.log.

Note

When performing schema-related operations in a sharded cluster, such as sharding collections as described in this document, or creating or dropping collections or databases, cluster nodes may end up with out of date local configuration-related cache data. Execute the flushRouterConfig command on each mongos node to clear these caches.

Back

Index Management

Working With Data