Text Indexes

Overview

To run legacy text search queries, you must have a text index on your collection. MongoDB provides text indexes to support text search queries on string content. text indexes can include any field whose value is a string or an array of string elements. A collection can only have one text search index, but that index can cover multiple fields.

Versions

To override the default version and specify a different version, include the option { "textIndexVersion": <version> } when creating the index.

Create Text Index

To create a text index, use the db.collection.createIndex() method. To index a field that contains a string or an array of string elements, include the field and specify the string literal "text" in the index document, as in the following example:

db.reviews.createIndex( { comments: "text" } )

You can index multiple fields for the text index. The following example creates a text index on the fields subject and comments:

db.reviews.createIndex(
   {
     subject: "text",
     comments: "text"
   }
 )

A compound index can include text index keys in combination with ascending/descending index keys. For more information, see Compound Index.

In order to drop a text index, use the index name. See Use the Index Name to Drop a text Index for more information.

db.collection.createIndex( { "$**": "text" } )

db.collection.createIndex( { a: 1, "$**": "text" } )

Case Insensitivity

The version 3 text index supports the common C, simple S, and for Turkish languages, the special T case foldings as specified in Unicode 8.0 Character Database Case Folding.

The case foldings expands the case insensitivity of the text index to include characters with diacritics, such as é and É, and characters from non-Latin alphabets, such as "И" and "и" in the Cyrillic alphabet.

Version 3 of the text index is also diacritic insensitive. As such, the index also does not distinguish between é, É, e, and E.

Previous versions of the text index are case insensitive for [A-z] only; i.e. case insensitive for non-diacritics Latin characters only . For all other characters, earlier versions of the text index treat them as distinct.

Diacritic Insensitivity

With version 3, text index is diacritic insensitive. That is, the index does not distinguish between characters that contain diacritical marks and their non-marked counterpart, such as é, ê, and e. More specifically, the text index strips the characters categorized as diacritics in Unicode 8.0 Character Database Prop List.

Version 3 of the text index is also case insensitive to characters with diacritics. As such, the index also does not distinguish between é, É, e, and E.

Previous versions of the text index treat characters with diacritics as distinct.

Tokenization Delimiters

For tokenization, version 3 text index uses the delimiters categorized under Dash, Hyphen, Pattern_Syntax, Quotation_Mark, Terminal_Punctuation, and White_Space in Unicode 8.0 Character Database Prop List.

For example, if given a string "Il a dit qu'il «était le meilleur joueur du monde»", the text index treats «, », and spaces as delimiters.

Previous versions of the index treat « as part of the term "«était" and » as part of the term "monde»".

Index Entries

text index tokenizes and stems the terms in the indexed fields for the index entries. text index stores one index entry for each unique stemmed term in each indexed field for each document in the collection. The index uses simple language-specific suffix stemming.

Supported Languages and Stop Words

MongoDB supports text search for various languages. text indexes drop language-specific stop words (e.g. in English, the, an, a, and, etc.) and use simple language-specific suffix stemming. For a list of the supported languages, see Text Search Languages.

If you specify a language value of "none", then the text index uses simple tokenization with no list of stop words and no stemming.

To specify a language for the text index, see Specify a Language for Text Index.

sparse Property

text indexes are always sparse and ignore the sparse option. If a document lacks a text index field (or the field is null or an empty array), MongoDB does not add an entry for the document to the text index. For inserts, MongoDB inserts the document but does not add to the text index.

For a compound index that includes a text index key along with keys of other types, only the text index field determines whether the index references a document. The other keys do not determine whether the index references the documents or not.

Restrictions

Storage Requirements and Performance Costs

text indexes have the following storage requirements and performance costs:

• text indexes can be large. They contain one index entry for each unique post-stemmed word in each indexed field for each document inserted.

• Building a text index is very similar to building a large multi-key index and will take longer than building a simple ordered (scalar) index on the same data.

• When building a large text index on an existing collection, ensure that you have a sufficiently high limit on open file descriptors. See the recommended settings.

• text indexes will impact insertion throughput because MongoDB must add an index entry for each unique post-stemmed word in each indexed field of each new source document.

• Additionally, text indexes do not store phrases or information about the proximity of words in the documents. As a result, phrase queries will run much more effectively when the entire collection fits in RAM.

Text Search Support

The text index supports $text query operations. For examples of text search, see the $text reference page. For examples of $text operations in aggregation pipelines, see Text Search in the Aggregation Pipeline.