Does Mongo cache the whole index or just the parts that it needs to service a Mongo Query?

Working with Data
1

If I have an index that is sized at 1 GB and have the WiredTiger cache set to 100 GB. Then, I perform that very first query that uses the 1GB index, but the query only returns back one row. Will Mongo store the whole 1 GB index in cache or just the parts that it needed to service that query?

Thank you,

John

2

Hello, @John_Cantu

MongoDB’s WiredTiger cache will store only the parts of the 1 GB index that are needed to service the query, not the entire index.

I hope this will be help you
chris1058

3

Hello, @John_Cantu

When using the WiredTiger storage engine in MongoDB, the cache behavior is interesting. Here’s how it works:

Cache Allocation:
WiredTiger doesn’t reserve separate portions of the cache for reads and writes.
Instead, it allocates its cache to the entire mongod instance.
There’s no per-database or per-collection cache allocation.
Index Caching:
When you perform a query, WiredTiger prioritizes index caching.
So, even if your query only returns one row, it will still load the relevant index pages into the cache.
However, it won’t necessarily load the entire 1 GB index; it’ll focus on the parts needed for the query.
Heavy Write Workload:
If there’s a heavy write workload, it can affect performance.
But WiredTiger still tries to keep index pages in cache for efficient querying.

I hope this info is helpful to you.

Best Regard,
Gregory Chavez

4

@chris_wright and @Gregory_Chavez, Could you please provide documentation that confirms your assertion?

5

Hello, @steevej

Here’s the requested documentation that confirms the behavior of the WiredTiger storage engine in MongoDB:

WiredTiger Storage Engine - MongoDB Manual:
The MongoDB Manual provides detailed information about the WiredTiger storage engine.
It covers topics such as cache allocation, index caching, and heavy write workloads.
You can find more details in the WiredTiger Storage Engine section of the MongoDB Manual here: https://www.mongodb.com/docs/manual/core/wiredtiger/.

I hope this is helpful to you.

Best Regard,
mysedgwick walmart

6

Thanks for the documentation link.

Shame for the spam link.

7

Thanks for your responses Chris, Gregory and Steeve.

I didn’t think it would be the best thing for Mongo to cache the entire index, but I needed to ask. The reason I asked this question was based on the following recommendation:

Mongo recommends to “Ensure Indexes Fit in RAM”
https://www.mongodb.com/docs/manual/tutorial/ensure-indexes-fit-ram/#std-label-indexes-ensure-indexes-fit-ram

If only parts of the index are used, then I don’t think it is necessary to have enough RAM to hold the entire index just the index pages frequently used.

Also, I am glad to hear that Mongo will will prioritize keeping index pages cached.

John

1 Like