I’ve been performing some performance tests with large transactions and I found that MongoDB is killing transactions when dirty cache reaches 10% of the max cache size.
I am consistently seeing this behavior across machines of different sizes and with different load profiles.
For example, on a machine with 32GB memory (approximately 16GB max WiredTiger cache size), MongoDB starts killing transactions when “tracked dirty bytes” hits approximately 1.6GB. If a single transaction crosses that limit, it is killed. If two concurrent transactions hit that limit together, one of them is killed.
It surprises me that this happens when dirty cache is at 10%, because all information that I find on the internet (including MongoDB documentation) points towards a limit of “dirty cache = 20% of max cache” instead of “dirty cache = 10% of max cache”.
For example, the documentation for the transactionTooLargeForCacheThreshold parameter says the following:
The dirty cache is limited to 20% of the total cache size. When
transactionTooLargeForCacheThresholdis set to0.75, the server only retries transactions that use less than 15% (0.75 * 20%) of the total storage engine cache.
Based on the behavior I am observing, a transaction could never even hit 15% of total storage engine cache because Mongo already kills the transaction when it hits 10% of total storage engine cache.
What am I missing? Is the behavior that I am seeing the expected behavior?
FYI: I am on MongoDB 6.0.x, but we’ve been through several version upgrades over the years.