Hi @Rami_Avital,
Recovering from data file corruption is unfortunately challenging and often unsuccessful.
However, I noticed you are running a very old version of MongoDB (3.0.12 was released in May, 2016).
The repair functionality for WiredTiger has been improved in more recent server versions (4.0.3+) per SERVER-19815.
This isn’t guaranteed magic, but I would:
-
download the latest version of MongoDB 4.0 (currently 4.0.23)
-
try running
mongod --repair(4.0.23 binary) against a copy of your data files following the procedure to Recover a Standalone after an Unexpected Shutdown.
If the repair procedure results in a usable deployment, I expect you will either have to remain on MongoDB 4.0 or mongodump and mongorestore into your 3.0 deployment.
Also to be clear: the --repair option is a last resort for data recovery and is definitely not a substitute for proper backups. The repair process will salvage data structures that can be read and skip those that cannot. This does not guarantee you can recover all of your data, and it is likely that some data will be missing depending on the nature of the data file corruption. However, if you don’t have any recent backups this has a chance of getting your deployment back online.
If you have at least one known good backup (even if not recent), I would compare data that is expected to be present in both your backup and the repaired database.
If the newer --repair process fails, this unfortunately is the stage at which you may need to accept that you have to return to your last good backup.
A final approach would be to try heroic efforts to salvage raw data. This is rarely successful and can involve a lot of deep dive effort. @alexbevi’s article on Recovering a WiredTiger collection from a corrupt MongoDB installation may be a useful guide, but note that this article predated the enhanced repair process I suggested above.
Regards,
Stennie