mongo3.0 restore fails with error Failed: restore error: insertion error: EOF

4
votes

I'm trying to restore a database with 100 collections (800GB) using mongorestore which was taken as a backup from mongo 2.4.4. I used the following command:

mongorestore -u <username> -p <password> -d bp_prod079 
             /data/backup/2015-11-28/bp_prod079 --batchSize=100

I'm using batchSize =1000, 500, 100 to try to make it work. I'm getting an insertion error after 10-15 collections. It almost runs for almost an hour and half perfectly when I get this error.

Failed: restore error: bp_prod079.ANEpisodeBreakdown: error restoring from /data/backup/2015-11-28/bp_prod079/ANEpisodeBreakdown.bson: insertion error: EOF

2
mongodbmongorestore
what's the output of ls -l /data/backup/2015-11-28/bp_prod079/ANEpisodeBreakdown.bson? Is it an empty file by any chance? – Asya Kamsky
ls -l /data/backup/2015-11-28/bp_prod079/ANEpisodeBreakdown.bson -rw-r--r-- 1 root root 1413518186 Nov 28 07:21 /data/backup/2015-11-28/bp_prod079/ANEpisodeBreakdown.bson – Manny
It is not an empty file. I opened a ticket with MongoDB-JIRA as well. They say there is a bug in 3.0.7. They asked me to use --batchSize=1 as last resort. If that doesn't work then I have to wait for 3.0.8. – Manny
You can use 3.0.6 mongorestore on that backup - I assume you're refering to TOOLS-939 in jira? – Asya Kamsky
Yes. It is TOOLS-939. – Manny

2 Answers

13
votes

For me with a minor batchSize works. First I had --batchSize=50 and changed it to 10 --batchSize=10 and perfect.

2
votes

On MongoDB servers using the WiredTiger storage engine (default on MongoDB v3.2.x), this error can also be thrown when running out of memory, causing both mongorestore and mongod to crash, as MongoDB developer @mpobrien explains here.

The solution is to add more RAM to the server, or tune the storage.wiredTiger.engineConfig.cacheSizeGB setting as described here: https://groups.google.com/d/msg/mongodb-user/Ms2MXCajreg/OsW6ixcSCgAJ