Error 10334 "BSONObj size is invalid" when importing backup using mongorestore

Question

I have a backup of the MongoDB collection containing many small documents. The backup was created by mongodump , but when I try to import it using mongorestore , I get the error message:

AssertionException processing request, closing client connection: 10334 BSONObj size: 18039019 (0x11340EB) is invalid. The size must be between 0 and 16793600 (16 MB).

I am running MongoDB version 3.0.3 (from the trunk).

Answer 1

Using --batchSize=100 fixes this problem for me every time.

eg. mongorestore -d my-database --batchSize=100 ./database-dump-directory

Answer 2

Basically mongoDB accepts document size, should be less than 16 MB. If you intend to use a document larger than 16 MB, you can use gridfs. Each document has a memory capacity of 2 sizes. Your application should provide the size of the bson document that it generates. Or you can use a different data model rather than embedding all the data in one document.

Answer 3

mongorestore send paste commands in batch mode in the document {"applyOps", entries} . This document (AFAIK) is limited to 16 MB, like any other document.

According to sources , there are "pathological cases where the overhead of an array of many small operations can overwhelm the maximum team size." The oplogMaxCommandSize variable oplogMaxCommandSize used to help mongorestore not interrupt in such cases. It was raised to 16.5 million. At some point during the development of 3.0 .... It was too optimistic. It was reduced to 8M later ( JIRA TOOLS-754 ).

If you need, you can adjust this value yourself according to your needs. And then recompile the tools.