Is there a limit to the number of documents one can bulk insert with PyMongo? And I don't mean the 16mb limit of document size for MongoDB, but the actual size of the list of documents I wish to insert in bulk through Python.
4
votes
1 Answers
10
votes
There is no limit on the number of documents for bulk insert via pymongo. According to the docs, you can provide an iterable to the collection.insert, and it will
insert each document in the iterable, sending only a single command to the server
Key point here is that pymongo will try to do your insert by sending one single message to the mongodb server.
Mongodb itself has a message size limit (maxMessageSizeBytes), that is equal to 48000000 bytes (maxBsonObjectSize * 3).
So, pymongo client driver should be responsible for splitting your large message into smaller messages to fit into mongodb max size limit. But, actually it's not yet implemented. See:
For now, you have to handle this situation by yourself.
Hope that helps.
