Pymongo bulk_write upsert=True not working to insert a new document

Question

Python 3.6.7
pymongo==3.10.1

Every time that a start my application, I need to maintain a given collection. We can change previous loaded data as well as insert new ones.

I have the following code :

from pymongo import UpdateOne

COLLECTION_NAME = 'profile'

data = [
    UpdateOne(
        { "type": "user"},
        { "$set": { "order": 1 , "active": True, "name.pt_br": "usuário"} },
        upsert = True
    ),
    UpdateOne(
        { "type": "consultant"},
        { "$set": { "order": 2 , "active": True, "name.pt_br": "consultor"} },
        upsert = True
    ),
    UpdateOne(
        { "type": "business"},
        { "$set": { "order": 3 , "active": True, "name.pt_br": "gestor"} },
        upsert = True
    )
]

def load(db):

    db[COLLECTION_NAME].bulk_write(data)
    return True

The documents already exist inside the collection. When the load() method is called I'm able to update any attribute.

But if I need to include a new document (type: "x") appending it to the data[], such as:

data = [
    UpdateOne(
        { "type": "user"},
        { "$set": { "order": 1 , "active": True, "name.pt_br": "usuário"} },
        upsert = True
    ),
    UpdateOne(
        { "type": "consultant"},
        { "$set": { "order": 2 , "active": True, "name.pt_br": "consultor"} },
        upsert = True
    ),
    UpdateOne(
        { "type": "business"},
        { "$set": { "order": 3 , "active": True, "name.pt_br": "gestor"} },
        upsert = True
    ),
    UpdateOne(
        { "type": "x"},
        { "$set": { "order": 4 , "active": True, "name.pt_br": "x"} },
        upsert = True
    )
]

I get the following error:

  File "/data/dev/python/myapp/software/app/lib/storages/mongodb/dataLoad/profile.py", line 30, in load
    db[COLLECTION_NAME].bulk_write(data)
  File "/home/kleysonr/.virtualenvs/graphql/lib/python3.6/site-packages/pymongo/collection.py", line 502, in bulk_write
    bulk_api_result = blk.execute(write_concern, session)
  File "/home/kleysonr/.virtualenvs/graphql/lib/python3.6/site-packages/pymongo/bulk.py", line 511, in execute
    return self.execute_command(generator, write_concern, session)
  File "/home/kleysonr/.virtualenvs/graphql/lib/python3.6/site-packages/pymongo/bulk.py", line 349, in execute_command
    _raise_bulk_write_error(full_result)
  File "/home/kleysonr/.virtualenvs/graphql/lib/python3.6/site-packages/pymongo/bulk.py", line 140, in _raise_bulk_write_error
    raise BulkWriteError(full_result)
pymongo.errors.BulkWriteError: batch op errors occurred

How can I do an Insert/Update based on the data[] variable ?

Belly Buster Belly Buster · Accepted Answer · 2020-03-23T17:41:50

You should trap the error to find out what is going wrong.

from pymongo.errors import BulkWriteError

def load(db):

    try:
        db[COLLECTION_NAME].bulk_write(data)
        return True
    except BulkWriteError as e:
        write_errors = e.details.get('writeErrors')
        print(write_errors)

Pymongo bulk_write upsert=True not working to insert a new document

1 Answers