Bulk Partial Upsert in Elasticseach with python

16
votes

I want to send n upsert partial requests to ES, is such a thing possible? So if the document doesn't exist, insert my partial doc. If it already exists, update it with the partial doc.

Using the bulk helpers, I've tried a ton of variations, but they all wipe out existing values in favour of the new ones.

data = [{
    "_index": 'my_index',
    "_type": 'my_type',
    "_id": 12345,
    "doc": {"newkey": 'newvalue'}
}]
helpers.bulk(es, data, index='my_index', doc_type='my_type')

or

data = [{
    "_index": 'my_index',
    "_type": 'my_type',
    "_id": 12345,
    "_source": {"newkey": 'newvalue'}
}]
helpers.bulk(es, data, index='my_index', doc_type='my_type')

Does not work either.

2
pythonelasticsearchupsert

2 Answers

17
votes

As J. Ku answered, he has given the right answer but the code provided is incomplete. So posting the complete answer.

data = [{
    "_op_type": 'update',
    "_index": 'my_index',
    "_type": 'my_type',
    "_id": 12345,
    "doc": {"newkey": 'newvalue'},
    "doc_as_upsert":True
}]

helpers.bulk(es, data, index='my_index', doc_type='my_type')
1
votes

I think you need to include the action as mentioned in the documentation and set upsert as true

data = [{
    "_op_type": 'update',
    "_index": 'my_index',
    "_type": 'my_type',
    "_id": 12345,
    "doc": {"newkey": 'newvalue'}
}]
helpers.bulk(es, data, index='my_index', doc_type='my_type')