Update Existing Documents in Cosmos db using Azure Data Factory

Question

I have created a data factory to Update my existing Cosmos DB using data stored in Blob as csv file. The csv file contains two fields column "X" and "Y" and their values. I want to insert new key in my existing Cosmos Db as Y ,filter criteria is column X which is unique

I already created the pipeline ,and upsert the documents in cosmosdb. But in current Pipeline old data is getting removed and only column Y is getting inserted.

Data in current Document in cosmos db

  {
    "_id" : ObjectId("5dad5adbfa882146ea8e7a0e"),
    "x" : "UUID",
   "old_key" : true
  }

Data in csv file

   x,new_key
   UUID, "new_value"

Expected output

   {
     "_id" : ObjectId("5dad5adbfa882146ea8e7a0e"),
      "x" : "UUID",
     "old_key" : true,
    "new_key":"new_value"

    }

My output which i tried(old keys are removed)

   {
     "_id" : ObjectId("5dad5adbfa882146ea8e7a0e"),
     "x" : "UUID"
     "new_key":"new_value"

   }

Jay Gong Jay Gong · Accepted Answer · 2019-10-22T07:53:15

You want to implement incremental update or partial update for cosmos db.

{
    "_id" : ObjectId("5dad5adbfa882146ea8e7a0e"),
    "x" : "UUID",
   "old_key" : true
}

change to:

{
     "_id" : ObjectId("5dad5adbfa882146ea8e7a0e"),
      "x" : "UUID",
     "old_key" : true,
    "new_key":"new_value"
}

As i know, partial update is not supported in the cosmos db so far.Please see this user voice and this feature statement(i know this feature lack lasts so long already....). So, you have to contains all the properties in your new documents except "_id".

I did a simple test:

1.load a csv file into cosmos db:

_id,x,A
5daeb1eda34d640184a71d47,123,ValueA

2.load a new csv file into cosmos db:

_id,x,B
5daeb1eda34d640184a71d47,123,ValueBBB

3.print two outputs:

Update Existing Documents in Cosmos db using Azure Data Factory

1 Answers