CosmosDB create unique document

Question

Currently, I'm using CosmosDB with SQL API in C#. Right now when it creates a new event in the Event collection, it needs to create a document in the EventData collection that has the same id, eventid and origin. Then it needs to set the count to 1. If the document already exists in Event collection, it throws an error that the eventid is unique and it needs to update the corresponding count in EventData. I set the partition key to the /origin and the unique key to /eventid. This is what I have come up with but not getting what I need with anything I try:

Data coming into Event:

{
    "id": randomly generated from cosmosdb,
    "eventid": "1234",
    "origin": "5.6.7.8"
}

What it needs to make in EventData:

{
    "id": same as the id from the Event collection,
    "eventid": "1234",
    "origin": "5.6.7.8",
    "count": 1
}

Then when an event comes in and has the event id "1234" it will find that it is already in the Event collection and increment the corresponding document in EventData:

{
    "id": same as the id from the Event collection,
    "eventid": "1234",
    "origin": "5.6.7.8",
    "count": 2
}

Azure Function:

public static async Task CreateEventAndUpdateCountAsync(string database, string eventCollection, string eventDataCollection, string input){
    MyClass myClass = Newtonsoft.Json.JsonConvert.DeserializeObject<MyClass>(input);
    try
    {
        //try to create a document, if not catch error
        //create document in Event collection, and if no error create a document with the count of 1

        await client.CreateDocumentAsync(UriFactory.CreateDocumentCollectionUri(database, eventCollection), myClass, new RequestOptions { PartitionKey = new PartitionKey(myClass.origin) });
        Count count = new Count();
        count.eventid = myClass.eventid;
        count.origin = myClass.origin;
        await client.CreateDocumentAsync(UriFactory.CreateDocumentCollectionUri(database, eventDataCollection), count, new RequestOptions { PartitionKey = new PartitionKey(myClass.origin) });
    }
    catch (DocumentClientException de)
    {
        if (de.StatusCode == HttpStatusCode.Conflict)
        {
            // had it trying to read the document and do a ReplaceDocumentAsync
            // need to find the document in the EventData Collection with the same eventid as myClass and increase count by 1               
        }


        else
        {
            log.Info(de.ToString());
            throw;
        }        

    }
}

public class myClass {
    public string eventid { get; set; }
    public string origin { get; set; }
}

public class Count {
    public string eventid { get; set; }
    public string origin { get; set; }
    public int count { get; set; } = 1;
}

Thanks

this sounds too relational for cosmos db. question is, is there a real reason why you need to have Events and EventData in separate collections? — dee zg
High volume of events being created and destroyed alongside the count being updated frequently (milliseconds) to where it would cause issues and possible dupes. — Jeff
1. you cannot do transactional operations across collections so you've already given up consistency (dupes) there. 2. your documents are almost identical across collections so i don't see benefit of multiple collections. Why wouldn't you put both documents into same collection, give them property type= Event | EventData & same partition key. then leave EventData to have whatever id it needs and give it relational field like eventId. Then have stored procedure (or trigger) that would insert into EventData based on what it finds Event. — dee zg
Important is, in that setup you would keep your docs within transactional boundaries (partition key). Other than that, you could consider using Cosmos DB Change Feed + Azure Functions. — dee zg
So should I add and increment the count to the Event collection? Or do that within a stored procedure and still put it in EventData? — Jeff

Nick Chapsas Nick Chapsas · Accepted Answer · 2018-09-20T14:53:33

The CosmosDB SDK supports the Upsert method. This method will create if something is missing and update if it exists.

You can change your second CreateDocumentAsync call to this:

await client.UpsertDocumentAsync(UriFactory.CreateDocumentCollectionUri(database, eventDataCollection), count, new RequestOptions { PartitionKey = new PartitionKey(myClass.origin) });

CosmosDB create unique document

1 Answers