Many tiny documents in CosmosDB

6
votes

I have many (order of 100s) pieces of data that I want to associate with a document in CosmosDB. Each piece of data is small (order of 100s of bytes).

My first solution was to store the data as an array inside the document. This works okay, but in order to append a new item to the array I need to read the document from CosmosDB, add the element, then replace the document back into CosmosDB.

Instead of doing this I would like to store each piece of data as its own document in the same partition. What are the drawbacks of having many tiny documents vs the one aggregated document?

3
azure-cosmosdb

3 Answers

2
votes

What are the drawbacks of having many tiny documents vs the one aggregated document?

I would like to say that i suggest you storing each piece of data,instead of one aggregated document.

Reason1:As you mentioned in your question,if you want to add the element into the document,you need to read the document from CosmosDB, then replace the document because the partial update is not supported by cosmos db so far.(Please refer to this feedback and follow it if you need:https://feedback.azure.com/forums/263030-azure-cosmos-db/suggestions/6693091-be-able-to-do-partial-updates-on-document) That's a huge and tedious work.

Reason2:If you store pieces of data,you can query them flat. (select * from c)

For one single array document,you need to use join to access the nested properties.(select a.array from c join array in c.array)

Reason3:If you store pieces of data,you could manage them into different partitions.Even though you don't need it now,why not keep the feature for the future.

Reason4:As to cost,it all depends the RUs and storage and requests to cosmos db will consume RUs. If you store pieces of data,you just need to access the specific document as you want which is more economical i think.

1
votes

Depends on your use case.

  1. For frequent add operations, you are first reading and updating the document back (2 operations) which will incur you more cost than creating a new document (1 operation).

  2. However, if the documents are having some sort of relationships (like foreign keys in traditional SQL), getting data would require multiple queries if you go with approach #1 above (have more cost) otherwise, you'll get the complete data in a single query (low cost).

I'd recommend to go through this and this posts which will give you better insights on which approach you can choose.

0
votes

I'm facing this question right now and I want to let my contribution here. I'm having to store some statuses, this status is a metric that I get once per hour, then i have two options:

  1. Create a register per status -> 24 registers per day
  2. Create a register per day and add status inside it -> 1 register per day with 24 status inside an array

I chose the second one because:

  1. Both options will have the same amount of operations on database
  2. I'm using this data on Power BI and after doing some tests the data from second option had a small size after importation