When and how is a Dynamodb GSI having a Partition Key and Sort Key partitioned?

Question

1. When and how is a Dynamodb GSI having a Partition Key and Sort Key partitioned?
2. Is there a maximum size limit on GSI Partitions like table partitions?
3. If yes then what happens when a uni-cardinal GSI (i.e. GSI having the same partition key across all records) exceeds the storage limit?

Answer 1

1) See my answer here https://stackoverflow.com/a/51240423/4985580 for how tables are partitioned. A GSI is essentially just a new table, it is partitioned in the same way as your base table.

https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/HowItWorks.Partitions.html

Global secondary indexes in DynamoDB are also composed of partitions. The data in a GSI is stored separately from the data in its base table, but index partitions behave in much the same way as table partitions.

2) Yes, 10GB

3) That's an interesting question and I don't have the answer. Dynamo accesses the correct partition based on the partition key of the data, so if you fill more than one partition with a single partition key, it seems likely you would have a problem. That said you'd probably need at least 2.5 Million items with the same partition key for this to happen (10GB/4KB). Is this a possible scenario for you?

Answer 2

If yes then what happens when a uni-cardinal GSI (i.e. GSI having the same partition key across all records) exceeds the storage limit?

DynamoDB partition can only handle 1000 WCUs per sec. Also, DDB only allows 400 WCU per record. If your GSI exceeds WCU the writes to main table also gets throttled. So, if your records is around 400KB, a request rate of around 2.5 can bring your partition down.

Answer 3

1. Yes, it works in the same way as table partitioning.
2. Yes, 10GB per partition, also the same as for the table itself.
3. The behaviour is also exactly the same as for the main table. Dynamo will start spreading the data between partitions using Sorting Key, i.e nothing special happens.

Here is a good blog post that explains how partitioning works: https://aws.amazon.com/blogs/database/choosing-the-right-dynamodb-partition-key/

All items with the same partition key are stored together, and for composite partition keys, are ordered by the sort key value. DynamoDB splits partitions by sort key if the collection size grows bigger than 10 GB.