DynamoDB - Global Secondary Index on set items

24
votes

I have a dynamo table with the following attributes :

  • id (Number - primary key )
  • title (String)
  • created_at (Number - long)
  • tags (StringSet - contains a set of tags say android, ios, etc.,)

I want to be able to query by tags - get me all the items tagged android. How can I do that in DynamoDB? It appears that global secondary index can be built only on ScalarDataTypes (which is Number and String) and not on items inside a set.

If the approach I am taking is wrong, an alternative way for doing it either by creating different tables or changing the attributes is also fine.

3
amazon-dynamodb

3 Answers

22
votes

The key schema for the index. Every attribute in the index key schema must be a top-level attribute of type String, Number, or Binary. Nested attributes and multi-valued sets are not allowed. Other requirements for the key schema depend on the type of index: For a global secondary index, the hash attribute can be any scalar table attribute. A range attribute is optional, and it too can be any scalar table attribute. For a local secondary index, the hash attribute must be the same as the table's hash attribute, and the range attribute must be a non-key table attribute.

  • Amazon recommends creating a separate one-to-many table for these kind of problems. More info here : Use one to many tables
6
votes

You will need to create a separate table for this query. If you are interested in fetching all items based on a tag then I suggest keeping a table with a primary key:
hash: tag
range: id

This way you can use a very simple Query to fetch all items by tag.

1
votes

This is a really old post, sorry to revive it, but I'd take a look at "Single Table Design"

Basically, stop thinking about your data as structured data - embrace denormalization

id (Number - primary key ) title (String) created_at (Number - long) tags (StringSet - contains a set of tags say android, ios, etc.,)

Instead of a nosql table with a "header" of this:
id|title|created_at|tags

think of it like this:

pk|sk    |data....
id|id    |{title, created_at}
id|id+tag|{id, tag} <- create one record per tag

You can still return everything by querying for pk=id & sk begins with id and join the tags to the id records in your app logic

and you can use a GSI to project id|id+tag into tag|id which will still require you to write two queries against your data to get items of a given tag (get the ids then get the items), but you won't have to duplicate your data, you wont have to scan and you'll still be able to get your items in one query when your access pattern doesn't rely on tags.

FWIW I'd start by thinking about all of your access patterns, and from there think about how you can structure composite keys and/or GSIs

cheers