DynamoDB | How to Query all records in sorted order without specifying a Partition Key

3
votes

I'm using AWS DynamoDB to store user session records for a web application.

Each record is of the following format: 

uuid  timestamp  type

where

  • uuid is a users id and partition key
  • timestamp is a unix timestamp and sort key
  • type is "connected" or "disconnected"

In our administrator dashboard I want to display a table with the latest XX sessions, with the newest session first.

Question: Using AWS DynamoDB, how can I query all sessions in sorted order and with a defined limit?

Specifying a partition key is required using a Query

A Scan does not return results in sorted order

All solutions I have seen seems a bit "hacky", and I suspect I have misunderstood something since this use case must be fairly common.

Im aware of this hacky solution:

  1. Define a variable to all rows with same value
  2. Create a secondary index and set the variable as partition key and timestamp as sortkey
  3. Query this secondary index (now all row has the same partition key)

I'm no expert in DynamoDB, but this solution appears to be a hack and in opposition to DynamoDB architecture.

1
amazon-web-servicesnosqlamazon-dynamodb

1 Answers

1
votes

You have a bunch of options to achieve this in dynamo, all of which will seem hacky because your dashboard use case is not something dynamoDB is optimised for.

Here are some options:

  1. Store another record with a partition key of "activesessions". This record contains an array of the partition keys for your active sessions. One dynamo record can store 400kb of data so you can store thousands of active session keys in this one record (and page it if you need more). Downside to this is you have to maintain this "activesessions" record as sessions are added and removed.

  2. Scan - you say the records are not ordered, however you can order them in your code after retrieval from dynamo. This is essentially what dynamo does for filter expressions:

A FilterExpression is applied after the items have already been read; the process of filtering does not consume any additional read capacity units.

Downside of scan - it does not scale and it uses a lot of read capacity. I'd only use this if your number of total sessions is guaranteed to be small (a few thousand maximum).

  1. Your suggested option would work, however you should still try to give your records a key as unique as possible. Rather than just storing them all with the same partition key, maybe make it more unique by using a format such as 'samevalue-todaysdate'. That way your queries will be able to find sessions for a particular day a lot faster than searching through every session.

  2. Don't use dynamodb - use AWS RDS: mySQL, SQL server etc.