Get largest value of primary key from DynamoDB table using boto3

4
votes

I am trying to get a largest value of a primary key (id in a particular case) using boto3.

Table structure: | id | name | |-----|-------| | 1 | Bob | | 4 | Alice | | 5 | Eve | where id is a primary key (hash key) and there is no sort key (range key).

As a result, I should get 5 (the id largest value).

I have looked thought docs and this answer but didn't find an answer.

ScanIndexForward attribute from table.scan() method requires sort key. table.query() method requires condition (such as Key={"name":"Bob"}).

The id value is not autoincrementing (table may have missing id values) that is why this solution doesn't help.

Question: Is it possible to get largest id from a table without sort key? (Of course, I do not want to scan all table and find it using python.)

Thanks for reading!

1
python-3.xamazon-web-servicesamazon-dynamodbboto3

1 Answers

2
votes

It's probably not what you want to hear- but you can't do this using table operations directly.

Dynamo gives you a lot of constraints in the name of making a scalable system. And most of these surround the options you have for querying; offering more of a key'd document store then a traditional database (further discussion about using scan with some, dubious, suggestions).

I suggest you take a look at the way you need to access your data and consider your schema design within the limits of partition/sort/indexes.

For situations like this (e.g. when you need to derive a property from the table), an architectural pattern you can use is:

DynamoDB
  -> DynamoDB Streams (shares Create/Update/Delete events)
  -> Lambda checks id against a stored value of 'current max id':
       - if create/update and higher, saves 'current max id' and 'last max id'
       - if delete and equal, replace 'current max id' with 'last max id'

In this case you could store the resulting 'view' of maximum id in another table, or a versioned s3 document. (n.b. there's still some limitations, you might need to do a scan if the two largest ids are deleted using this implementation).