What does DynamoDB scan return if item with Exclusive Start Key does not exist in the table?

Question

I'm trying to implement pagination for my API. I have a DynamoDB table with a simple primary key.

Since the ExclusiveStartKey in a DynamoDB scan() operation is nothing but the primary key of the last item fetched in the scan operation before, I was wondering what would DynamoDB return if I perform a scan() with an ExclusiveStartKey that does not exist in the table?

# Here response contains the same list of items for the same 
# primary key passed to the scan operation

response = table.scan(ExclusiveStartKey=NonExistentPrimaryKey)

I expected DynamoDB to return no items (correct me if this assumption of mine is what's wrong), i.e the scanning should resume from the ExclusiveStartKey, if it exists in the table. If not, it should return no items.

But what I do see happening is, the scan() still returns items. When I give the same non-existent primary key, it keeps returning me a list starting from the same item.

Does DynamoDB simply apply the hash function on the ExclusiveStartKey and from the result of this hash decide from which partition it has to start returning items or something?

# My theory as to what DynamoDB does in a paginated scan operation
partitionId = dynamodbHashFunction(NonExistentPrimaryKey)

return fetchItemsFromPartition(partitionId)

My end goal is that when an invalid ExclusiveStartKey is provided by the user (i.e a non-existent primary key), I want to return nothing or even better, return a message that the ExclusiveStartKey is invalid.

I would have expected exactly the explanation you gave, it hashes the key and based on that hash it asks the next dynamo nodes, wether or not the hash actually exists does not matter. — luk2302
@luk2302 - So it's upto us to do a pre-check if the key exists in the table or not, and ONLY if it exists, then perform a scan? — Sidharth Ramesh
I guess so, yes. I do not see what the point of that is, but you can do that. Do you actually want a scan operation? Maybe you want a query returning multiple items instead. — luk2302
@luk2302 - No, it's a scan operation indeed that I want. Basically, the API I'm developing lets the user scan the table, with the LastEvaluatedKey returned to the user on pagination. The user can then use this to invoke the API again, but this time with the LastEvaluatedKey as well. — Sidharth Ramesh
At that point there is no reason to make the check and instead let the scan behave exactly as it does right now. — luk2302

smac2020 smac2020 · Accepted Answer · 2021-02-05T15:22:02

Looks like you want to return items based on a value. If that value does not exist, then you want to have an empty result set. This is possible with the Java V2 DynamoDbTable object's scan method:

https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/enhanced/dynamodb/DynamoDbTable.html

For this solution, one way is to scan an AmazonDB table and return a result set based on the value of specific column (including the key). You can use an Expression object. This lets you set the value that you want to return in a result set.

For example, here is Java logic that returns all items where a date column is 2013-11-15. If there are no items that meet this condition, then no items are returned. There is no need for a pre-check, etc. You need to setup the ScanEnhancedRequest properly.

public static void scanIndex(DynamoDbClient ddb, String tableName, String indexName) {

        System.out.println("\n***********************************************************\n");
        System.out.print("Select items for "+tableName +" where createDate is 2013-11-15!");

        try {
            // Create a DynamoDbEnhancedClient and use the DynamoDbClient object.
            DynamoDbEnhancedClient enhancedClient = DynamoDbEnhancedClient.builder()
                    .dynamoDbClient(ddb)
                    .build();

            // Create a DynamoDbTable object based on Issues.
            DynamoDbTable<Issues> table = enhancedClient.table("Issues", TableSchema.fromBean(Issues.class));

            // Setup the scan based on the index.
            if (indexName == "CreateDateIndex") {
                System.out.println("Issues filed on 2013-11-15");

                AttributeValue attVal = AttributeValue.builder()
                        .s("2013-11-15")
                        .build();

                // Get only items in the Issues table for 2013-11-15.
                Map<String, AttributeValue> myMap = new HashMap<>();
                myMap.put(":val1", attVal);

                Map<String, String> myExMap = new HashMap<>();
                myExMap.put("#createDate", "createDate");

                Expression expression = Expression.builder()
                        .expressionValues(myMap)
                        .expressionNames(myExMap)
                        .expression("#createDate = :val1")
                        .build();

                ScanEnhancedRequest enhancedRequest = ScanEnhancedRequest.builder()
                        .filterExpression(expression)
                        .limit(15)
                        .build();

                // Get items in the Issues table.
                Iterator<Issues> results = table.scan(enhancedRequest).items().iterator();

                while (results.hasNext()) {
                    Issues issue = results.next();
                    System.out.println("The record description is " + issue.getDescription());
                    System.out.println("The record title is " + issue.getTitle());
                }
            }

        } catch (DynamoDbException e) {
            System.err.println(e.getMessage());
            System.exit(1);
        }
    }

What does DynamoDB scan return if item with Exclusive Start Key does not exist in the table?

1 Answers