Querying DynamoDB GSI with a list of values

Question

I have a table of "team memberships" with primary partition key of "team_id" , a primary sort key of "email" and a GSI with "email" as partition key.

Given a list of emails I would like to find all the memberships for all the emails in the list in the most efficient way possible.

I know that for a single email I can do the following:

 Membership membership = new Membership();
 membership.setEmail(email.toLowerCase());

 DynamoDBQueryExpression<Membership> expression = new DynamoDBQueryExpression<Membership>()
                .withHashKeyValues(membership)
                .withIndexName(EMAIL_INDEX)
                .withConsistentRead(false);

 return mapper.query(Membership.class, expression);

which returns a list of memberships for the given email.

(So for my list of emails, I could do the above for each email in the list, but I would like to know if it is possible to do in one step).

I would like to be able to do something like:

List<Membership> memberships = allEmails.stream().map(email -> {
            Membership m = new Membership();
            m.setEmail(email.toLowerCase());
            return m;
        }).collect(Collectors.toList());

        DynamoDBQueryExpression<Membership> expression = new DynamoDBQueryExpression<Membership>()
                .withHashKeyValues(memberships) // <--- passing a list
                .withIndexName(EMAIL_INDEX)
                .withConsistentRead(false);

which clearly doesn't work.

What are the alternatives?

Whats your finding? I also got stuck with same problem. So far, could not find from AWS documentation also. Did you find any better solution for this? — user3681970

notionquest notionquest · Accepted Answer · 2018-07-27T16:06:05

Batch get item api can be used to get multiple partition keys from main table. However, it can't be used to get multiple items from GSI.

I appreciate that this answer may not resolve your problem. Simply, there is no other alternate solution.

Batch get item java

try {

            TableKeysAndAttributes forumTableKeysAndAttributes = new TableKeysAndAttributes(forumTableName);
            // Add a partition key
            forumTableKeysAndAttributes.addHashOnlyPrimaryKeys("Name", "Amazon S3", "Amazon DynamoDB");

            TableKeysAndAttributes threadTableKeysAndAttributes = new TableKeysAndAttributes(threadTableName);
            // Add a partition key and a sort key
            threadTableKeysAndAttributes.addHashAndRangePrimaryKeys("ForumName", "Subject", "Amazon DynamoDB",
                "DynamoDB Thread 1", "Amazon DynamoDB", "DynamoDB Thread 2", "Amazon S3", "S3 Thread 1");

            System.out.println("Making the request.");

            BatchGetItemOutcome outcome = dynamoDB.batchGetItem(forumTableKeysAndAttributes,
                threadTableKeysAndAttributes);

            Map<String, KeysAndAttributes> unprocessed = null;

            do {
                for (String tableName : outcome.getTableItems().keySet()) {
                    System.out.println("Items in table " + tableName);
                    List<Item> items = outcome.getTableItems().get(tableName);
                    for (Item item : items) {
                        System.out.println(item.toJSONPretty());
                    }
                }

                // Check for unprocessed keys which could happen if you exceed
                // provisioned
                // throughput or reach the limit on response size.
                unprocessed = outcome.getUnprocessedKeys();

                if (unprocessed.isEmpty()) {
                    System.out.println("No unprocessed keys found");
                }
                else {
                    System.out.println("Retrieving the unprocessed keys");
                    outcome = dynamoDB.batchGetItemUnprocessed(unprocessed);
                }

            } while (!unprocessed.isEmpty());

        }
        catch (Exception e) {
            System.err.println("Failed to retrieve items.");
            System.err.println(e.getMessage());
        }

Querying DynamoDB GSI with a list of values

1 Answers