When multiple DynamoDB clients delete a specific item, (how) can each client determine if their DeleteItem call was meaningful?

Question

When a DynamoDB client calls DeleteItem on an item, it can use the ReturnItems request parameter to be informed of the item's pre-deletion contents in the response: https://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_DeleteItem.html#DDB-DeleteItem-request-ReturnValues

When 2 or more clients delete the same item, simultaneously, can they use this parameter to determine which of them truly deleted the item, and therefore which clients made a call which wasn't strictly necessary? In other words, using this parameter, will exactly-one client see the contents of the item in the DeleteItem response, where all other clients will see a response containing no item contents -- allowing them to correctly deduce that they didn’t materially delete the item?

I believe that it's well-known that, in normal operation, writes to DynamoDB require acknowledgement from 2 of the 3 storage nodes in a partition. I can't find any documentation, formal or otherwise, mentioning equivalent guarantees around deletes; though I can, of course, see why it would be the same.

Given that an update is mentioned as being treated as a delete then a put (https://en.wikipedia.org/wiki/Amazon_DynamoDB#Query_execution: "when the log propagator receives an Update operation, it issues both a Delete operation and a Put operation to all indices”), I would happily believe a delete is literally treated to the same underlying guarantees as writes.

If a delete does require acknowledgement from 2 of the 3 storage nodes then, outside significant system failure scenarios, I think I'm safe to use the ReturnItems request parameter to determine which client actually managed to delete an item. Is there anything which invalidates this assumption, in either normal operation or overloaded-and-slow-but-not-failed storage node operation? I’m explicitly not thinking about “DynamoDB is down in $REGION” mode, for the sake of argument ...

[My use case is for this to act as a coordination mechanism for which client should work on the specific item just deleted; hence I don't believe there are any issues around inconsistent-read-after-delete, etc.]

Maurice Maurice · Accepted Answer · 2021-01-05T14:00:37

My understanding of the underlying mechanism is similar to yours, it should work. If you're interested in how this works under the hood, I recommend watching this great re:invent talk by Rick Houlihan. He is a more authoritative voice on this.

Although it should work that way, I would still go a route that makes what's going on more explicit to the reader/developer. I've used something like this in the past. I'm making a conditional delete on the item with the condition being that the item exists. I'll get an exception that the item doesn't exist and can handle that.

import boto3
import botocore.exceptions

from boto3.dynamodb.conditions import Key

# Create table resource etc.

try:
    response = table_resource.delete_item(
        Key={
            "key": item["key"]
        },
        ConditionExpression=Key("key").eq(item["key"])
    )
except botocore.exceptions.ClientError as e:
    if e.response['Error']['Code'] == 'ConditionalCheckFailedException':
        # This is fine, it just means the object doesn't didn't exist.
        # It means either another writer was faster or there was no such item in the first place.
        pass
    else:
        LOGGER.error(e)
        raise e

Another solution (for java) is proposed here.

When multiple DynamoDB clients delete a specific item, (how) can each client determine if their DeleteItem call was meaningful?

1 Answers