AWS - Identify data read by Lambda from DynamoDB Streams

Question

I have a DynamoDB table that has TTL and DynamoDB Streams enabled/configured.

I want to implement a lambda function that will read the TTL deleted data from DynamoDB Streams and forward it to maybe Kinesis Firehose or S3 (this I need to decide which one is better considering cost).

Is there any flag/property that will help to identify that the TTL deleted record is already being read/processed by some lambda function? For instance, I have 10 records in DynamoDB Stream, a lambda function has read 5 records and did some processing on it (forwarding it to Kinesis Firehose or S3). So, I want to identify those 5 records which were processed.

Answer 1

you can use the userIdentity field in the stream record to identify which record came from DynamoDB's TTL deletion.

See the details here: https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/time-to-live-ttl-streams.html

Answer 2

Sadly there is no such "flag/property" which indicates that a given DynamoDb record is being processed already or not yet by a lambda function. You would need to design a custom solution for that.

One would would be to have other DynamoDb table, just to keep track of a current status of the deleted items. For example, when your lambda accepts a record it writes it in the table. Any other function or interested processes would have to check if the record exists or not in the table. The presence of the record would indicate that the item deleted is still being process.