They seem to be doing the same thing to me. Can anyone explain to me the difference?
1 Answers
42
votes
High level difference between the two:
Kinesis Streams allows you to produce and consume large volumes of data(logs, web data, etc), whereas DynamoDB Streams is a feature local to DynamoDB that allows you to track the granular changes to your DynamoDB table items.
More details:Amazon Kinesis Streams
Amazon Kinesis Streams is part of Big Data suite of services at AWS. From the developer documentation:
You can use Streams for rapid and continuous data intake and aggregation. The type of data used includes IT infrastructure log data, application logs, social media, market data feeds, and web clickstream data. The following are typical scenarios for using Streams:
Accelerated log and data feed intake and processing ...
Real-time metrics and reporting ...
Real-time data analytics ...
Complex stream processing ...
DynamoDB Streams
DynamoDB is the NoSQL option at AWS. It's delivery unit is a Table that stores Items. DynamoDB Streams is a DynamoDB feature you can turn on at Table level to record all changes to all Items (in the exact order they happened). This can then be streamed in real time as the changes happen, without perf impact. When you turn on the feature, you choose what is written to the stream:
- Keys only—only the key attributes of the modified item (including LSIs), but no way to add anything else).
- New image—the entire item, as it became after it was modified.
- Old image—the entire item, as it was just before it was modified.
- New and old images—both the new and the old images of the item
DynamoDB streams are commonly used for replication or table audits. More information can be found at the developer guide on DynamoDB streams.
The primary restrictions imposed by DynamoDB Streams are:
- only 1 or 2 consumers (you need to use fanout patterns beyond that)
- only 24h retention. While absolutely all changes are recorded, and in strict order, there is a hard limit on the retention - you need to grab them and do something, quick.
I can see where you might have gotten confused if you stumbled across this article first, which says that they are similar. They are different services which share similar API calls. The consumption experience is hence very similar.
