3
votes

I'm quite new to Kinesis Streams and have spent a bunch of time wading through documentation but haven't been able to determine if it's possible to have multiple "topics" (for lack of a better term) on a single shard. Originally I thought that this is what partition keys were for, however after reading this:

A partition key is used to group data by shard within a stream. The Streams service segregates the data records belonging to a stream into multiple shards, using the partition key associated with each data record to determine which shard a given data record belongs to.

...it sounds to me like the partitions (can) span multiple shards.

I currently don't require more than a single shard but I do need to be able to separate messages into different channels/topics within that shard- is this not possible with Kinesis?

-- EDIT --

Worth mentioning- am using both the KCL and the KPL in my application.

1

1 Answers

3
votes

With only a single shard you won't be able to separate your data into separate channels. To consume the records you'll have to call GetShardIterator first. GetShardIterator takes the stream name, shard id and an iterator type.

The returned shard iterator is your pointer into the stream. Kinesis will return records to you if you call GetRecords, passing in the shard iterator. The response will contain a batch of records including the data, a timestamp, a sequence number and the partition key.

Today there is no way to specify a topic or any other filter when calling GetRecords/GetShardIterator.