How to create a Kinesis enhanced fan-out consumer for lambda

Question

I want to use lambda (python) for reading data from a Kinesis data stream and push it to another Kinesis data stream. I want to create an enhanced fan-out consumer for the source stream such that it can be used by the lambda function. Can anyone one please let me know how that can be done or point me to some clear documentation as I didn't have much luck trying to find a simple tutorial or way.

pllx pllx · Accepted Answer · 2021-02-17T22:09:56

You can create the consumer through AWS CLI e.g.

aws kinesis register-stream-consumer --stream-arn arn:aws:kinesis:us-west-2:123:stream/my_foo_stream --consumer-name FooConsumer

# Response
{
    "Consumer": {
        "ConsumerName": "FooConsumer",
        "ConsumerARN": "arn:aws:kinesis:us-west-2:123:stream/staging_transactions/consumer/FooConsumer:12345",
        "ConsumerStatus": "CREATING",
        "ConsumerCreationTimestamp": "2021-02-17T16:53:51-05:00"
    }
}

After this step, you should see your fan-out consumer under the "Enhanced Fan-out" tab on your Kinesis Stream, and also on your Lambda configs when you add a new Kinesis trigger under "Consumer".

https://docs.aws.amazon.com/cli/latest/reference/kinesis/register-stream-consumer.html

How to create a Kinesis enhanced fan-out consumer for lambda

1 Answers