Spring cloud stream - support for rebalancing consumers with Kinesis

0
votes

Does spring cloud stream support Kafka-like rebalancing for kinesis also? There has been a recent commit to the issue https://github.com/spring-projects/spring-integration-aws/issues/99

Thanks

1
spring-integrationspring-cloudspring-cloud-stream

1 Answers

1
votes

There is no exact, Kafka-like behavior in the AWS Kinesis at all. In Kafka the rebalancing is performed on the broker side and all the consumers are notified about their own assignments.

What we have currently in the KinesisMessageDrivenChannelAdapter is some kind of leader election between shard consumers. But solution is not full yet with the automatic rebalancing when a new consumer is joined the group.

There is something like auto-rebalance in the Kinesis Client Library and already have a KclMessageDrivenChannelAdapter, but it is not release ready yet from the Spring Integration AWS perspective.