The latest (version: Fishtown.SR2) spring-cloud-stream doesn't support kafka client version 2.1.0 or higher?

Question

I am using latest Greenwich.SR1 which include spring-cloud-stream (version: Fishtown.SR2) and locally starting kafka client version:2.2.0 using kafka_2.12-2.2.0.jar

I want to use the latest kafka client 2.1 or higher with spring-cloud-stream because it contains some important bug fixes. But when I run my spring app, its logs says

INFO 37812 --- [main] o.a.kafka.common.utils.AppInfoParser : Kafka version : 2.0.1

How can I use spring-cloud-stream with the latest kafka client? I want to use max.task.idle.ms from the latest StreamsConfig of kafka client but it seems the latest spring-cloud-stream kafka stream binder doesn't support kafka client 2.1.0 or higher version?

Answer 1

Spring versioning rules don't allow us to upgrade to a new version of the kafka clients in a point release. Since Fishtown uses Spring for Apache Kafka 2.2, it uses the 2.0.x kafka-clients.

The next version of Spring for Apache Kafka will use the 2.2.0 clients (or 2.3.0 if it's available by then), so the next version of the binder will be based on the newer clients.

The message channel binder works when the kafka-clients is overridden but, unfortunately, the streams binder does not due to some internal API changes.