KafkaConsumer assignment() returns empty

Question

I am using

<dependency>
    <groupId>org.apache.kafka</groupId>
    <artifactId>kafka-clients</artifactId>
    <version>2.0.1</version>
</dependency>

The following piece of codes returns the non-empty assigned partitions, but poll(0) deprecated.

val records = kafkaConsumer.poll(0) // <= deprecated
logInfo(s"Dummy call ${records.count()}")

val partitions = kafkaConsumer.assignment()
logInfo(s"partitions=${partitions}")

The following return empty partitions:

val records = kafkaConsumer.poll(Duration.ofMillis(0)) // <= not working
logInfo(s"Dummy call ${records.count()}")

val partitions = kafkaConsumer.assignment()
logInfo(s"partitions=${partitions}")

Why? Any ideas? Thanks

amethystic amethystic · Accepted Answer · 2019-04-26T01:21:30

The difference of these two calls is the way the metadata is fetched. The deprecated poll waits indefinitely until the metadata is retrieved successfully, whereas the other poll only tries once, often failing to connect to the coordinator within a very shot time interval(0 for your case), and returns with nothing useful. That's why you see an empty assignment after calling poll(Duration.ofMillis(0)) once.

KafkaConsumer assignment() returns empty

1 Answers