Kafka Consumer with enable.auto.commit = false still committing the offsets

Question

Kafka consumer has processed the messages 1, 2, 3, 4 and the enable.auto.commit is set to false.

But on restarting the consumer, it is not reprocessing the above messages again, from CLI I could see that the offset has been incremented and there is no lag (hence it is committing).

Can you please help on this, to understand how the consumer is still committing the offsets though the property enable.auto.commit is set to false.

Below are the consumer properties

allow.auto.create.topics = true
auto.commit.interval.ms = 0
auto.offset.reset = latest
bootstrap.servers = [localhost:9092]
enable.auto.commit = false
fetch.max.bytes = 52428800
fetch.max.wait.ms = 500
fetch.min.bytes = 1
group.id = EmployeeConsumer

Currently using the spring-kafka-2.5.0.RELEASE.jar as the dependency

Gary Russell Gary Russell · Accepted Answer · 2020-05-19T13:33:57

You need to show your Spring configuration.

enable.auto.commit=false tells the kafka-clients not to commit offsets, but Spring will commit offsets by default.

Set the listener container ackMode property to AckMode.MANUAL to disable the container commits.

Furthermore:

auto.offset.reset = latest means that a consumer that has never committed an offset will start consuming from the current end of the topic/partition so it won't get existing records.

Kafka Consumer with enable.auto.commit = false still committing the offsets

1 Answers