1
votes

I am writing a kafka consumer using poll mechanism in which I am getting 100 messages per poll. After consuming messages, I commit an offset manually one by one. While committing an offset, sometimes I get an error for one of the message out off 100 messages. Rest offsets commit successfully.

Offsets
1,2,3,4,5,6,7,....100

Let's say offset 5 commit fails but excluding offset 5 all offsets committed successfully.

So, in this case, what happens to the commit failed offset? Since offset moves sequentially, will I get the commit failed offset in next poll?

My primary investigation shows it reads the failed offset in next poll. Need expertise view on this.

1

1 Answers

1
votes

It is not required to commit each offset individually. And if you commit offset X it implies that all smaller offset are committed, too.