KafkaConsumer position() vs committed()?

Question

I'm not understanding the difference between KafkaConsumer position() and committed() method from the javadoc.

position: public long position(TopicPartition partition) Get the offset of the next record that will be fetched (if a record with that offset exists).

committed: Get the last committed offset for the given partition (whether the commit happened by this process or another). This offset will be used as the position for the consumer in the event of a failure. This call will block to do a remote call to get the latest committed offsets from the server.

Does it mean if a consumer.poll() fetched lets say 50 messages from offset 101 to 150, and consumer has manual offset commit. And the consumer is still processing those 50 messages, so the last committed offset is 100. Now the committed() will return 100, but position will return 151 (coz. message 101 to 150 are already fetched)?

Answer 1

Yes that's correct.

Position is updated automatically when you poll() or seek() and correspond to the latest message offset the consumer has received.

Committed position refers to the latest offset the client has used in a call to commit (manual or auto).

For example, you could have a consumer with auto commit disable and also never manually call commit. In that case, the committed position would never change, whereas position would be updated as the client receives messages.