1
votes

I am often seeing issue in which kafka consumer gets current offset and lag set to unknown
Earlier consumer was having offset and lag

GROUP, TOPIC, PARTITION, CURRENT OFFSET, LOG END OFFSET, LAG, OWNER  
G.AB_KAFF, T.AB_KAFF, 0, 3, 6, 3, consumer-2
G.AB_KAFF, T.AB_KAFF, 1, 8, 11, 3, consumer-2

When I subscribed this consumer again after few days its offset and lag is set to unknown

GROUP, TOPIC, PARTITION, CURRENT OFFSET, LOG END OFFSET, LAG, OWNER  
G.AB_KAFF, T.AB_KAFF, 0, unknown, 7, unknown, consumer-2
G.AB_KAFF, T.AB_KAFF, 1, unknown, 11, unknown, consumer-2  

Is kafka removing previous offsets for that consumer since I am unsubscribing whole consumer group from topic ?

1

1 Answers

4
votes

Is kafka removing previous offsets for that consumer since I am unsubscribing whole consumer group from topic ?

yes, you are right. offsets.retention.minutes=1440 (24 hours) default is 24 hours.
Log retention window in minutes for offsets topic