Kafka Offsets information not available

0
votes

When I login to my Kafka Manager(version 0.8.2.0), I can see the latest offset information for each topic and partition. I want to be able to get latest offsets programmatically. So I followed the example on https://cwiki.apache.org/confluence/display/KAFKA/Committing+and+fetching+consumer+offsets+in+Kafka but I do not get anything in response.

Also then I read on https://cwiki.apache.org/confluence/display/KAFKA/Kafka+data+structures+in+Zookeeper that offset information should be saved in Zookeeper under /consumers/[groupId]/offsets/[topic]/[partitionId] -> long (offset)

So I checked my Zookeeper (listed in Kafka Manager, under Cluster Information) and I do not see anything under /consumer (basically an empty document).

Can someone please help me to find this information? Is there something wrong with my Kafka configuration that latest offsets are not stored under /consumer? But then, where is the Kafka Manager getting that information from? Some other Zookeeper?

1
apache-kafkaoffset

1 Answers

0
votes

The information related to the offsets it's handled by kafka now, so the information it's stored in a private (not deletable) topic call "__consumer_offsets-" you can find where this topic by checking the log.dirs property in "config/server.properties"