Kafka Consumer Group Metadata

Question

Kafka Version 2.4.1 Zookeeper 3.4.10

I am having trouble deleting a consumer group created by kafka streams application. Even though the application is down, but the group always report it as 'rebalancing' state. And kafka-consumer-groups doesn't allow me to delete it as it says 'The group is not empty'

Refer: Consumer group stuck in 'rebalancing' even though there are no consumers

Questions:

Where is consumer group information stored ? ( I tried zkcli.sh command and /consumers is empty)
Where does group coordinator store information related to consumer & its state
Is there other commands which can be used to query the group information ( or any hack)
Where are consumer group members stored ?

Giorgos Myrianthous Giorgos Myrianthous · Accepted Answer · 2020-05-01T23:09:07

In order to list all consumer groups

bin/kafka-consumer-groups.sh  \
  --bootstrap-server localhost:9092 \
  --list

Now to get information about a specific consumer group you can simply run

bin/kafka-consumer-groups.sh \
    --bootstrap-server localhost:9092 \
    --describe \
    --group my-consumer-group

Where are consumer group members stored ?

The metadata of consumer groups are stored in internal topic __consumer_offsets.

Kafka Consumer Group Metadata

1 Answers