Describe Kafka Consumer Groups Response

Question

I have a Kafka 0.10.2.0 cluster and I'm using the Apache Kafka command line tools to debug why a particular Consumer Group is not receiving messages.

Using the $KAFKA_HOME/bin/kafka-consumer-groups.sh --bootstrap-server <kafka_brokers> --list I get a response that includes the Consumer Group ID I'm trying to debug.

Next, I use $KAFKA_HOME/bin/kafka-consumer-groups.sh --bootstrap-server <kafka_brokers> --describe –group <consumer_group_id> to retrieve details about topics, consumers, and offsets in relation to my misbehaving Consumer Group. I see a response I don't understand:

TOPIC                          PARTITION  CURRENT-OFFSET  LOG-END-OFFSET  LAG        CONSUMER-ID                                       HOST                                      CLIENT-ID
-                              -          -               -               -          external-ems-link-status-f3ecd2cb-ab84-4043-b4f5-aa0c7d7d6bf0/10.100.0.18                 external-ems-link-status
-                              -          -               -               -          external-controller-topo-notification-b87e5e28-3cd2-48cf-ac6b-ed9a04b3881c/10.100.0.18    external-controller-topo-notification
-                              -          -               -               -          external-interface-status-alrm-c0190022-d359-4cd6-a872-8c50852f4c53/10.100.0.18            external-interface-status-alrm
-                              -          -               -               -          external-ctlr-link-status-5b34eefa-6a06-45cd-ad13-b78fa6c499a7/10.100.0.18                 external-ctlr-link-status
-                              -          -               -               -          external-flow-ctrl-status-2712c302-d7de-4d50-8dc8-473c05205b4b/10.100.0.18                 external-flow-ctrl-status
-                              -          -               -               -          external-ctrl-status-e9a9d77a-6dc5-445d-9e38-0d9c5c3a4af4/10.100.0.18                       external-ctrl-status
-                              -          -               -               -          external-bw-util-notification-4f746863-0c10-4d4c-8785-93401e904777/10.100.0.18             external-bw-util-notification
link-status                    0          2602            2602            0          -                                                 -                              -

What does it mean when I see no topic, or partition information for a particular consumer/host, client ID?

I can confirm that messages are being produced on those topics when I create a console consumer using the --zookeeper flag.

Any help would be greatly appreciated.

vahid vahid · Accepted Answer · 2017-08-24T16:17:35

This tells you that there are a number of consumers in the group with no partitions assigned to them. If your link-status topic has only one partition, and the other consumers are subscribed to the same topic, this is the expected result (simply because each partition in the group can be consumed by one and only one consumer).

Describe Kafka Consumer Groups Response

1 Answers