Kafka consumer groups metrics java

0
votes

we want to get Kafka consumer group metrics (throttling and byte rate, for example).

We have done this already, using:

  1. JMX Mbean of the Kafka consumer Java application
  2. CLI utility:

bin/kafka-consumer-groups.sh --describe --group group_name --bootstrap-server localhost:port

.

Question: Can this be done programmatically using some Java libraries?

So far, we have tried 

kafka.admin.ConsumerGroupCommand

which is also used by the bin/kafka bin/kafka-consumer-groups.sh, but we got only the subset of expected values.

Thank you for your help.

1
apache-kafkakafka-consumer-api

1 Answers

0
votes

You can give a consumer a metric.reporters configuration that points to a class or classes that implement org.apache.kafka.common.metrics.MetricsReporter. From the Kafka source code:

A list of classes to use as metrics reporters. Implementing the org.apache.kafka.common.metrics.MetricsReporter interface allows plugging in classes that will be notified of new metric creation. The JmxReporter is always included to register JMX statistics.

There are also several projects that can export JMX metrics for various monitoring systems like Prometheus.