Kafka consumer groups deletion

2
votes

I have a large number of consumer groups created in Kafka. When I check the status most of them are with Empty status and without active consumers.

I have two questions

  1. Automatic deletion of consumer group is supported in Kafka and can we configure a time period to delete the empty consumer groups?
  2. Is there any specific Kafka version which supports automatic deletion of empty Consumer groups?
1
apache-kafkakafka-consumer-api
This might help. – mike
You can look at the retention policy for the offsets topic. If there's no active consumers, groups get removed from that topic – OneCricketeer
@OneCricketeer I tried to find this in the kafka documentation, but was not able to find any details regarding the deletion of groups. Can you please share a link where its mentioned "offsets.retention.minutes" and deleting of groups are mentioned. – Vineeth NG
offsets.retention.minutes is a broker config. The docs dont say "group is deleted"; it says its offsets expire (which cause it not to be in the topic anymore, and therefore cannot be found when queried) – OneCricketeer

1 Answers

3
votes

To delete empty consumer groups, you have 2 options:

  • Do nothing and wait offsets.retention.minutes. This settings defaults to 7 days. Groups without any members for this duration are automatically deleted. This mechanism exists since Kafka 0.8.2!

  • Use the delete consumer group API. This can be used via the Admin API's deleteConsumerGroups method or the kafka-consumer-groups.sh script. This API exists since Kafka 1.1.