How Kafka 0.9+ selects a broker to be the offset manager (or group coordinator) for a new consumer group?

2
votes

How does Kafka 0.9+ select a broker to be the offset manager or group coordinator for a new consumer group? Does it keep track of the number of groups managed by each broker and try to load balance, assigning new group to the broker who is managing the least number of groups? Does it take into account the number of consumers in each group so the number of connections to each broker is balanced too?

1
apache-kafkakafka-consumer-api

1 Answers

1
votes

Currently it follows the rule below:

  1. Determine the target partition for the group by calculating:

Math.abs(groupID.hashCode()) % numPartitionsOfGroupMetadataTopic

By default, numPartitionsOfGroupMetadataTopic is the __consumer_offsets topic with a default partition count of 50)

  1. Find out the leader broker for this partition and specify this broker as the coordinator for this group.