What if kafka offset manager is down

Question

A confluence doc shows how to fetching consumer offsets stored kafka, as follows: https://cwiki.apache.org/confluence/display/KAFKA/Committing+and+fetching+consumer+offsets+in+Kafka

It seems one broker is assigned as the offset manager, all the offset fetch and commit are done to this broker. but what if this broker is down?

Broker offsetManager = metadataResponse.coordinator();
// if the coordinator is different, from the above channel's host then reconnect
channel.disconnect();
channel = new BlockingChannel(offsetManager.host(), offsetManager.port(),
    BlockingChannel.UseDefaultBufferSize(),
    BlockingChannel.UseDefaultBufferSize(),
    5000 /* read timeout in millis */);
channel.connect();

Sandhya Km Sandhya Km · Accepted Answer · 2016-08-30T06:15:31

By configuring: 1. offsets.topic.num.partitions : The number of partitions for offset to commit the topic. & 2. offsets.topic.replication.factor: replication factor for the offset topic" in server.properties file, we are going to have Offset manager with one broker acting as leader and rest as followers and hence it follows the same leader failure mechanism in kafka.

Hence, when the offset manager that handles commitment of offset is down, Broker Controller eventually elects one of the ISR as the next offset manager Leader.

What if kafka offset manager is down

1 Answers