Consuming from a replica

9
votes

Kafka replicates each partition of a topic up-to specified replication factor.

As far as I know, all write and read requests are routed to the leader of the partition. Is there any way to consume from the followers not from the leader?

Is the replication in Kafka only for fail-over?

1
apache-kafkakafka-consumer-api
There's a KIP for extending Kafka, such that consumers can read from the closest replica. cwiki.apache.org/confluence/display/KAFKA/… – user152468

1 Answers

15
votes

In Kafka 2.3 and older, you can only consume from the leader -- this is by design. Replication is for fault-tolerance only.

If the leader fails, one of the follower will be elected as a new leader.

Have a look at this blog post for more details: http://www.confluent.io/blog/hands-free-kafka-replication-a-lesson-in-operational-simplicity/

Update:

As of Kafka 2.4, fetching from follower replicas is possible. For details, check out KIP-392: https://cwiki.apache.org/confluence/display/KAFKA/KIP-392%3A+Allow+consumers+to+fetch+from+closest+replica