I'm new in Kafka and while reading about synchronous replication in Kafka here I was surprised by the following paragraph:
"For simplicity, reads are always served from the leader. Only messages up to the HW are exposed to the reader"
Does anybody know if this means that it is not possible to read from replicas of a partition in Kafka? The link is from February 2013, maybe things have changed since. Any explanation about about reads with synchronous replication will be appreciated.
Reading more about the topic here, this is correct and both writes and reads go always to the leader of a topic partition. This makes sense as only committed messages are ever given out to the consumer and it is the leader the only one that knows when a message is committed.
This said, it is important to distribute topic partitions and leaders among all brokers in order to scale in the number of consumers per broker. For example, if all topic partition leaders were in one broker, the capacity of this broker (CPU, disk, memory, and especially bandwidth) to respond to writers and reader would determine the maximum number of concurrent writers and readers.
