Does zookeeper returns Kafka brokers dns? (which has the leader partition)

Question

I'm new to Kafka and I want to ask a question.

If there are 3 Kafka brokers(kafka1, kafka2, kafka3)(they are in same Kafka Cluster) and topic=test(replication=2) kafka1 has leader partition and kafka2 has follower partition.

If producer sends data to kafka3, then how do data stored in kafka1 and Kafka2?

I heard that, if producer sends data to kafka3, then the zookeeper finds the broker who has the leader partition and returns the broker's dns or IP address. And then, producer will resend to the broker with metadata.

Is it right? or if it's wrong, plz tell me how it works.

Thanks a lot!

Answer 1

Every kafka topic partition has its own leader. So if you have 2 partitions , kafka would assign leader for each partition. They might end up being same kafka nodes or they might be different.

When producer connects to kafka cluster, it gets to know about the the partition leaders. All writes must go through corresponding partition leader, which is responsible to keep track of in-sync replicas.

All consumers only talk to corresponding partition leaders to get data.

If partition leader goes down , one of the replicas become leader and all producers and consumers are notified about this change