Kafka's Replicas handling mechanism?

0
votes

I have two questions.

  1. I wonder how to sychronized leader partion and follower partions. If leader partition receive a message, then the leader broadcasting to follower partition on background communication? but It seemed kafka config file does not include these features(synchronization port info etc.)

  2. If assume the following architecture.

    • Two brokers - Two partition - Two replicas

    • Broker#1 - leader partition#1, follower partition#2

    • Broker#2 - leader partition#2, follower partition#1

Sending messages will be round-robin to these two brokers...

If message#1 go to Broker#1(partition#1) and Broker#1 was shut down,

then broker#2 open the follower partition#1 and broker#2 has active two leader partition (for delivering the message#1)?

1
apache-kafka

1 Answers

0
votes
  1. This is already handled by Kafka. You only need to define the replication factor for a topic. According to Kafka docs,

The partitions of the log are distributed over the servers in the Kafka cluster with each server handling data and requests for a share of the partitions. Each partition is replicated across a configurable number of servers for fault tolerance.

Each partition has one server which acts as the "leader" and zero or more servers which act as "followers". The leader handles all read and write requests for the partition while the followers passively replicate the leader. If the leader fails, one of the followers will automatically become the new leader. Each server acts as a leader for some of its partitions and a follower for others so load is well balanced within the cluster.

  1. Your question is not clear. I believe my answer to this question should shed some light with regards to kafka partitions, distribution of messages and fault tolerance.