- Does Kafka consumer keep checking the health of the broker(Kafka Server) or vice versa?
- Let's say by anyhow, Consumers and brokers know each other's health so how exactly the consumer will read from the partition?
- Let's say I have 48 partitions for a topic and have two consumer groups for the topic, so how many threads will be consuming the data from all partitions?
0
votes
1 Answers
0
votes
- Consumers send out healthchecks so that the broker knows if the consumers are healthy. Brokers' health is controlled by
Controller
, which is a Kafka service that runs on every broker in a Kafka cluster, but only one can be active (elected) at any point in time. - See this video for detailed description. In a nutshell, first consumer in the group is a leader and decided on the assignments for the rest of the consumers in the same group. This data is send to the broker and distributed across consumers.
- Thread management is your own responsibility. Both
KafkaProducer
andKafkaConsumer
are single-threaded components.