kafka consumer AbstractCoordinator: Discovered coordinator Java client

9
votes

I have 3 brokers running with broker id s 0 1 and 2. The Consumer (Java Client) picks up broker 0 as group coordinator and starts to consume messages correctly. But when the broker 0 which is the group coordinator is down, the consumer does not do anything and stalls on the poll() method. The process resumes only when that broker 0 is up and running.

How to handle this scenario of group co-ordinator change in the Java client?

I get this error when group coordinator dies:

16/09/22 17:42:45 INFO internals.AbstractCoordinator: Discovered coordinator datascience1.sv2.trulia.com:9092 (id: 2147483647 rack: null) for group group2.
16/09/22 17:42:45 INFO internals.AbstractCoordinator: (Re-)joining group group2
16/09/22 17:42:45 INFO internals.AbstractCoordinator: Marking the coordinator datascience1.sv2.trulia.com:9092 (id: 2147483647 rack: null) dead for group group2
1
apache-kafkakafka-consumer-api
Did you get an answer to this? – manocha_ak
I am struggling with the same issue, the producer keeps posting, but the consumer is stalled until coordinator is back. – Pouriya Zarbafian
What kafka version are you using? is the log cleaner enabled? consumer offset topic is a compacted log, if log cleaner is disabled, this could lead to coordinator issues. – Lior Chaga
Does your topic has replication factor 3? Were all replicas synced with the leader? – MeetJoeBlack

1 Answers

0
votes

See this answer: https://stackoverflow.com/a/50954843/7321097 and https://stackoverflow.com/a/50595475/7321097

The isue is in the offsets.topic.replication.factor & replication.factor configs.