Kafka consumer stuck in (Re-)joining group

8
votes

Whats the default behavior of kafka (version 0.10) consumer if it tries to rejoin the consumer group. I am using a single consumer for a consumer group but it seems like it got struck at rejoining. After each 10 min it print following line in consumer logs.

2016-08-11 13:54:53,803 INFO o.a.k.c.c.i.ConsumerCoordinator [pool-5-thread-1] ****Revoking previously assigned partitions**** [] for group image-consumer-group

2016-08-11 13:54:53,803 INFO o.a.k.c.c.i.AbstractCoordinator [pool-5-thread-1] (Re-)joining group image-consumer-group

2016-08-11 14:04:53,992 INFO o.a.k.c.c.i.AbstractCoordinator [pool-5-thread-1] Marking the coordinator dead for group image-consumer-group

2016-08-11 14:04:54,095 INFO o.a.k.c.c.i.AbstractCoordinator [pool-5-thread-1] Discovered coordinator for group image-consumer-group.

2016-08-11 14:04:54,096 INFO o.a.k.c.c.i.AbstractCoordinator [pool-5-thread-1] (Re-)joining group image-consumer-group

Restart consumer application is not helping.

1
apache-kafkakafka-consumer-api
I am facing the exactly same issue after I set session.timeout.ms to a very large value to allow for completion of a complex long running pipeline. Have you found any solution to this? – Deeps
Yes, 1.you have to call pause on consumer after fetching some number of records. 2.Then create a background thread which sent heartbeat to kafka repeatedly by calling poll(0) in a loop. Meanwhile your main consumer thread is processing. 3. After processing tell background thread to stop from main consumer thread 4. now you can call resume on topic partition. – Bharat Bhagat
@BharatBhagat. I am getting the same issue even after adding session.timeout it did not work. Can you please tell me how you have added the logic you mentioned in Kafka Stream. Thanks a lot! – Alchemist
@BharatBhagat If there is a genuine failure in consumer thread(network outage or some exception while processing the message), the consumer will not be able to stop the heartbeat thread. And broker will keep receiving the heartbeats and rebalance will never get initiated. – Suyash Soni
Sorry guys I am not working on that project anymore. So do not have much info. Can you try Kafka Dev email group or post another question. – Bharat Bhagat

1 Answers

2
votes

If you're gonna to have only one consumer instance in a group, then use the consumer with manual assignment strategy. (Simple Consumer).

Manual topic assignment does not use the consumer's group management functionality so heart beats are not required.