kafka remove connection from group

2
votes

I using kafka and spring boot application for consuming message after read some message from topic I will receive this error

[2020-07-07 17:05:32,265] INFO [GroupCoordinator 1001]: Member consumer-1-708fe639-905d-432a-b6ba-17bf58adcf03 in group biker-service-prod has left, removing it from the group (kafka.coordinator.group.GroupCoordinator) [2020-07-07 17:05:32,267] INFO [GroupCoordinator 1001]: Preparing to rebalance group biker-service-prod in state PreparingRebalance with old generation 460 (__consumer_offsets-7) (reason: removing member consumer-1-708fe639-905d-432a-b6ba-17bf58adcf03 on LeaveGroup) (kafka.coordinator.group.GroupCoordinator) [2020-07-07 17:05:32,267] INFO [GroupCoordinator 1001]: Group biker-service-prod with generation 461 is now empty (__consumer_offsets-7) (kafka.coordinator.group.GroupCoordinator)

this is my config in consumer side

spring.kafka.listener.poll-timeout=3000000

spring.kafka.consumer.heartbeat-interval=500

spring.kafka.consumer.fetch-max-wait=3000000

spring.kafka.consumer.auto-commit-interval=1000

how can i fix it?

1
apache-kafkaspring-kafka

1 Answers

1
votes

The broker has decided your consumer is dead, as it didn't call poll() within the needed interval.

Take a look for these consumer-side params:

  • max.poll.interval.ms
  • session.timeout.ms

These will decide when your consumer will be considered dead and throw the exception you are showing. Take care of the time you need processing a message, if it is bigger than the defined max.poll.interval.ms, you should:

  1. Decrease the time needed to process the msg and call poll() again.

or

  1. Increase the max.poll.interval.ms and session.timeout.ms intervals.