Consumer timeout during rebalance

0
votes

When a consumer drops from a group and a rebalance is triggered, I understand no messages are consumed -

But does an in-flight request for messages stay queued passed the max wait time?
Or does Kafka send any payload back during the rebalance?

UPDATE
For clarification, I'm referring specifically to the consumer polling process.
From my understanding, when one of the consumers drop from the consumer group, a rebalance of the partitions to consumers is performed.
During the rebalance, will an error be sent back to the consumer if it's already polled and waiting for max time to pass?
Or does Kafka wait the max time and send an empty payload?
Or does Kafka queue the request passed max wait time until the rebalance is complete?

Bottom line - I'm trying to explain periodic timeouts from consumers.
This may be in the docs, but I'm not sure where to find it.

1
apache-kafka
In flight requests are from producers, not consumer groups. No messages are sent to brokers from consumers, only poll/fetch requests. Can you clarify the question? – OneCricketeer

1 Answers

0
votes

Kafka producers doesn't directly send messages to their consumers, rather they send them to the brokers.

The inflight requests corresponds to the producer and not to the consumer.

Whether the consumer leaves a group and a rebalance is triggered or not is quite immaterial to the behaviour of the producer.

Producer messages are queued in the buffer, batched, optionally compressed and sent to the Kafka broker as per the configuration.

In-flight requests are the maximum number of unacknowledged requests the client will send on a single connection before blocking.

Note that when we say ack, it is acknowledgement by the broker and not by the consumer.

Does Kafka send any payload back during the rebalance?

Kafka broker doesn't notify of any rebalance to its producers.