Kafka Producer is not retring when Broker is Down

Question

I have setup up Kafka using version 0.9 with the basic configuration as 1 Broker 1 Topic and 1 Partition.

Below are Producer Configurations that I have added to enable the retry from Producer.

    props.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, "localhost:9092");
    props.put(ProducerConfig.RETRIES_CONFIG, 5);
    props.put(ProducerConfig.RECONNECT_BACKOFF_MS_CONFIG, 500);
    props.put(ProducerConfig.ACKS_CONFIG, "all");
    props.put(ProducerConfig.MAX_BLOCK_MS_CONFIG, 500);
    props.put(ProducerConfig.METADATA_MAX_AGE_CONFIG, 50);

I understand from the documents that

Setting a value greater than zero will cause the client to resend any record whose send fails with a potentially transient error. Note that this retry is no different than if the client resent the record upon receiving the error.

Both my Broker & Zookeeper are down and the retry operation is not working.

ERROR o.s.k.s.LoggingProducerListener - Exception thrown when sending a message to topic TestTopic1| org.apache.kafka.common.errors.TimeoutException: Failed to update metadata after 500 ms.

I need to know if I am missing anything here for the retry to work.

Artem Bilan Artem Bilan · Accepted Answer · 2017-02-08T14:36:15

Resend (retry) works only if you have connection to the Broker and something happened during sending a message.

So, if your Broker is dead, there is no any reason to send message at all - no connection. And that is an exception about.

Kafka Producer is not retring when Broker is Down

3 Answers