Should replication factor be one to provide at-most-once delivery semantics in Apache Kafka?

Question

As I understand, the producer must not retry any of failures to send, and the consumer must commit before performing some processings to provide at-most-once delivery semantics. But is replication factor associated to the delivery semantics too? the annotation in the sample project in reactor-kafka says like below:

A topic with replication factor one combined with a producer with acks=0 and no retries ensures that messages that could not be sent to Kafka on the first attempt are dropped

Should replication factor be one to provide at-most-once delivery semantics in Apache Kafka?

Answer 1

I think the concept of at-most-once is mainly relevant for the consumer and has nothing to do with replication factor. Instead, from the producer point of view, I think that phrase is correct because the producer does not wait for an ack and only one replica exists (no failover is possible).

If you disable auto-commit on your consumer and call commitSync to ack the message (commit the offset) as soon as you get it and before processing, then you are enforcing at-most-once delivery semantics. At this point, if the consumer is unable to handle the message due to some exception, the message is lost. Well, it is still stored in Kafka, but the consumer has moved past the message's offset and won't recover it without a manual offset reset.