How does kafka handle ordering of different batch?

Question

In the documention of kafka about retries, it says:

Allowing retries without setting max.in.flight.requests.per.connection to 1 will potentially change the ordering of records because if two batches are sent to a single partition, and the first fails and is retried but the second succeeds, then the records in the second batch may appear first.

According the paragraph,if two batches are send to the same partition, kafka can commit the second batch while fail the first batch.
But this seems contradic to kafka's guarantee about ordering in same partition. Because usually if one batch failed, all subsequent batches should fail,otherwise how could it guarantee ordering? Besides,how does kafka guarantee that the order of batches send by producer is the same as the order reveiced by broker.

So my question is: how does kafka guarantee the ordering of different batch (of same partition) or it just does not guarantee.

Answer 1

how does kafka guarantee the ordering of different batch (of same partition) or it just does not guarantee.

It does guarantee the ordering within a partition either by setting

• retries=0 or
• retries>0 and max.in.flight.requests.per.connection=1.

If you increase the retries to a number larger than 0, then the configuration max.in.flight.requests.per.connection comes into effect which defaults to 5. An in-flight request means, that there can be up to 5 producer request that have not been acknowledged yet by the broker.

As described in the quoted paragraph of the documentation that could mean, that your batches B1, B2, B3, B4, B5 can all be send in parallel. If sending B3 fails and you have retries > 0, then it could potentially be written to the broker after B5.