spring-cloud-stream-kafka configuration: instanceCount & instanceIndex

Question

Running spring-boot 1.5.7.RELEASE program using spring-cloud Dalston.SR4. Worked myself through cloud-stream documentation, especially regarding kafka-binder usage, but am still unclear on some configuration aspects:

1. Do we really need to set spring.cloud.stream.{instanceCount,instanceIndex} when we're consuming from kafka brokers? I can't see why this would be required, given spring.cloud.stream.bindings.<channel>.group is set. Or is it safe to say these 2 properties are only required if cloud.stream.kafka.bindings.<channel>.autoRebalanceEnabled is set false?

2. Is there any benefit in setting cloud.stream.default.consumer.partitioned: true for kafka consumers?

3. Bonus: what's the difference between spring.cloud.stream.kafka.binder.configuration.auto.offset.reset and spring.cloud.stream.kafka.bindings.<channelName>.consumer.startOffset? Is it simply latter configuring offset-reset per channel? In any case, neither of these two are picked up, and auto.offset.reset value is still reported during startup as earliest.

Answer 1

1. Correct; it's only needed if manual partition assignment is being used.

2. Not really on the consumer side; but on the producer side, for example, if you need strict ordering - e.g., if you want all "orders" for customer "foo" to go to the same partition; you also need appropriate partitioning configuration. On the consumer side (for Kafka), it will just ensure there is a group provisioned.

3. startOffset is to override the global setting. There have been some bug fixes in the order these properties are applied example here. What version are you using? The fix is in the 1.3.0 release; it's also on the 1.2.x branch but not yet released; we need a 1.2.2.RELEASE for it to be available; please open a github issue for that, if you can't move to 1.3.