Kafka bootstrap setting precedence between cli option and configuration file

Question

I use this to consume data :

kafka-console-consumer.sh --bootstrap-server kafka01:9092 --topic test --consumer.config /test/kafka/config/consumer.properties

consumer.properties file has the following setting :

[...]

bootstrap.servers=kafka01:9092,kafka02:9092,kafka03:9092

[...]

So I'm asking what takes precedence regarding bootstrap.server ? What will be the bootstrap server which will be finally finally ? :

Only kafka01:9092 (as specified with the parameter "--bootstrap-sever) ? Or kafka01:9092,kafka02:9092,kafka03:9092 as specified in consumer.properties file ?

Thanks !

You could test this by providing a wrong server address in your --bootstrap-server configuration. — mike
You could also look at the source code, but I'm guessing that the config file overrides all other options — OneCricketeer

ppatierno ppatierno · Accepted Answer · 2020-11-28T09:09:50

The answer is confirmed by the source code as well in the ConsoleConsumer where we have

private[tools] def consumerProps(config: ConsumerConfig): Properties = {
val props = new Properties
props ++= config.consumerProps
props ++= config.extraConsumerProps
setAutoOffsetResetValue(config, props)
props.put(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG, config.bootstrapServer)
CommandLineUtils.maybeMergeOptions(
  props, ConsumerConfig.ISOLATION_LEVEL_CONFIG, config.options, config.isolationLevelOpt)
props}

So as you can see it uses the consumerProps first that are provided via the config file and then the extraConsumerProps that are the ones provided as command-line argument as defined below:

val extraConsumerProps = CommandLineUtils.parseKeyValueArgs(options.valuesOf(consumerPropertyOpt).asScala)
val consumerProps = if (options.has(consumerConfigOpt))
  Utils.loadProps(options.valueOf(consumerConfigOpt))
else
  new Properties()

Kafka bootstrap setting precedence between cli option and configuration file

2 Answers