How to provide SSL configuration for spring-cloud-starter-bus-kafka 1.2.0.RELEASE

Question

I am using Spring Boot 1.5.6.RELEASE, Spring Cloud Config 1.3.2.RELEASE, and Spring Cloud Starter Bus (Kafka) 1.2.0.RELEASE, and the Kafka version is 0.10.1.1. I am trying to configure it to use two ssl-enabled brokers, but I end up getting a repeated sequence of warning messages (one for each broker) that looks like this:

Bootstrap broker broker1:9993 disconnected
Bootstrap broker broker2:9993 disconnected

In my configuration, I have (just the relevant portion included here):

spring:
    cloud:
        stream:
            kafka:
                binder:
                    zkNodes: zkhost1:2181,zkhost2:2181
                    brokers: broker1:9993,broker2:9993
                    configuration:
                        security.protocol: SSL
                        ssl.enabled.protocols: TLSv1.1,TLSv1.2
                        ssl.endpoint.identification.algorithm: HTTPS

However, when ProducerConfig and ConsumerConfig spit out their configuration values, they both indicate:

security.protocol = PLAINTEXT
ssl.enabled.protocols = [TLSv1.2, TLSv1.1, TLSv1]
ssl.endpoint.identification.algorithm = null

Also, I get the following warning messages:

The configuration 'key.deserializer' was supplied but isn't a known config.
The configuration 'value.deserializer' was supplied but isn't a known config.
The configuration 'ssl' was supplied but isn't a known config.
The configuration 'security' was supplied but isn't a known config.

I am going to need to add configuration for the truststore, keystore, and their passwords and store types, and I would also like to know how to provide those without hard-coding them in the yaml configs. My preferred method is to populate these config properties with jvm properties.

Any idea what I'm doing wrong?

Here is an interesting update after trying various things. If I create a @Configuration class annotated with @EnableKafkaStreams, I can create a StreamsConfig with all of the various properties configured as specified, but this creates a new and separate producer and consumer. The spring cloud bus kafka producer and consumer are still not configured beyond the defaults.

Steve Storck Steve Storck · Accepted Answer · 2017-10-20T12:38:28

If I follow the advice at github and put the properties in application.properties in the jar, and not configure them via spring-cloud-config, it works. The same configuration properties configured via spring-cloud-config cause the configuration to crash since it does not seem to like the brackets in the property names. The errors that it reports do not directly complain about malformed property keys, though. When you only perform this configuration via spring-boot, it seems to work just fine. So, it seems that there is a discrepancy in configuration parsing between spring-boot and spring-cloud-config.

How to provide SSL configuration for spring-cloud-starter-bus-kafka 1.2.0.RELEASE

1 Answers