Python Confluent-Kafka SSL Configuration

Question

A basic Confluent-Kafka producer and consumer have been created to send plaintext messages.

After successfully sending messages from producer to consumer, additional configs were added to use SSL rather than PLAINTEXT.

The following Configs have been implemented, which result in the following error.

"Message Timed Out"

Producer Configs:

bootstrap.servers: localhost9093
security.protocol: SSL
ssl.keystore.location: ../keystore.p12
ssl.keystore.password: [password]
ssl.ca.location: ../CARoot.pem
ssl.key.location: ../key.pem
ssl.certificate.location: ../cert.pem
ssl.key.password: [password]

Server Configs:

ssl.keystore.type= PKCS12
ssl.keystore.location= ../keystore.p12
ssl.keystore.password= [password]
ssl.ca.location= ..\\CARoot.pem
ssl.certificate.location= ..\\cert.pem
ssl.key.password= [password]
ssl.key.location= ../key.pem
security.inter.broker.protocol= SSL
listeners = PLAINTEXT://localhost:9092,SSL://localhost:9093
advertised.listeners = PLAINTEXT://localhost:9092,SSL://localhost:9093

Are there additional configs required to implement SSL?

Additionally, can anyone summarize the CARoot? From what I have read, it is a chain of certificates. Therefore, if there is only one certificate, should CARoot.pem and cert.pem be identical? This file might be the problem. The certificate and private key were created in PyOpenSSL. keystore.p12 was converted from a .jks keystore using keytool.

Is there a way to create a CARoot.pem file using this library?

Thank you.

TruBlu TruBlu · Accepted Answer · 2019-12-10T04:37:58

The producer was timing out due to the formatting of the CA Certificate file.

The solution to the following question was used to resolve the time-out error, which uses OpenSSL rather than PyOpenSSL.

Note: OpenSSL is available in Git Bash.

How to export CA certificate chain from PFX in PEM format without bag attributes

Additionally, there were some changes that were made to the configuration of both the server and producer.

Producer Configurations:

bootstrap.servers: localhost9093
security.protocol: SSL
ssl.ca.location: ../CARoot.pem
ssl.key.location: ../key.pem
ssl.certificate.location: ../cert.pem

Server Configurations:

ssl.protocol= SSL
ssl.keystore.type= JKS
ssl.keystore.location= ../keystore.jks
ssl.keystore.password= [password]
ssl.client.auth= required
security.inter.broker.protocol= SSL
listeners = PLAINTEXT://localhost:9092,SSL://localhost:9093
advertised.listeners = PLAINTEXT://localhost:9092,SSL://localhost:9093

Python Confluent-Kafka SSL Configuration

1 Answers