Kafka: Docker container and NoBrokersAvailable error

Question

I run Kafka using:

docker run -p 2181:2181 -p 9092:9092 --env ADVERTISED_HOST=192.168.99.100 --env ADVERTISED_PORT=9092 spotify/kafka

According to https://ngeor.wordpress.com/2017/03/25/kafka-with-docker-a-docker-introduction/ - the -p flag is used to publish a network port. Inside the container, ZooKeeper listens at 2181 and Kafka at 9092. If we don’t publish them with -p, they are not available outside the container, so we can’t really use them. - the –env flag sets up environment variables. These are needed as per the documentation of the spotify/kafka image - the last part specifies the image we want to run: spotify/kafka

Now I want to sent messages with a producer to this kafka port. From within another docker container:

from kafka import KafkaProducer
import json

producer = KafkaProducer(bootstrap_servers='localhost:9092',
                         value_serializer=lambda v: json.dumps(v).encode('utf-8'))

However this gives the error: kafka.errors.NoBrokersAvailable: NoBrokersAvailable

How could I make sure that the two Docker containers could communicate?

Abhimanyu Abhimanyu · Accepted Answer · 2018-01-15T05:01:39

Instead of localhost, you have to provide the docker container IP in which Kafka server is running.

Like

producer = KafkaProducer(bootstrap_servers=':9092', value_serializer=lambda v: json.dumps(v).encode('utf-8'))

Kafka: Docker container and NoBrokersAvailable error

1 Answers