Unable to get docker and zookeeper to connect docker

1
votes

I have the following two containers started but kafka says connection refused.

docker run zookeeper and docker run wurstmeister/kafka

I run it just like that and the connection is refused. In the logs it says:

INFO Opening socket connection to server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error) (org.apache.zookeeper.ClientCnxn)

then:

WARN Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect (org.apache.zookeeper.ClientCnxn) java.net.ConnectException: Connection refused

zookeeper is started first and it on 2181, when starting kafka it looks at 2181 "zookeeper" then blows up on me. Advice?

4
docker

4 Answers

2
votes

Hi Drew you need to provide specific zookeeper docker container IP. Instead of localhost in Kafka configuration. You can get Docker IP by running

docker ps

docker inspect

And you need to make sure that Docker containers are able to communicate with each other. Please refer the following https://docs.docker.com/engine/userguide/networking/default_network/container-communication/

2
votes

Had the same issue. What helped was actualy:

  1. Creating a bridge network: docker network create -d bridge kafka-network

  2. Running a Zookeeper within this network: docker run -d -p 2181:2181 --network kafka-network --name zookeeper zookeeper:latest along with exposing port 2181

  3. Running a Kafka docker attaching it to the network and using Zookeeper's container name as a host: docker run -d -e KAFKA_ZOOKEEPER_CONNECT=zookeeper:2181 -e KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://localhost:9092 -p 9092:9092 --network kafka-network --name kafka confluentinc/cp-kafka:latest

0
votes

This means the zookeeper is not running on the localhost ip(127.0.0.1). You need to explicitly get it via

docker inspect zookeeper --format='{{ .NetworkSettings.IPAddress }}'

The above command should give you the zoo-keeper's container IP which can be used to start the Kafka server.

To Sum up, below commands should work:

docker run --name zookeeper  -p 2181:2181 -d zookeeper

# if bash/zsh
zk_ip=$(docker inspect zookeeper --format='{{ .NetworkSettings.IPAddress }}')
# if fish shell, do below
# set -l zk_ip (docker inspect zookeeper --format='{{ .NetworkSettings.IPAddress }}')

docker run -p 9092:9092 --name kafka  \
 -e KAFKA_ZOOKEEPER_CONNECT=${zk_ip}:2181 \
 -e KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://localhost:9092 \
 -e KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR=1 \
 -d confluentinc/cp-kafka
0
votes

Using the Docker 'host' network mode instead of the default 'bridge' mode might help, at least this is the working solution for my project. The 'host' network mode bypasses the IP issue by using the network stack of the Docker host directly instead of creating an isolated Docker network.

As an example see: https://github.com/mwatzer/kafka-connect-transforms-json-extractor-test/blob/main/docker/kafka/docker-compose.yml