Giving the following docker-compose yml, I keep running into the problem that the zookeeper forgets about created topics. I can stop and restart the zookeeper container, and also the kafka container. The problems arises when I run docker-compose down.
After a restart I see kafka reopening correctly with the topics in the /data directory, but after calling the describe topic function I get an empty list, as if it does not exist.
What am I doing wrong? Command I run to create the topics:
unset JMX_PORT
kafka-topics.sh --create --topic users --replication-factor 1 --partitions 2 --zookeeper zk:2181
to check if there are two partitions created:
kafka-topics.sh --topic users --describe --zookeeper=zk:2181
This is the docker-compose file, but it seems ok to me ...
version: '2'
volumes:
kafka-data:
external: true
kafka-zk:
external: true
services:
zookeeper:
image: zookeeper:3.4
volumes:
- kafka-zk:/data
kafka:
image: ches/kafka
environment:
KAFKA_ADVERTISED_HOST_NAME: '127.0.0.1'
KAFKA_BROKER_ID: '0'
ZOOKEEPER_CONNECTION_STRING: zk:2181
ports:
- "9092:9092"
volumes:
- kafka-data:/data
links:
- zookeeper:zk
