Dockerized Kafka TimeoutException with localhost

Question

Hi I got an issue with my dockerized kafka server. It does not create messages when send by kafka-console-producer.bat to localhost The eror message is:

ERROR Error when sending message to topic test with key: null, value: 2 bytes with error: (org.apache.kafka.clients.producer.internals.ErrorLoggingCallback) org.apache.kafka.common.errors.TimeoutException: Topic test not present in metadata after 60000 ms.

The zookeeper and kafka servers are running fine and my yml looks like this:

version: '3' 

services:
  zookeeper:
    image: zookeeper 
    ports:
      - 2181:2181
    networks:
      - test
  kafka:
    image: kafka:0.1
    hostname: kafka
    ports:
      - 9092:9092
    environment:
      - KAFKA_ZOOKEEPER_CONNECT=zookeeper:2181
      - KAFKA_LISTENERS=DOCKER_INTERNAL://kafka:29092,LOCALHOST://localhost:9092
      - KAFKA_ADVERTISED_LISTENERS=DOCKER_INTERNAL://kafka:29092,LOCALHOST://localhost:9092
      - KAFKA_LISTENER_SECURITY_PROTOCOL_MAP=DOCKER_INTERNAL:PLAINTEXT,LOCALHOST:PLAINTEXT
      - KAFKA_INTER_BROKER_LISTENER_NAME=DOCKER_INTERNAL
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
    networks:
      - test
networks:
  test:

The kafka dockerfile basically just writes the properties to the config/server.properties of kafka and executes kafka-server-start.sh from the latest kafka distribution 2.12-2.3.0

I created a topic during start up and that is retrievable from zookeeper properly: .\kafka-topics.bat --describe --topic outbound --zookeeper localhost:2181 Result:

Topic:inbound   PartitionCount:1        ReplicationFactor:1     Configs:
    Topic: inbound  Partition: 0    Leader: 1       Replicas: 1     Isr: 1

But even using this topic to write a message I get the same error as with a new topic.

Maybe someone had a similar issue, Christian

You described --topic output but you show topic:inbound... And you produced to test... If test topic was not already existing, then the connection would timeout during producing until the topic is created (which assumes that auto topic creation is enabled). — OneCricketeer
Also, you wrote your own Docker image? Why not use wurstmeister or confluentinc versions? — OneCricketeer

cbac cbac · Accepted Answer · 2019-09-12T09:45:53

I got it to work :) So the kafka localhost listener has to bind to all interfaces (maybe something to do with the docker network bridge but I am not sure). So the yml property is: KAFKA_LISTENERS=DOCKER_INTERNAL://kafka:29092,LOCALHOST://:9092 KAFKA_ADVERTISED_LISTENERS=DOCKER_INTERNAL://kafka:29092,LOCALHOST://localhost:9092

Dockerized Kafka TimeoutException with localhost

1 Answers