How to deploy Kafka on Google cloud

0
votes

I deployed Kafka on Google cloud, I changed listeners to

PLAINTEXT://[internal ip address]:9092

And when I try

sudo ./bin/kafka-topics.sh --list --zookeeper [external IP address]:2181

I can get the topic on the broker. However when I try to produce message to the Kafka broker

sudo ./bin/kafka-console-producer.sh --broker-list [external IP address]:9092 --topic test

following error shows up:

ERROR Error when sending message to topic test with key: null, value: 5 bytes with error: (org.apache.kafka.clients.producer.internals.ErrorLoggingCallback) org.apache.kafka.common.errors.TimeoutException: Expiring 1 record(s) for test-0: 1506 ms has passed since batch creation plus linger time

I wonder what properties did I set wrong and how to fix it?

1
apache-kafkagoogle-cloud-platformcloud

1 Answers

1
votes

You need to set advertised.listeners to the external IP so that clients can correctly connect to it. Otherwise they'll try to connect to the internal IP (since advertised.listeners will default to listeners unless explicitly set)

Ref: https://kafka.apache.org/documentation/#brokerconfigs