kafka consumer not receiving message over remote

Question

Hi I'm new to kafka and I have a quick question.

I implemented a kafka producer and consumer zookeeper and producer is running in another server (192.168.10.233) Consumer is running in another server (192.168.10.234) Both are locally connected

Problem is Consumer get connected with producer but not listening any message but if I move this listening part to same server (192.168.10.233) , it is receiving the messages

this is my code for consumer

def listen(): Unit = {
    val props = new Properties();
    props.put("bootstrap.servers", "192.168.10.233:9092");
    props.put("group.id", "groupId");
    props.put("enable.auto.commit", "true");
    props.put("auto.commit.interval.ms", "1000");
    props.put("session.timeout.ms", "30000");
    props.put("key.deserializer", "org.apache.kafka.common.serialization.StringDeserializer");
    props.put("value.deserializer", "org.apache.kafka.common.serialization.StringDeserializer");
    val consumer = new KafkaConsumer(props);

    println("calling ---- but yet to receive the message")

    consumer.subscribe(List("test"));
    while (true) {
      val records = consumer.poll(100);
      for (record <- records)
      println("offset = %d, key = %s, value = %s", record.offset(), record.key(), record.value());
    }

  }

I also checked 192.168.10.233:9092 from outside ,weather the port is not blocked by anything.

I believe that there may be a problem of the offset. You can try to set the offset like: props.put("auto.offset.reset", "earliest"); If it works, you can tune that value — NangSaigon
which version of Apache Kafka are you using? 0.9 or 0.10? You should have the same version for you client API. — NangSaigon
@NangSaigon not necessarily. Kafka is backwards compatible and new brokers can handle messages from older clients. But not vice versa — serejja

serejja serejja · Accepted Answer · 2016-07-28T08:34:56

Most likely you have to set advertised.host.name in your kafka/config/server.properties to a value that is routable from outside.

kafka consumer not receiving message over remote

1 Answers