Config cluster ZK_HOST on Storm Topology

Question

Now I have just set up successfully my storm topology with single node on single machine. I use KafkaSpout as below:

String zkHostPort = "localhost:2181";
String topic = "sentences";

String zkRoot = "/kafka-sentence-spout";
String zkSpoutId = "sentence-spout";
ZkHosts zkHosts = new ZkHosts(zkHostPort);

SpoutConfig spoutCfg = new SpoutConfig(zkHosts, topic, zkRoot, zkSpoutId);
KafkaSpout kafkaSpout = new KafkaSpout(spoutCfg);
return kafkaSpout;

Now I set up cluster zookeeper(three node: server1.com:2181, server2.com:2181. server3.com:2181) and cluster kafka (three node). I wonder how I can change code on Storm Topology for this purpose. Please help me!!

I am not sure: did you try String zkHostPort = "server1.com:2181,server2.com:2181,server3.com:2181";? (This must be of course the ZK nodes used by your Kafka brokers -- the broker information will be fetched from there. As an alternative, you can omit ZkHosts and define static brokers as described here: storm.apache.org/releases/1.0.0/storm-kafka.html — Matthias J. Sax

Ilya Lapitan Ilya Lapitan · Accepted Answer · 2016-05-19T07:59:55

Please, use the configuration below:

String zkHostPort = "server1.com:2181,server2.com:2181,server3.com:2181";
String topic = "sentences";

String zkRoot = "/kafka-sentence-spout";
String zkSpoutId = "sentence-spout";
ZkHosts zkHosts = new ZkHosts(zkHostPort);

SpoutConfig spoutCfg = new SpoutConfig(zkHosts, topic, zkRoot, zkSpoutId);
KafkaSpout kafkaSpout = new KafkaSpout(spoutCfg);
return kafkaSpout;

Note: the most common issue here is space placed between hosts after comma, there must not be space between hosts.

Correct:

server1.com:2181,server2.com:2181,server3.com:2181

Wrong:

server1.com:2181, server2.com:2181, server3.com:2181

Config cluster ZK_HOST on Storm Topology

1 Answers