Deploying topology on Storm Server Error: NoNode for /brokers/topics/blockdata/partitions

0
votes

I am trying to deploy storm/flux topology to remote server. The topology gets deployed succesfully but on the servers storm UI, it shows spout lag error:

Unable to get offset lags for kafka. Reason: org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /brokers/topics/blockdata/partitions at org.apache.zookeeper.keeperExceptiom.create...

Although I have checked in my zookeeper using following:

ls /brokers/topics/blockdata/partitions, it exists.

Here is my spout config:

  - id: "zkHosts"
    className: "org.apache.storm.kafka.ZkHosts"
    constructorArgs:
      - "172.25.33.191:2181"

  - id: "startingOffsetTime"
    className: "kafka.api.OffsetRequest"
    factory: "LatestTime"



  - id: "spoutConfig"
    className: "org.apache.storm.kafka.SpoutConfig"
    constructorArgs:
      - ref: "zkHosts"
      - "blockdata"
      - "/brokers/topics"
      - "myId"
    properties:
      - name: "ignoreZkOffsets"
        value: true
      - name: "scheme"
        ref: "stringMultiScheme"
      - name: "startOffsetTime"
        ref: "startingOffsetTime"

NOTE: This topology runs fine in local mode.

Please see the screenshots below:

Error Screenshot

2
apache-kafkaapache-storm

2 Answers

1
votes

This is coming from a part of Storm UI that is supposed to show you how far your spout is behind the log end offset. If you're interested the code is here https://github.com/apache/storm/blob/v1.2.1/external/storm-kafka-monitor/src/main/java/org/apache/storm/kafka/monitor/KafkaOffsetLagUtil.java

Please try to let the topology run for a bit, then try to reopen Storm UI and see if the error keeps happening. It's possible you just happened to open Storm UI before the Kafka spout managed to write to Zookeeper.

0
votes

You may not use collector.ack(tuple); in your bolt.It cause the Spout repeat send tuple and do not record offset in zookeeper.