1
votes

I am confused by the Storm-UI statistics.

For example:

Storm-UI

Topology Stats show a number of 69 million

kafka-spout shows a number of 34 million

__acker is at 17 million

es-bolt shows 17 million too

My toplogy is kafka-spout --> es-bolt and I am not sure how the numbers above add up?

If Kafka-spout is emitting only 34 million, why do topology stats show 69 million?

And again if Kafka-spout emitted 34 million, why does es-bolt say 17 million?

I see a pattern of tuples being halved from top-to-down, but not sure I understand why? Is it because of ack-tuples or heart-beat bolts?

Are they always half of the upstream spout?

3

3 Answers

1
votes

You can turn off system stats, then numbers will make sense. There's button on the bottom of Storm UI stats page.

0
votes

Topology Stats show a number of 69 million This is sum of all your spouts and bolts: 34.x + 17.x +17.x

And emitting numbers of the spout and bolt are not necessarily related. It's related to your code.

0
votes

Keep in mind that those metrics are sampled at the rate o of topology.stats.sample.rate, 0.05 by default. If you turn it up to 1.0 you'll see full-resolution, though at the price of more time spent collecting metrics.