0
votes

My Storm Spout contains a Map<Object, Foo> of message Id objects with their corresponding Foo objects I'm emitting. Every time the spout processes a message in nextTuple(), the message Id with the Foo object is added to the map. When fail is called in the Spout, I use the message Id object to retrieve the Foo object from the map and replay the message (guaranteed message processing). On ack, I remove the message from the map.

This works for a single instance of the spout. I'm trying to verify if this will work if I parallelize the Spout, i.e. is a message always acked/failed by the spout that emitted it or can another instance of the spout pick up the ack/fail call?

1

1 Answers

1
votes

If I'm interpreting Storm's guaranteed message processing documentation correctly, then the Spout that emitted the tuple will always receive the ack/fail call:

"Note that a tuple will be acked or failed by the exact same Spout task that created it. So if a Spout is executing as many tasks across the cluster, a tuple won't be acked or failed by a different task than the one that created it."