0
votes

Devices will send multiple messages to Azure Eventhub (still not decided). Each message will have different formats and contents (though need different processing). Each message will contain a unique ServiceID identifying what kind of msg.

We have now two scenarios in mind how to handle it.

a) 1. Spout will receive msg and pass it to a ParserBolt 2. ParserBolt will read ServiceID and based on the ID pass it to a different bolt

In this case each msg (based on the ServiceID) will go through a different path in the topology. All messages could be handled by one topology.

b) 1. Multiple Topologies - for each ServiceID one. 2. Spout will receive msg and pass it to Bolt 3. Bolt will read ServiceID - if correct serviceID will pass it to the next bolt, if not correct serviceID will dismiss this job

Since Storm guarantees that each msg is processed atleast once - each msg will come at some point to the correct topology. But the topologies will dismiss a lot of messages which were not meant for them.

1
Hey Erik, welcome to stackoverflow. Your question is pretty open ended and there isn't a clear answer. Could you rephrase to ask a more specific question? See here for more infoKit Menke

1 Answers

0
votes

If you cannot split your input stream outside of Storm, you should definitely go with a single topology. This will save a lot of network traffic and unnecessary data replication.