apache storm unit testing a bolt in a live topology

Question

I have a storm topology that have a spout that connects to a kafka queue and the forwards the tuple to my bolt for processing. I want to do unit testing on the bolt only, not the whole segment from kafka -> spout -> bolt. However, I also want to test the bolt in a storm topology instance, not just the pure functionality of it. The reason is that the bolt actually sends the processed data to a cassandra database.

So one way for me to achieve this is to make a test spout, connect it to the bolt, and send test tuples through the test spout to the bolt. However, that seems like maybe too much work for a test. Is there a better way to do this? Like hijacking the original spout in test to send some test tuples?

Answer 1

You might look at replacing your Kafka spout with either https://storm.apache.org/releases/0.9.6/javadocs/backtype/storm/testing/FixedTupleSpout.html or https://storm.apache.org/releases/0.9.6/javadocs/backtype/storm/testing/FeederSpout.html for tests.

A benefit of using FixedTupleSpout for testing is that it implements https://github.com/apache/storm/blob/a4afacd9617d620f50cf026fc599821f7ac25c79/storm-client/src/jvm/org/apache/storm/testing/CompletableSpout.java, so can be used with completeTopology https://github.com/apache/storm/blob/64e29f365c9b5d3e15b33f33ab64e200345333e4/storm-server/src/main/java/org/apache/storm/Testing.java#L405. This can let you write tests that set up some tuples initially, runs the topology until all tuples are acked/failed, and then lets you assert e.g. that data was written to Cassandra and all tuples were acked.

There's an example that uses completeTopology here https://github.com/xumingming/storm-lib/blob/master/src/jvm/storm/TestingApiDemo.java#L83.