How incorporate Storm component-specific configuration data?

0
votes

I have a Storm topology containing spouts/bolts. There is some configuration data that is specific to a particular spout and also a particular bolt that I would like to use (i.e. read from a config file) so that it is not hard coded. Examples of config data is a filename that the spout is to read from and a filename that a bolt is to write to. I think config data is passed into the open and prepare methods.

How can I incorporate the component-specific data from a configuration file?

1
apache-storm

1 Answers

0
votes

There are at least two ways to do this:

1) Include application-specific configuration in Storm config, which will be available during IBolt.prepare() ISpout.open() method calls. One strategy you could use is to have application prefix for the configuration keys, avoiding potential conflicts.

Config conf = new backtype.storm.Config();    
// Storm-specific configuration
// ...
// ..
// .

conf.put("my.application.configuration.foo", "foo");
conf.put("my.application.configuration.bar", "foo");

StormSubmitter.submitTopology(topologyName, conf, topology);

2) Include component configuration during Spout/Bolt constructor.

Properties properties = new java.util.Properties();
properties.load(new FileReader("config-file"));

BaseComponent bolt = new MyBoltImpl(properties);