I am using splitter aggregator pattern in my application. I have the following configuration -
<int:splitter inputChannel="CH1" outputChannel="CH2" ref="foo" method="bar"/>
<int:aggregator inputChannel="CH2" outputChannel="CH3" ref="oof" method="rab" ---other required configurations/>
<task:scheduler --with threadpool of size 30 />
<!--there are other channels as well. Some are queue and some are direct channels-->
All my channels(CH1, CH2,CH3) are QueueChannel.
Source to the Splitter input channel CH1 is a file.
In my testing, I observed that even If add two files in the CH1 channel, only 1 file was being processed at a given time. So I added a Poller to my CH1 channel and now multiple input messages on CH1 channels are getting processed concurrently.
On the Aggregator side as well, I am observing that execution is always single threaded i.e. until first thread completes execution, second thread doesn't start execution.
What I have tried so far to solve this -
- Adding poller to the
<int:aggregate>config - Adding poller to the input channel CH2 (input to aggregator)
- Adding dispatcher to the input Channel CH2
Can someone help me make the aggregator processing multithreaded as well.
Update1 What I am trying to do:
I recieve files containing stock data. I create message for each file and put it in CH1 channel. CH1 channel has a splitter which splits it into individual records and send it to CH2 channel. CH2 channel has a aggregator associated with it which will try to aggregate records based on the file name and 'oof.rab' writes the records in database. My expectation is to see multiple batches of the same file being concurrently written to database.
Update2: Aggregation is done here to batch the lines, so that we can write it out to database in batch mode rather than writing the lines individually
