Parallel ordered processing of spring integration events

2
votes

am pretty new to the integration framework. I have a spring message listener which receives updates for stocks ( S1, S2, S3 ...). The updates for same stock need to be processed sequentially where as updates for different stocks should be processed parallel.

e.g. if sequence of updates are S1-1, S1-2, S2-1, S1-3, S3-1, S2-2, S1-3, S3-2 .. then there should be three parallel stream of processing

S1-1, S1-2, S1-3

S2-1, S2-1

S3-1, S3-2

Note that there could be thousands of such stocks.

Currently I am processing everything in parallel using executor on the channel. how can i achieve my requirements. please advise. thanks.

1
spring-integration
Thanks for the response. yes, i'm trying to process updates to one stock in order. it is not practical to have as many threads as stocks. i was thinking more like a threadpool, and if there is a thread already processing the stock, queue it for the same thread. pick a new thread rom the threadpool , only if it is a different stock. - user13792643

1 Answers

0
votes

Well, since we can't parallel "thousands of such stocks" because spawning so many threads is really inefficient, so I would go something like partitioning and queue channels for each of them.

The router component could decide by the request message which partition it belongs and send to respective queue channel.

This way a poller on that queue channel would pull messages sequentially, but different queues would be processed in parallel.

See more info in docs:

https://docs.spring.io/spring-integration/docs/current/reference/html/message-routing.html#router

https://docs.spring.io/spring-integration/docs/current/reference/html/core.html#channel-implementations-queuechannel