0
votes

Is there any issue if we use Partitioner along with multi threaded steps ?

My idea is that if I have to process huge records from 2 different database tables , then I use partitioner to split the tables and then each step with Step scope will execute queries against a table in a multi thread manner ? I am planning to use JDBCPagingItemReadr and JDBCBatchItemWriter

1
Did you face any issue with what you are proposing? - Sandeep Kumar
not really..everything looks good...except that sometimes the final commit to the database is not matching with commit-interval/chunk size... - LBPS

1 Answers

0
votes

In my experience I don't see any problem with that, as the workers in partitioned Spring Batch are nothing but Steps themselves. So nothing prevents you from doing so.

See the documentation.

But you should keep in mind whether you are going to launch your workers on the same machine or not. And thus if you will gain something from multithreading you workers or not.

Hope this helps