spring batch multi-threaded processor

Question

I am trying to process records in processor step using multiple processor classes. These classes can work in parallel. Currently I have written a multi threaded step where I

1. Set input and output row for a processor class
2. Submit it to Executor service
3. Get all future objects and collect final output

Now as soon as I make my job parallel by adding taskExecutor ; I get issues as input objects set in step 1 get overwritten in step 2 and processors are called with overwritten values. I tried to search if I can write composite processor that delegates task to multiple steps in parallel but they work only in sequential manner.

Any inputs would be greatly helpful. Thanks !

Answer 1

Welcome to concurrency. You can get yourself into allot of trouble when you do not follow the path which keeps you in safe deterministic world. You can get rid of all your issues if you use pure functions. As in your functions do not have any side effects, all your variables should be final, you'll find that you wont have any concurrency issues if you stick to this. In general stay away for the threading libraries that get shipped with Java. You should treat thread pools and executors etc. as a resource. Probably should do a bit of reading about concurrency, locks, volatile variables, why these lower level constructs are hard to use, and then look at higher order constructs such as promises, futures and actors.