Spring batch - multi threaded environment

Question

I want to write a spring boot batch application where I have a database table full of events. What I want to do is have a multi threaded spring boot batch application which will work in this way:

I want to have 5 threads running each thread will keep an offset to track what event it has read so that no other thread reads the same event again. How am I thinking to do it:

Thread 1 will pick up the event if eventId % 5 == 1
Thread 2 will pick up the event if eventId % 5 == 2
Thread 2 will pick up the event if eventId % 5 == 3
Thread 2 will pick up the event if eventId % 5 == 4
Thread 2 will pick up the event if eventId % 5 == 0

so I want to be able to keep offset in a database table for each thread. Is there a way to make Spring boot environment to work in this way?

Answer 1

You can partition your table according to what you described (I guess you will have 5 partitions) and use a partitioned step configured with a TaskExecutorPartitionHandler of 5 threads. With this setup you will achieve what you are looking for.

Hope this helps.