Queue job instances in Spring Batch

2
votes

We have a job running in Spring batch each weekday, triggered from another system. Sometimes there are several instances of the job to be run on the same day. Each one triggered from the other system. Every job runs for about an hour and if there are several job instances to be run we experience some problems with the data.

We would like to optimize this step as following, if no job instance is running then start a new one, if there is a job instance running already put the new one in a queue. Each job instance must be COMPLETED before the next one is triggered. If one fail the next one must wait. The job parameters are an incrementer and a timestamp.

I've Googled a bit but can't find anything that I find useful. So I wonder if this is duable, to queue job instances in spring batch? If so, how do I do this? I have looked into Spring integration and job-launching-gateway but I don't really see how to implement it, I guess I don't understand how it works. I try to read about these things but I still don't understand.

Maybe I have the wrong versions of spring batch? Maybe I am missing something?

If you need more information from me please let me know! Thank you!

We are using spring-core and spring-beans 3.2.5, spring-batch-integration 1.2.2, spring-integration-core 3.0.5, spring-integration-file, -http, -sftp, -stream 2.0.3

2
spring-integrationspring-batchspring-batch-admin
Check this stackoverflow.com/questions/37405659/… – Amit K Bist

2 Answers

0
votes

Well, if you are good to have Spring Integration in your application alongside with the Spring Batch, that really would be great idea to leverage the job-launching-gateway capability.

Right, you can place your tasks into the queue - essentially QueueChannel.

The endpoint to poll that channel can be configured with the max-message-per-poll="1" to poll from the internal queue only one task at a time.

When you have just polled one message, send it into the job-launching-gateway and at the same time to the Control Bus component the command to stop that polling endpoint do not touch other messages in queue until the finish of the current job. When job is COMPLETED, you can send one more control message to start that polling endpoint.

Be sure that you use all the Spring Integration modules in the same version: spring-integration-core 3.0.5, spring-integration-file, -http, -sftp, -stream 3.0.5, as well.

0
votes

If you still require an answer one could use a ThreadPoolTaskExecutor with a core size of 1 and max size of 1 and then a queue size that you desire.

i.e.

<bean id="jobLauncherTaskExecutor"
    class="org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor">
    <property name="corePoolSize" value="1" />
    <property name="maxPoolSize" value="1" />
    <property name="queueCapacity" value="200" />
</bean>

and then pass that to the SimpleJobLauncher

i.e.

<bean id="jobLauncher" class="org.springframework.batch.core.launch.support.SimpleJobLauncher">
    <property name="jobRepository" ref="jobRepository" />
    <property name="taskExecutor" ref="jobLauncherTaskExecutor" />
</bean>