Spring batch - How to run multiple jobs in parallel

Question

I have multiple scheduled jobs in BatchScheduler which runs at a particular time. Simple inbuilt JobLauncher which is sync. in nature is used initially. Now, I want to run the jobs in parallel so that no job can wait for other to finish.

I have tried with the @Async annotation on my different jobs but it did not worked.

I have also tried creating different JobLauncher object for each and every job but it also did not worked.

Then, I tried with setting the jobLauncher.setTaskExecutor(new SimpleAsyncTaskExecutor()). But it did not worked.

I have also tried @Bean

public JobLauncher jobLauncher() {
    final SimpleJobLauncher jobLauncher = new SimpleJobLauncher();
    jobLauncher.setJobRepository(jobRepository);
    final SimpleAsyncTaskExecutor simpleAsyncTaskExecutor = new SimpleAsyncTaskExecutor();
    jobLauncher.setTaskExecutor(simpleAsyncTaskExecutor);
    return jobLauncher;
 }

I have tried all the combinations given in different stackoverflow answers but it did not worked.

@Bean
public JobLauncher jobLauncher() {
    final SimpleJobLauncher jobLauncher = new SimpleJobLauncher();
    jobLauncher.setJobRepository(jobRepository);
    final SimpleAsyncTaskExecutor simpleAsyncTaskExecutor = new SimpleAsyncTaskExecutor();
    jobLauncher.setTaskExecutor(simpleAsyncTaskExecutor);
    return jobLauncher;
 }

Actual: But it also did not worked.

As when I am checking the starttime for the batch jobs in batch tables. The job are starting when 1 job is finished.

Expected: Jobs should run in parallel.

<beans:bean id="taskExecutor" class="org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor" p:corePoolSize="30" p:maxPoolSize="150" p:queueCapacity="10" /> — OhadR
SimpleJobLauncher with an asynchronous task executor is the way to go (The ThreadPoolTaskExecutor as mentioned by @OhadR is a good option). This will make jobs run in parallel using different threads from the pool. — Mahmoud Ben Hassine
Possible duplicate of Spring batch - running multiple jobs in parallel — Mahmoud Ben Hassine
I also tried using TaskExecutor. But it is still not happening. — s001

Paul Wong Paul Wong · Accepted Answer · 2019-07-16T08:00:55

this configuration works for me:

import org.springframework.batch.core.configuration.annotation.DefaultBatchConfigurer;
import org.springframework.batch.core.launch.JobLauncher;
import org.springframework.batch.core.launch.support.SimpleJobLauncher;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.scheduling.concurrent.ThreadPoolTaskScheduler;


@Configuration
public class TaskExecutorBatchConfigurer extends DefaultBatchConfigurer {

	@Bean
	public ThreadPoolTaskScheduler batchTaskScheduler() {
		ThreadPoolTaskScheduler threadPoolTaskScheduler = new ThreadPoolTaskScheduler();
		threadPoolTaskScheduler.setPoolSize(10);
		threadPoolTaskScheduler.afterPropertiesSet();
		return threadPoolTaskScheduler;
	}
	

	@Override
	protected JobLauncher createJobLauncher() throws Exception {
		SimpleJobLauncher jobLauncher = new SimpleJobLauncher();
		jobLauncher.setJobRepository(super.getJobRepository());
		jobLauncher.setTaskExecutor(batchTaskScheduler());
		jobLauncher.afterPropertiesSet();
		return jobLauncher;
	}
}

Spring batch - How to run multiple jobs in parallel

1 Answers