Return job id "immediately" for spring batch job before it completes

Question

I am working on a project where we are using Spring Boot, Spring Batch and Camel.

The batch process is started by a call to a rest endpoint. The rest controller starts a camel route that starts the spring batch job flow (via spring batch camel component).

I have no control over the external application that calls my application. My application is part of a bigger nightly work flow.

The batch job can take a long time to complete and therefore the external application periodically polls my batch job via another rest endpoint asking if the job is complete. It does this by polling a status rest endpoint with the id of the jobExecution it wants a status on.

To accomplish this flow I have implemented a rest controller that starts the camel route via a ProducerTemplate. My problem is returning the job execution id immediately after starting the camel route. I don't want the rest call to wait until the job is complete to return.

startJobViaRestCall ------> createBatchJob ----> runBatchJobUntilDone
                                  |
                                  |
       Return jobExecutionData    |
<----------------------------------

I have tried using async calls and futures, but with no luck. I have also tried to use Camels wiretap to no avail. The problem is that there is only "onComplete" events. I need an hook that returns as soon as the job has been created, but not run.

For example, the following code waits until the batch job is done before returning the JobExecution data I want to send back (as json). It makes sense as extractFutureBody will wait until the response is ready.

@RestController
@Slf4j
public class BatchJobController {

    @Autowired
    ProducerTemplate producerTemplate;

    @RequestMapping(value = "/batch/job/start", method = RequestMethod.GET)
    @ResponseBody
    public String startBatchJob() {
        log.info("BatchJob start called...");

        String jobExecution = producerTemplate.extractFutureBody(producerTemplate.asyncRequestBody(BatchRoute.ENDPOINT_JOB_START, ""), String.class);

        return jobExecution;
    }

}

The camel route is a simple call to the spring-batch-component

public class BatchRoute<I, O> extends BaseRoute {

    private static final String ROUTE_START_BATCH = "spring-batch:springBatchJob";

    @Override
    public void configure() {

        super.configure();
        from(ENDPOINT_JOB_START).to(ROUTE_START_BATCH);

    }
}

Any ideas as to how I can return the JobExecution data as soon as it is available?

Karthik Prasad Karthik Prasad · Accepted Answer · 2016-12-29T09:36:53

Not sure How you could do it in Camel, but here is sample Job execution using spring-rest.

@RestController
public class KpRest {

    private static final Logger LOG = LoggerFactory.getLogger(KpRest.class);
    private static String RUN_ID_KEY = "run.id";

    @Autowired
    private JobLauncher launcher;

    private final AtomicLong incrementer = new AtomicLong();


    @Autowired
    private Job job;


    @RequestMapping("/hello")
    public String sayHello(){

        try {
            JobParameters parameters = new JobParametersBuilder().addLong(RUN_ID_KEY, incrementer.incrementAndGet()).toJobParameters();
            JobExecution execution = launcher.run(job, parameters);
            LOG.info("JobId {}, JobStatus {}", execution.getJobId(), execution.getStatus().getBatchStatus());
            return String.valueOf(execution.getJobId());
        } catch (JobExecutionAlreadyRunningException | JobRestartException | JobInstanceAlreadyCompleteException
                | JobParametersInvalidException e) {
            LOG.info("Job execution failed, {}", e);
        }
        return "Some Error";
    }
}

You can make the Job async by modifying JobLauncher.

    @Bean
    public JobLauncher simpleJobLauncher(JobRepository jobRepository){
        SimpleJobLauncher jobLauncher = new SimpleJobLauncher();
        jobLauncher.setJobRepository(jobRepository);
        jobLauncher.setTaskExecutor(new SimpleAsyncTaskExecutor());
        return jobLauncher;
    }

Refer the documentation for more info

Return job id "immediately" for spring batch job before it completes

1 Answers