Spring-batch step not re-executed (cache)

Question

I'm working on a project that includes Spring batch, before copying the code snippets, I'm going to summarize easily how the job works with a cron.

the cron calls a rest API on my project (@PostMapping("/jobs/external/{jobName}"))
in the post method, I get the job and execute it.
in each execution, I'm supposed to run a step.
the step contains a reader (external rest call to elastic API to get documents) and a processor.

now my problem: in the catalina.out, I'm able to see the rest call from the cron every 10 minutes as configured in my cron. BUT, the step doesn't seem to make that call to elastic every 10 minutes, the batch process always has the same set of data, which is fetched one time when the batch is called during tomcat restart.

job rest api :

@PostMapping("/jobs/external/{jobName}")
@Timed
public ResponseEntity start(@PathVariable String jobName) throws BatchException {
    log.info("LAUNCHING JOB FROM EXTERNAL : {}, timestamp : {}", jobName, Instant.now().toString());
    try {
        Job job = jobRegistry.getJob(jobName);
        JobParametersBuilder builder = new JobParametersBuilder();
        builder.addDate("date", new Date());
        return Optional.of(jobLauncher.run(job, builder.toJobParameters()))
            .map(BatchExecutionVM::new)
            .map(exec -> ResponseEntity
                .ok()
                .headers(HeaderUtil.createAlert("jobManagement.started", jobName))
                .body(exec))
            .orElseGet(() -> ResponseEntity.badRequest().build());
    } catch (NoSuchJobException aEx) {
        log.warn(JOB_NOT_FOUND, aEx);
        throw new BatchException();
    } catch (JobInstanceAlreadyCompleteException | JobExecutionAlreadyRunningException | JobRestartException aEx) {
        log.warn("Job execution error.", aEx);
        throw new BatchException();
    } catch (JobParametersInvalidException aEx) {
        log.warn("Job parameters are invalid.", aEx);
        throw new BatchException();
    }
}

job configuration :

@Bean
public Job usualJob() {
    return jobBuilderFactory
        .get("usualJob")
        .incrementer(new SimpleJobIncrementer())
        .flow(readUsualStep())
        .end()
        .build();
}

@Bean
public Step readUsualStep() {
    // TODO: simplifier on n'a pas besoin de chunk
    return stepBuilderFactory.get("readUsualStep")
        .allowStartIfComplete(true)
        .<AlertDocument, Void>chunk(25)
        .readerIsTransactionalQueue()
        .reader(rowItemReader())
        .processor(rowItemProcessor())
        .build();
}
@Bean
public ItemReader<AlertDocument> rowItemReader() {
    return new UsualItemReader(usualService.getLastAlerts());
}
@Bean
public UsualMapRowProcessor rowItemProcessor() {
    return new UsualMapRowProcessor();
}

i don't know why usualService.getLastAlerts() is called just once and not every 10 minutes.

because your rowItemReader bean is a singleton and not job or step scoped. — M. Deinum
i'm sorry but i really don't master spring batch. a singleton means one instance for me, but this instance should make calls every 10 minutes no ? — user1436883
No. This instance will be created once and reused every 10 minutes. The configuration is loaded once not every 10 minutes. Hence you need to make your bean step of job scoped so that for each time it executes it recreates that bean and thus executes the request. — M. Deinum
THANK YOU M. Deinum, i annotated my steap bean with @StepScope and it worked. — user1436883

user1436883 user1436883 · Accepted Answer · 2021-10-26T14:22:55

thanks to M. Deinum, this is basically the solution :

    @Bean
@StepScope
public ItemReader<AlertDocument> rowItemReader() {
    return new UsualItemReader(usualService.getLastAlerts());
}

annotating the step bean with stepScope annotation will make it reinstantiate every step.

Spring-batch step not re-executed (cache)

1 Answers