Spring Batch restart functionality not working when using @StepScope

Question

I want to use Spring Batch (v3.0.9) restart functionality so that when JobInstance restarted the process step reads from the last failed chunk point forward. My restart works fine as long as I don't use @StepScope annotation to my myBatisPagingItemReader bean method.
I was using @StepScope so that i can do late binding to get the JobParameters in my myBatisPagingItemReader bean method @Value("#{jobParameters['run-date']}"))
If I use @StepScope annotation on myBatisPagingItemReader() bean method the restart does not work as it creates new instance (scope=step, name=scopedTarget.myBatisPagingItemReader).
If i use stepscope, is it possible for my myBatisPagingItemReader to set the read.count from the last failure to get restart working?

I have explained this issue with example below.

@Configuration
@EnableBatchProcessing
public class BatchConfig {
    @Bean
    public Step step1(StepBuilderFactory stepBuilderFactory,
        ItemReader<Model> myBatisPagingItemReader,
        ItemProcessor<Model, Model> itemProcessor,
         ItemWriter<Model> itemWriter) {

         return stepBuilderFactory.get("data-load")
            .<Model, Model>chunk(10)
            .reader(myBatisPagingItemReader)
            .processor(itemProcessor)
             .writer(itemWriter)
             .listener(itemReadListener())
             .listener(new JobParameterExecutionContextCopyListener())
             .build();
   }
   @Bean
   public Job job(JobBuilderFactory jobBuilderFactory, @Qualifier("step1") 
               Step step1) {
           return jobBuilderFactory.get("load-job")
            .incrementer(new RunIdIncrementer())
            .start(step1)
            .listener(jobExecutionListener())
            .build();
   }
   @Bean
   @StepScope
   public ItemReader<Model> myBatisPagingItemReader(
         SqlSessionFactory sqlSessionFactory,
         @Value("#{JobParameters['run-date']}") String runDate) 
  {
     MyBatisPagingItemReader<Model> reader = new 
     MyBatisPagingItemReader<>();
     Map<String, Object> parameterValues = new HashMap<>();
     parameterValues.put("runDate", runDate);
     reader.setSqlSessionFactory(sqlSessionFactory);
     reader.setParameterValues(parameterValues);
     reader.setQueryId("query");
     return reader;
   }
}

Restart Example when I use @Stepscope annotation to myBatisPagingItemReader(), the reader is fetching 5 records and I have chunk size(commit-interval) set to 3.

Job Instance - 01 - Job Parameter - 01/02/2019.
chunk-1:
- process record-1
- process record-2
- process record-3
writer - writes all 3 records
chunk-1 commit successful

chunk-2:
process record-4
process record-5 - Throws and exception
Job completes and set to 'FAILED' status

Now the Job is Restarted again using same Job Parameter.
Job Instance - 01 - Job Parameter - 01/02/2019.
chunk-1:
process record-1
process record-2
process record-3
writer - writes all 3 records
chunk-1 commit successful

chunk-2:
process record-4
process record-5 - Throws and exception
Job completes and set to 'FAILED' status

The @StepScope annotation on myBatisPagingItemReader() bean method creates a new instance , see below log message.
Creating object in scope=step, name=scopedTarget.myBatisPagingItemReader
Registered destruction callback in scope=step, name=scopedTarget.myBatisPagingItemReader
As it is new instance it start the process from start, instead of starting from chunk-2.

If i don't use @Stepscope, it restarts from chunk-2 as the restarted job step sets - MyBatisPagingItemReader.read.count=3.

When you use StepScope and the job fails the first attempt, can you please share the value of read.count from the step execution context before the restart? How do you restart the job? Even if there is a new instance of the reader, this instance will be initialized with the read.count from the last (failed) step execution and continue from there. I don't see how using StepScope or not can impact restartability. — Mahmoud Ben Hassine

Michael Minella Michael Minella · Accepted Answer · 2019-02-04T13:08:06

The issue here is that you are returning an ItemReader instead of the fully qualified class (MyBatisPagingItemReader) or at least ItemStreamReader. When you use Spring Batch's step scope, we create a proxy to allow for late initialization. The proxy is based on the return type of the method (ItemReader in your case). The issue you are running into is that because the proxy is of ItemReader, Spring Batch does not know that your bean also implements ItemStream and it is that interface that enables restartability. By default, Spring Batch will automatically register all beans of type ItemStream for you (you can also explicitly register the beans yourself, but it's typically not needed).

To address your issue, the following should work (note the change in the return type):

   @Bean
   @StepScope
   public MyBatisPagingItemReader<Model> myBatisPagingItemReader(
         SqlSessionFactory sqlSessionFactory,
         @Value("#{JobParameters['run-date']}") String runDate) {

     MyBatisPagingItemReader<Model> reader = 
         new MyBatisPagingItemReader<>();

     Map<String, Object> parameterValues = new HashMap<>();
     parameterValues.put("runDate", runDate);

     reader.setSqlSessionFactory(sqlSessionFactory);
     reader.setParameterValues(parameterValues);
     reader.setQueryId("query");

     return reader;
   }

This is why it is my recommendation that where possible, when using @Bean annotated methods, you should return the most concrete type possible to allow Spring to help as much as possible.

Spring Batch restart functionality not working when using @StepScope

1 Answers