retry in spring batch asyncprocessor

Question

We are building a spring batch which calls an third party API in processor section in spring batch

The requirement is when the 3rd party API is not available the spring batch need to re-try for specified threshold limit and abort the batch

I have tried spring batch retry with fault tolerant step it is working fine in a single threaded program but is is not stopping the batch when we include aysncprocessor and aysncitemwriter

https://github.com/mminella/LearningSpringBatch/blob/master/src/asyncItemProcessorItemWriter/src/main/java/io/spring/batch/configuration/JobConfiguration.java

Can some please give me some example of retry limit / custom code on how to exit in case of any exception using aysncprocessor / asyncitemwriter

Update 1-: I have tried with chunk size of 100 with 5 aysnc threads with retry limit 3, I see few records are processed only twice and few are processed thrice and finally job fails with ExhaustedRetryException

Is this the expected behavior of Spring batch ?

Update 2:- I see many post in forums with spring retry what is the exact difference between spring retry and faultTolerant (Spring Batch) step retry ?

stackoverflow.com/questions/24266598/spring-batch-retry-limit — Kavithakaran Kanapathippillai
@KavithakaranKanapathippillai: This does not work in aysncprocessor :( — ravicandy1234
it is working fine in a single threaded program but is is not stopping the batch when we include aysncprocessor and aysncitemwriter: Since its working fine let's see why it is not stopping. Are you using a thread pool executor for async components? If yes, do you shutdown this thread pool after your job? Spring Batch does not shutdown thread pools by itself. — Mahmoud Ben Hassine
yes we are using task executor but we are not shutting it down , we though SpringApplication.exit(context) would take care — ravicandy1234
it is working fine in a single threaded program but is is not stopping the batch when we include aysncprocessor and aysncitemwriter: Here I meant to say even though exception is thrown the batch continues to process and processor does not exit — ravicandy1234

Kavithakaran Kanapathippillai Kavithakaran Kanapathippillai · Accepted Answer · 2020-06-29T15:19:43

Retry is configurable for both Sync and Async but there is a difference

In oder to observe this, reduce the chunk size to 2
When it is Async, it submits every item in the chunk (in this case 2 and lets say both items are throwing exception during processing), then it will retry these 2 items again, and it will do 3 (retryLimit) rounds.

    @Bean
    public Step step1() throws Exception {
        return stepBuilderFactory.get("step1")
                .chunk(2)
                .reader(pagingItemReader())
                .processor(asyncItemProcessor())
                .faultTolerant().retryLimit(3)
                                .retry(YourException.class)
                .writer(asyncItemWriter())
                .build();
    }

Summary:

When you use synchronous processor, items in the chunk are processed sequentially. So when one item fails, only that item is retried as per retryLimit
When you use asynchronous processor, items in that chunk are processed in parallel. So when one or more items in that fails, all of them are retried 3 times as per retryLimit

retry in spring batch asyncprocessor

1 Answers