How do I prevent rollback when exception occurs in ItemWriter?

Question

Our writer is designed to write records to a relational database. If exception occurs on any of the records, Spring Batch performs a rollback and retry write operation on each record in the chunk. This results in SQL Duplicate Key exception to occur since previously processed records in the chunk were successfully written to the database.

We have tried making use of noRetry() and noRollback(), specifying explicitly a list of exceptions that should not trigger retry or rollback.

According to Spring Batch online documentation noRollback() could be used to prevent rollback when error occurs on ItemWriter: https://docs.spring.io/spring-batch/4.1.x/reference/html/step.html#controllingRollback

However, this contradicts java doc in the source code which says that FaultTolerantStepBuilder.noRollback() is ignored during write: https://docs.spring.io/spring-batch/4.1.x/api/index.html?org/springframework/batch/core/step/builder/FaultTolerantStepBuilder.html

Here is a sample of our Job definition:

    @Bean("my-job")
    public Job job(Step step) {
        return jobBuilderFactory.get("my-job")
                .start(step)
                .build();
    }

    @Bean
    public Step step() {
        return stepBuilderFactory.get("skip-step")
            .<String, String>chunk(3)
            .reader(reader())
            .processor(myprocessor())
            .writer(this::write)
            .faultTolerant()
            .skipLimit(1)
            .skip(JobSkippableException.class)
            .noRollback(JobSkippableException.class)
            .noRetry(JobSkippableException.class)
            .processorNonTransactional()
            .build();
    }

    public ItemReader<String> reader() {
        return new ItemReader<String> () {

            @Override
            public String read() throws Exception, UnexpectedInputException, ParseException, NonTransientResourceException {
                String s = randomUUID().toString(); 
                logger.debug("READ STRING {}", s);  
                return s;
            }
        };
    }

    public void write(List<? extends String> items) {
        for(String s : items) {
            logger.debug("WRITE STRING {}", s);
            throw new JobSkippableException("My skippable exception");
        }
    }

    public ItemProcessor <String, String> myprocessor() {
        return new ItemProcessor<String, String>() {

            @Override
            public String process(String item) throws Exception {
                logger.debug("PROCESS STRING {}", item);
                return item;
            }
        };
    }

Our expected behavior is that exceptions that occur in write doesn’t trigger a retry or rollback. This would prevent repeat calls to database and hence not cause SQL Duplicate Key exception.

Possible duplicate of Spring Batch skip exception for ItemWriter — Mahmoud Ben Hassine
Possible duplicate of stackoverflow.com/questions/54628388/… — Mahmoud Ben Hassine

Stefan Reisner Stefan Reisner · Accepted Answer · 2019-09-04T19:56:48

Not a solution, but at least an explanation for why the framework does not behave as you expect I found in lines 335-350 of FaultTolerantChunkProcessor:

                    try {
                        doWrite(outputs.getItems());
                    }
                    catch (Exception e) {
                        if (rollbackClassifier.classify(e)) {
                            throw e;
                        }
                        /*
                         * If the exception is marked as no-rollback, we need to
                         * override that, otherwise there's no way to write the
                         * rest of the chunk or to honour the skip listener
                         * contract.
                         */
                        throw new ForceRollbackForWriteSkipException(
                                "Force rollback on skippable exception so that skipped item can be located.", e);                   }

How do I prevent rollback when exception occurs in ItemWriter?

1 Answers