In spring batch, how to mark a record a skipped record (without retry) during the writing phase

0
votes

Spring batch has facility to provide the declarative skip policy (i.e. skippable-exception-classes) to state that the particular record needs to be skipped in the batch processing.

This is quite straight forward in case of ItemReader and ItemProcessor (as they operate record by record basis).

However in case of ItemWriter, when the writing of the record fails (because of the DB Constraint violation), I want to skip that record and let other records go through.

As far as I have researched, I can implement this in two ways,

1) Throw the skippable exception, and Spring Batch will start retry operation with one item per batch, and so if the original batch size is 1000, then the batch will call the writer (and processor if it's transactional) 1000 times (once for each record) and record the skipCount for such item which fails with skip exception (which is most probably the same item which had failed in normal operation)

2) ItemWriter catches the SQLException, and resumes the processing the next record till the end of the items list.

The 2nd approach has a problem of losing the statistics about how many records did not go through (i.e. skipped records) and the batch will record all the items are successfully written and hence update the write count with improper value.

The 1st approach is a little bit tricky in my use-case as it involves re-execution of all the items (on DB side we have complex SPs + triggers) and therefore unnecessarily takes more time.

I am looking for some legal alternative to retry to just record the skipped record count during writing phase.

If none, I will go for the 1st option.

Thanks !

1
spring-batch

1 Answers

0
votes

This specifies after how many executions of writer the transaction is commited.

    <chunk ... commit-interval="10"/>

As you want to skip all the items that fail while persisted to DB you need commit-interval to be 1 in order to actually persist the good items and not be rolled back along a bad one.

Assuming the reader sends only one item to the processor (and not the list of 1000) reader, processor and writer get executed in order for each item. In this case option 2) is not useful as writer receives only one item always.

You can control how the skip count is incremented by calling StepContribution.html#incrementWriteCount and other increment*Count methods from this class.