Spring Batch - Commit Interval and Skip Limit

1
votes

Solution for this scenario - commit interval is 10, skip limit is 10, Total Input Record is 20, First 9 records valid, remaining invalid.

When 9 records are already read, the 10th one is invalid. The chunk size is 10 and skip limit is 10, so will Spring batch write those 9 records in output file? If not, if it goes on reading remaining records, when it reads 20th record, count of the error record will be 11 and skip limit defined is 10. So the process will stop immediately. What about the fate of valid records read first.

Will the first 9 records be written in output file or not.

Please let me know, if any solution available. Thanks!

2
javaspringtransactionsspring-batchcommit
Good question.I think if an item writer fails to commit a chunk (here 10 items) thereby causing a rollback. stackoverflow.com/questions/11429197/… – Sajith
You ask for a "solution" but you haven't really specified a problem. Did you observe that the first 9 records were not found in the output (they should be if I understood your scenario)? – Dave Syer

2 Answers

2
votes

There is more than one possible outcome. You have to be specific about the invalid records.

Are they invalid reads: in this case, SB will just call the ItemReader next value (no rollback). so you will end up with an Skip limit exception since 11 records were found invalid.

Are they invalid writes : In this case, SB will redo the first chunk on a commit--interval of 1 until it find the bad record. In this case, you should find the first 9 records in the database.

Regards

1
votes

For this scenario - commit interval is 10, skip limit is 10, Total Input Record is 20, First 9 records valid, remaining invalid.

When 9 records are already read, the 10th one is invalid. The chunk size is 10 and skip limit is 10, so will Spring batch write those 9 records in output file.

Answer - SB will skip 10th invalid record and read 11th record and then will got writer to write correct 10th record and as only one record was invalid skip count will be 1 .

sample config -