How to commit a file(entire file) in spring batch without using chunks - commit interval?

Question

Commit interval will commit the data at specified intervals. I want to commit the entire file at a single shot since my requirement is to validate the file (line by line) and if it fails at any point . roll back. no commit. is there any way to achieve this in spring batch?

Answer 1

You can either set your commit-interval to Integer.MAX_VALUE (2³¹-1) or create your own CompletionPolicy.

Here's how you configure a step to use a custom CompletionPolicy :

<chunk reader="reader" writer="writer" chunk-completion-policy="completionPolicy"/>

<bean id="completionPolicy" class="xx.xx.xx.CompletionPolicy"/>

Then you have to either choose an out-of-the-box CompletionPolicy provided by Spring Batch (a list of implementations is available on previous link) or create your own.

Answer 2

What do you mean by "commit"?

You are talking about validating and not about writing the read data to another file or into database.

As mentioned in the comment by Michael Prarlow, memory problems could arise, if the size of the file changes.

In order to prevent this, I would suggest to start your job with a validation step. Simply read the data chunkwise, check the data line by line in your processor and throw a none-skippable exception, if the line is not valid. Use a passthroughwriter, so nothing is persisted. If there is a problem, the whole job will fail.

If you really have to write the data into a db or another file, you could do this in a second step. Since you have validated your data, you shouldn't observe any problems.

Simple PassThroughItemWriter

public class PassThroughItemWriter<T> implements ItemWriter<T> {
    public void write(List<? extends T> items) {
        // do nothing
    }
}

or, if you use the Java-Api to build your job and steps, you could simply use a lambda:

stepBuilders.get("step")
  .<..., ...>chunk(..)
  .reader(...)
  .processor(...)  // your processor with the validation logic

  .writer(items -> {}) // empty lambda expression
  .build();