Spring Batch retry-policy, and skip-policy issue

7
votes

I have following step in batch job.

    <batch:step id="parse-step">
        <batch:tasklet>
            <batch:chunk reader="xmlCommonReader"
                         processor="xmlCommonProcessor"
                         writer="xmlCommonWriter"
                         commit-interval="1">
                <batch:skip-policy>
                    <bean class="org.springframework.batch.core.step.skip.AlwaysSkipItemSkipPolicy" scope="step"/>
                </batch:skip-policy>
                <batch:retry-policy>
                    <bean class="org.springframework.retry.policy.NeverRetryPolicy" scope="step"/>
                </batch:retry-policy>
            </batch:chunk>
        </batch:tasklet>
        <batch:next on="FAILED" to="file-failed-step"/>
        <batch:next on="COMPLETED" to="file-success-step"/>
        <batch:listeners>
            <batch:listener ref="parseStepExecutionListener"/>
            <batch:listener ref="parseStepSkipListener"/>
        </batch:listeners>
    </batch:step>

When some exception throws, i catch him in parseStepSkipListener and log in database.

I expect the following behavior:

  1. Job Started
  2. Executing previous steps
  3. Start execution of parse-step
  4. Read item
  5. Process item
  6. Write
    1. Ooooops, exception.
    2. Catch exception, log in database, go to next chunk(Read, Process, Write).
  7. Continue execute other steps.
  8. Finish job.

But actually i get following behavior:

  1. Job Started
  2. Executing previous steps
  3. Start execution of parse-step
  4. Read item
  5. Process item
  6. Write
    1. Ooooops, exception.
    2. Process item
    3. Write item
      1. Ooooops, exception.
      2. Catch exception, log in database, go to next chunk(Read, Process, Write).
  7. Continue execute other steps.
  8. Finish job.

So, one chunk of data try to process and write two times.

1
springspring-batch

1 Answers

10
votes

In few words:

This happens because when an error occured in write step SB doesn't know which object caused the exception so a rollback is performed and every single item of last uncommited chunk is processed/writed again as a mini-chunk to detect which object was the cause of main write error. You can read more (with diagrams) here