Spring Batch Processor doesn't see updated data

Question

I'm using Spring Batch 3.0.7, EclipseLink 2.6.4 and Oracle 11/12.

I have a one to many relationship in the database. The batch reader reads the one side (parent) the processor reads the many side (children). I quite sure both are inserted transactionally by a different application.

The batch-Reader is derived from JdbcCursorItemReader and just sets a rowMapper and the preparedStatementSetter.

The processor is an ItemProcessor using Spring Data JpaRepository and adds the data to the parent. The Repository has @Transactional(readOnly = true).

The reader seems to use always the same database session and the processor always using a different database session.

With this setup I've got from time to time an ORA-01555 (Snapshot Too Old) error. But there is even a bigger problem: If there are new parent-children data between the job runs, the reader finds the parent but the processor does not see any children. Only the first parent/child insert works (in some way the processor stays on a fix snapshot of the Oracle data).

What I've done to fix this: on every job run I close the Spring Context and then recreating it. This resolves both problems (ORA-01555 errors and not seeing updated data by the processor).

The purpose of recreating the Spring Context is to get a new database session. I am not aware of a simpler method doing this.

I think it should not be necessary to recreate the Context but I can't find the reason for this behavior.

UPDATE: Here you can find a skeleton of the code https://github.com/th-e/SpringBatchDataPump

Add some code and configuration to your question without it is hard to see. also closing and recreating a context is a bad idea. — M. Deinum
Recreating the context was just to have a fix asap. But it was interesting that it resolves both problems. More details are coming... — Thomas_E

Igor Konoplyanko Igor Konoplyanko · Accepted Answer · 2017-08-16T08:46:22

You need to understand that transaction wraps read-process-write sequence. So if some other db write will happen between - this transaction will fail.

The reader seems to use always the same database session and the processor always using a different database session.

That is strange statement, if you are sure that this is true, then you have to check your configuration once more. It should not be the case.

You can check your transaction attributes.

    <batch:tasklet>
      <batch:transaction-attributes isolation="READ_COMMITTED" propagation="REQUIRES_NEW" timeout="200"/>
      <batch:chunk reader="myItemReader" writer="myItemWriter" commit-interval="20"/>
    </batch:tasklet>

For example if you don't mind to have dirty reads - you can change isolation to READ_UNCOMMITTED

Another solution: you can configure Retry mechanism provided by spring batch:

<batch:tasklet>
  <batch:chunk reader="myItemReader" writer="myItemWriter" commit-interval="20" retry-limit="15">
    <batch:retryable-exception-classes>
       <batch:include class="com.stackoverflow.MyRetryableException" />
    </batch:retryable-exception-classes>
  </batch:chunk>
</batch:tasklet>

Spring Batch Processor doesn't see updated data

1 Answers