fault tolerant spring batch job stuck after reading few records

Question

Im using repository item reader to read transactions from DB, process and write to file using flatfileitem writer. Im using fault tolerance to skip any record if it is faulty. To check if a record is faulty, we do some validations in processor and throw custom exception.

batch config

return stepBuilderFactory.get("test")
        .<Transaction, Transaction>chunk(1)
        .reader(this.reader.read())
        .processor(transactionProcessor)
        .writer(transactionWriter)
        .faultTolerant()
        .skipPolicy(new UnlimitedSkipPolicy()
        .listener(new TransactionSkipListenerTransaction())
        .stream(this.transactionwriter)

transaction reader bean

 @Override
@StepScope
public RepositoryItemReader<Transaction> read() throws Exception {
    final ZonedDateTime zonedDateTime = LocalDate.now().atStartOfDay(ZoneId.systemDefault());
    final Date today = Date.from(zonedDateTime.toInstant());
    final Date yesterday = Date.from(zonedDateTime.minusDays(1L).toInstant());
    RepositoryItemReader<Transaction> reader = new RepositoryItemReader<>();
    reader.setRepository(transactionDao);
    reader.setMethodName("findByTransactionDateGreaterThanEqualAndTransactionDateLessThan");
    reader.setArguments(Arrays.asList(yesterday, today));
    reader.setSort(Collections.singletonMap("transactionId", Sort.Direction.ASC));
    reader.setPageSize(10000);
    return reader;
}


Processor bean 
@Override
public Transaction process(Transaction transaction) throws Exception {
if(utility.isInvalidTransaction(transaction)) {
  log.error("Invalid transaction {} and skipped from reporting", transaction.getTransactionId());
  throw new CustomException("invalid transaction" + transaction.getTransactionid());
}

And writer is just a flatfile writer. The issue is, after few transactions are read, in my case after reading and processing 9 invalid transactions, the batch job is stuck. not sure why.

One thing to notice is if i add noRetry(Exception.class) and noRollbBack(Exception.class) to config the batch is running fine. can anyone pls explain what is happening internally.

.reader(this.reader.read())
        .processor(transactionProcessor)
        .writer(transactionWriter)
        .faultTolerant()
        .skipPolicy(new UnlimitedSkipPolicy()
        .noRetry(Exception.class)
        .noRollBack(Exception.class)

Output

Invalid transaction 1

Invalid transaction 2

Invalid transaction 9

Stuck here

Please share the last few lines of your debug log before the job is stuck. It is necessary to share this detail to know where exactly the job is blocked. Otherwise, please provide a minimal example that reproduces the issue to be able to help you in an efficient way. — Mahmoud Ben Hassine

msucil msucil · Accepted Answer · 2021-05-04T14:31:51

In my view the responsibilities of process bean is to add custom transformation or logic on the basis of which the data will be passed to the writer in that case I simply log the error and return null in that case where the writer handles the null object reference i.e. invalid transaction (null) is not passed to writer.

fault tolerant spring batch job stuck after reading few records

1 Answers