ItemWriter commit juste the first commit-interval

Question

I'm trying to select data from database, update each object and then update the database in an itemwriter.

I tried to flush DAO after each update but it's change nothing.

The configuration is very basic with a reader, a writer and a commit-interval of 100.

The reader is working as expected:

@Override
public Order read() throws Exception, UnexpectedInputException, ParseException, NonTransientResourceException {
    if(iterator == null) {
        List<Order> all = findOrders();
        iterator = all.listIterator();
    }

    if (iterator.hasNext()) {
        return iterator.next();
    } else {
        return null;
    }
}

The writer is very basic too:

public void write(List<? extends Order> items) throws Exception {
    @SuppressWarnings("unchecked")
    List<Order> listOrder = (List<Order>) items;

    try {
        for(Order o : listOrder) {
            etatCommandeServiceImpl.updateEtatCommande(o);
        }
    }catch(Exception e) {
        if (log.isErrorEnabled()) {
            log.error("ERROR {}", e);
        }
        throw e;
    }
}

The issue is that the first 100 records are commited but not the rest. Spring batch table shows that it reads all records and commits multiple times but when I check in the database it's just commit once.

The version of Spring-batch is 2.2.6.

UPDATE

I think this problem is due to the transaction with database, because now it doesn't commit at all to the database. But I can't figure it out for the moment.

What are you doing inside etatCommandeServiceImpl.updateEtatCommande(o) method? — Tech Guy
It just updates data. I don't think this point is problematic because it already works with a request hard-coded in the XML. But I need to go through classes to build request dynamically. — Youri C.
Are you sure your reader is not returning null after the first chunk? — Mahmoud Ben Hassine
I checked, it returns null when the iterator has no more elements — Youri C.

Tech Guy Tech Guy · Accepted Answer · 2019-05-06T15:37:46

The issue is that you are returning null from your ItemReader. If you return null Spring batch would assume end of the feed is reached and it would terminate the job. Since you are returning null after every chunk (100 in your case),Sprint batch is getting terminated after processing 100 records.

Please create method with @PostConstruct and fetch all the records that you want to process and change your ItemReader to return 1 at a time and null when all the records are read.

ItemWriter commit juste the first commit-interval

1 Answers