Spring Batch reading file with related records

Question

i am using spring batch to read a flat file. The file has related records. ie, there can be a parent record and any number of child records i wanted to read all the records and call web service to store it. I also wanted to capture the relationship and store it.one challenge is child record can be anywhere in the file. And child can also have many children records.I am unable to find the solution for this problem with spring batch. please provide your suggestions

update: I dont have any option to use database as temporary storage of data.

Michail Nikolaev Michail Nikolaev · Accepted Answer · 2013-03-07T10:05:21

I had solved such problem by processing the file multiple times.

On every pass I'll try to read\process every record in the file with such alg:

if record has parent - check if parent already stored. If no - I skip it in processor
if record unchanged (or already stored if updates are not possible) - skip it in processor
else - store in db

And then declare loop and decider:

    <batch:step id="processParentAndChilds" next="loop">
        <batch:tasklet>
            <batch:chunk reader="processParentAndChildsReader"
                         commit-interval="1000">
                <batch:processor>
                    <bean class="processParentAndChildsProcessor"/>
                </batch:processor>
                <batch:writer>
                    <bean class="processParentAndChildsWriter"/>
                </batch:writer>
            </batch:chunk>
        </batch:tasklet>
    </batch:step>
    <batch:decision decider="processParentAndChildsRetryDecider" id="loop">
        <batch:next on="NEXT_LEVEL" to="processprocessParentAndChilds"/>
        <batch:next on="COMPLETED" to="goToNextSteps"/>
    </batch:decision> 


public class ProcessParentAndChildsRetryDecider implements JobExecutionDecider{
@Override
public FlowExecutionStatus decide(JobExecution jobExecution, StepExecution stepExecution) {
    // if no on record written - no sense to try again
    if (stepExecution.getWriteCount() > 0) {
        return new FlowExecutionStatus("NEXT_LEVEL");
    } else {
        return FlowExecutionStatus.COMPLETED;
    }
}

}

Spring Batch reading file with related records

1 Answers