I have a spring batch job which is expected to process 'N' job-ids sequentially, based on FIFO. There are 5 steps in this spring batch job.
We use DECIDER to determine any more job-id is present.If yes, go to the first step and run all the steps for that job-id.
I see "duplicate step" message in the log emitted by spring-batch, which appears to be fine until and unless the step in the first job (say job-id=1) gets an UNKNOWN state. In such event, the same step for second job (job-id =2) fails to start stating "Step is in UNKNOWN state, it is dangerous to restart....". Is there a better approach to define spring-batch job to process 'N' job-ids.
There is a table which holds the job information. Each Job places orders in to Order table. It is possible that two jobs needs to be processed on the same day. Job can insert/update the same order number having same revision(with difference in other details) or different revision of same order number. The batch program must process these jobs in the FIFO model based on success_time in the job table.
Assume table structure as below
Job_Id job_name success_time 1 job1 2014-09-29 10:00:00 2 job2 2014-09-29 13:00:00 Order_id order_number order_revision order_details job_id 1 ABC 1 Test1 1 2 XYZ 1 Test2 1 3 ABC 2 Test1-Rev2 2
Sample configuration is shown below. For brevity, I have removed metadata definitions and reused the reader and writer.
<batch:step id="abstractParentStep" abstract="true">
<batch:tasklet>
<batch:chunk commit-interval="100" />
</batch:tasklet>
</batch:step>
<-- Using same reader and writer to simplify scenario depiction -->
<batch:job id="OrderProcessingJob">
<batch:step id="Collect-Statistics-From-Staging-Tables" next="Validate-Order-Mandatory-Fields" parent="abstractParentStep">
<batch:tasklet>
<batch:chunk reader="orderReader" writer="orderWriter" />
</batch:tasklet>
</batch:step>
<batch:step id="Validate-Order-Mandatory-Fields" next="Validate-Item-Mandatory-Fields" parent="abstractParentStep">
<batch:tasklet>
<batch:chunk reader="orderReader" writer="orderWriter" />
</batch:tasklet>
</batch:step>
<batch:step id="Validate-Item-Mandatory-Fields" next="decision" parent="abstractParentStep">
<batch:tasklet>
<batch:chunk reader="orderReader" writer="orderWriter" />
</batch:tasklet>
</batch:step>
<batch:decision id="decision" decider="processMoreJobsDecider">
<batch:next on="REPEAT" to="Validate-Order-Mandatory-Fields" />
<batch:end on="COMPLETED" />
</batch:decision>
</batch:job>
In the first step, we would check how many jobs (count) needs to be processed and places that in to ExecutionContext. In the decider, we check if the total no of jobs processed matches the count and returns REPEAT status if there are more job_ids to process.
We ran into exception as mentioned above when the first job's step remained in UNKNOWN state and second job (since decider decided there is one more job_id to process) got the exception message as shown above.
step.Duplicate step [setCurrentPartyType] detected in execution of job=[i82.job]. If either step fails, both will be executed again on restart. - org.springframework.batch.core.job.SimpleStepHandler @ 112
– danidemi