Getting progress information from Spring Batch Tasklet

Question

I am trying to set and retrieve progress information for a currently running spring batch tasklet.

I have a simple spring batch job based on one step, eg.

   public Step jobStep() {
        return stepBuilderFactory.get(JOB_STEP_1)
                .tasklet((contribution, chunkContext) -> {
...
                }).build();
   }

The tasklet is a simple loop doing some stuff, which may take a lot of time.

After every iteration in the loop, I am setting some progress information in the JobExecutionContext eg.

chunkContext.getStepContext().getStepExecution().getJobExecution().getExecutionContext().put("myJobStatus", "10 of 100 finished");

From another thread (e.g. a rest service) I want to check the progress of the actual tasklet, im using the JobExplorer for this

String status = (String)jobExplorer.getJobExecution(jobId).getExecutionContext().get("myJobStatus");

Unfortunately the myJobStatus property is only set when the step or the job is finished. I even tried to replace the simple loop with a loop of steps, but with the same result.

Is there another way to set and access progress information for currently running jobs?

I'm using the DefaultBatchConfigurer with a JDBC based JobRepository etc.

Answer 1

This step sounds like a good candidate to be refactored into more of a Reader->Writer step rather than a loop in a Tasklet. After every chunk of X items, the Spring Batch framework will update the Step Execution with the latest read/write counts.

For long-running steps, this also has the advantage of committing with each chunk, making restartability less burdensome rather than waiting until ALL the work is done before the Tasklet issues a single, large commit.

Answer 2

Your issue is that you are saving the progress information in the job execution context, which is saved in the job repository after the step is completed. You need to use the step execution context instead since it is saved at every commit of the tasklet. Here is an extract from the Execution context section in the reference docs:

They are two different ExecutionContexts. The one scoped to the Step is saved at every commit point in the Step, whereas the one scoped to the Job is saved in between every Step execution.

So if you store the progress info in the step execution context (and not job execution context), you should be able to get it from the other thread using the job explorer. I assume your tasklet does some work, save the progress in the step execution context, then return CONTINUABLE, until no more work is left to do and return FINISHED at that point. I have a complete example in this repo, but here is the relevant part:

class MyTasklet implements Tasklet {

    private int progress;

    @Override
    public RepeatStatus execute(StepContribution contribution, ChunkContext chunkContext) throws Exception {
        RepeatStatus repeatStatus;
        if (moreWork()) {
            doSomeWork();
            repeatStatus = RepeatStatus.CONTINUABLE;
        } else {
            repeatStatus = RepeatStatus.FINISHED;
        }
        reportProgress(chunkContext);
        return repeatStatus;
    }

    private void reportProgress(ChunkContext chunkContext) {
        chunkContext.getStepContext().getStepExecution()
                .getExecutionContext().putInt("progress", this.progress++);
    }

    private void doSomeWork() throws Exception {
        Thread.sleep(5000);
        System.out.println("doing some work..");
    }

    private boolean moreWork() {
        return this.progress < 100;
    }
}