Mixing sequential and parallel step processing

Question

I have extended the FlowJob class and I set the flow as follows:

final Flow subflow1 = new FlowBuilder<Flow>("subflow1")
    .start(new StepBuilder("SubFlow1-step")
            .chunk(1000)
            .reader(reader)
            .writer(writer)
            .repository(jobRepository)
            .transactionManager(txManager).build())
    .end();
final Flow subflow2 = new FlowBuilder<Flow>("subflow2")
    .start(new StepBuilder("SubFlow2-step")
            .chunk(1000)
            .reader(reader)
            .writer(writer)
            .taskExecutor(new SimpleAsyncTaskExecutor())
            .repository(jobRepository)
            .transactionManager(txManager).build())
    .end();

setFlow(new FlowBuilder<Flow>("Flow")
            .start(new StepBuilder("Flow-step1")
                    .chunk(10000)
                    .reader(reader)
                    .writer(writer)
                    .repository(jobRepository)
                    .transactionManager(txManager).build())
            .next(new StepBuilder("Flow-step2")
                    .tasklet(processor)
                    .repository(jobRepository)
                    .transactionManager(txManager).build())
            .split(new SimpleAsyncTaskExecutor())
            .add(subflow1, subflow2)
            .end());

As you can see, what I would like to do is as follows:

Execute step 1
Execute step 2
Execute concurrently subflow1 and subflow2. subflow1 step will be executed sequentially. subflow2 step will be executed in parallel (chunks dispatched to available threads).

When I run my job, steps 1 and 2 are executed correctly, then the job terminates.

Any idea?

EDIT: Related question on SO Spring-batch flow / split after a step.

Michael Minella Michael Minella · Accepted Answer · 2015-09-03T15:08:45

I believe there is a bug logged about this. Currently the work around is to create a flow with your split and add the flow to the job like as follows:

@Bean
public Job splitJob(@Qualifier("foo") Flow foo, @Qualifier("foo") Flow bar) {
    FlowBuilder<Flow> flowBuilder = new FlowBuilder<>("split");

    Flow flow = flowBuilder.split(new SimpleAsyncTaskExecutor())
            .add(foo, bar)
            .end();

    return jobBuilderFactory.get("splitJob")
            .start(myStep1())
            .next(myStep2())
            .on("COMPLETED").to(flow)
            .end()
            .build();
}

Mixing sequential and parallel step processing

1 Answers