Spring Data Flow overriding job parameters at launch

Question

I am currently facing an issue with my Tasks invocation in Spring Cloud Data Flow.

I have an Spring Batch (containing single-tasklet-step job) application registered on SCDF and a task definition based on this app. During my first launch of this task, I used a couple job parameters/arguments. Due a reason that I do now know, all my subsequent launches are getting their parameters overriden by the first set I used.

I am using SCDF 1.4.0 + MSSQL Database, but the very same behavior happens using SCDF 1.3.2 + H2 or MSSQL as well.

BatchConfig.java

    @Configuration
    public class BatchConfig {

        @Autowired
        TaskletStep taskletStep;

        @Autowired
        public JobBuilderFactory jobBuilderFactory;

        @Autowired
        public StepBuilderFactory stepBuilderFactory;


        @Bean
        public Step step1() {
            return stepBuilderFactory.get("step1")
                    .tasklet(taskletStep)
                    .build();
        }

        @Bean
        public Job job() throws Exception {
            return jobBuilderFactory.get("job")
                    .incrementer(new RunIdIncrementer())
                    .start(step1())
                    .build();
        }
}

TaskletStep.java:

@Configuration
@StepScope
public class TaskletStep  implements Tasklet{


    @Value("#{jobParameters['filePath']}")
    private String filePath;

    @Value("#{jobParameters['informante']}")
    private String informante;

    @Autowired
    RemessaParser remessaParserService;

    @Override
    public RepeatStatus execute(StepContribution contribution, ChunkContext chunkContext) throws Exception {
        GICLogger.info("Recebido job com path: " + filePath + " para o informante "+ informante);
        try{
            Path remessa = Paths.get(filePath);
            if(Files.exists(remessa)){
                String idRemessa = remessaParserService.remessaReader(remessa, informante);
                GICLogger.info("### TaskletStep:" + idRemessa + " é o ID da Remessa!");
                return RepeatStatus.FINISHED;
            }else{
                GICLogger.error("Não foi possível encontrar a remessa em "+filePath);
            }
        }catch(Exception e){
            e.printStackTrace();
        }
        return RepeatStatus.FINISHED;
    }
}

My launch command:

dataflow> task launch negTask --arguments "filePath=/tmp/jars/remessa.txt informante=CaixaB --spring.cloud.task.closecontext_enable=false"

Application log:

2018-04-04 13:33:28 [main] INFO c.b.g.n.BatchNegativacaoApp - Started BatchNegativacaoApp in 13.938 seconds (JVM running for 14.599)

2018-04-04 13:33:28 [main] INFO o.s.b.a.b.JobLauncherCommandLineRunner - Running default command line with: [filePath=/tmp/jars/remessa.txt, informante=Caixa, --spring.cloud.task.closecontext_enable=false, --spring.cloud.task.executionid=17]

2018-04-04 13:33:28 [main] INFO o.s.b.c.l.support.SimpleJobLauncher - Job: [SimpleJob: [name=job]] launched with the following parameters: [{filePath=/home/enrico/PROJETOS/GIC/java/remessa.txt, -spring.cloud.task.executionid=8, informante=Caixa, -spring.cloud.task.closecontext_enable=false, run.id=12, time=1522842134819}]

Do you guys have any idea of why does it happen?

Thanks for your attention and any input!

Best regards, Enrico

Not that this is the issue, but you don't need @Configuration on your TaskletStep. Just use @Component. What was the result of the previous launch (COMPLETE or FAILED)? — Michael Minella
Do I need to scan the component package on my SpringBootApp class then? Anyway, the status was FAILED because I was only testing part of the tasklet, it shouldn't succeed. Does this affect the upcoming tasks? — Enrico Bergamo
Quick update. By deleting the registries from BATCH_JOB_EXECUTION_PARAMS I managed to run a new task with the correct params, but the upcoming tasks keep giving me the same behavior. — Enrico Bergamo
If your job is run with the same identifying parameters and the last attempt failed, it will attempt to restart the previous run by default — Michael Minella
Thanks for clearing this up for me! Is there any workaround to avoid this behavior or to alter it? By identifying parameters you mean the same parameters names, not their values, right? Is there any best practice on how to programmaticaly finish a task with the correct status to avoid this from happening? Thanks in advance! — Enrico Bergamo

Guilherme Chafy Guilherme Chafy · Accepted Answer · 2018-04-07T15:16:48

Hi Enrico I was the similarity problem, try it matbe it's works.

@Bean
@Qualifier("load")
public Job load(JobCompletionNotificationListener listener, Step step1, 
@Qualifier("stepValidation") Step stepValidation) {
    return jobBuilderFactory.get("load")
            .incrementer(new SampleIncrementer())
            .listener(listener)
            .flow(stepValidation)
            .next(step1)
            .end().build();
}

public class SampleIncrementer implements JobParametersIncrementer {

    public JobParameters getNext(JobParameters parameters) {
        if (parameters==null || parameters.isEmpty()) {
            return new JobParametersBuilder().addLong("run.id", 1L).toJobParameters();
        }

        long id = parameters.getLong("run.id",1L) + 1;
        return new JobParametersBuilder().addLong("run.id", id)
            .toJobParameters();
    }
}

Spring Data Flow overriding job parameters at launch

1 Answers