Spring Batch: Job can't be started with different JobParameters and JobParameters can't be accessed

Question

I have to issues with Spring Batch. Both regarding the JobParameters that are passed in via the command line.

First issue:

I'm using Eclipse to develop my application and test it. Therefore, I added Program arguments to the Run Configurations. These arguments are:

-ts=${current_date} -path="file.csv"

Running the application will throw an exception. The exception is:

Caused by: org.springframework.batch.core.repository.JobInstanceAlreadyCompleteException: 
A job instance already exists and is complete for parameters={ts=20210211_1631, path=file.csv}.  
If you want to run this job again, change the parameters.

As you can see the JobParameters should be different for each execution, because one of the parameters is a timestamp that is changing each minute. I had a look at this question Spring Batch: execute same job with different parameters, but here the solution is to set a new name for each job execution (e.g. name + System.currentTimeMillis()). Is there another solution to this problem? I don't want to create a 'random' name for the job each time it is executed. My Job is implemented as this:

@Bean(name = "inJob")
public Job inJob(JobRepository jobRepository) {
    return jobBuilderFactory.get("inJob")
            .repository(jobRepository)
            .incrementer(new RunIdIncrementer())
            .start(truncateTable())
            .next(loadCsv())
            .next(updateType())
            .build();
}

I'm using a custom implementation of the JobRepository to store the metadata in a different database schema:

@Override
public JobRepository createJobRepository() throws Exception {
    JobRepositoryFactoryBean factory = new JobRepositoryFactoryBean();
    factory.setDataSource(dataSource);
    factory.setTransactionManager(transactionManager);
    factory.setTablePrefix("logging.BATCH_");
    return factory.getObject();
}

Second issue:

My second issue is accessing the JobParameters. One of the above parameters is a file path I want to use in the FlatFileItemReader:

@Bean(name = "inReader")
@StepScope
public FlatFileItemReader<CsvInfile> inReader() {       
    FlatFileItemReader<CsvInfile> reader = new FlatFileItemReader<CsvInfile>();
    reader.setResource(new FileSystemResource(path));
    DefaultLineMapper<CsvInfile> lineMapper = new DefaultLineMapper<>();
    lineMapper.setFieldSetMapper(new CsvInfileFieldMapper());
    DelimitedLineTokenizer tokenizer = new DelimitedLineTokenizer();
    tokenizer.setDelimiter("|");
    tokenizer.setNames(ccn.names);
    lineMapper.setLineTokenizer(tokenizer);
    reader.setLineMapper(lineMapper);
    reader.setLinesToSkip(1);
    reader.open(new ExecutionContext());
    return reader;
}

To get the path from the JobParameters I used the BeforeStep annotation to load the JobParameters and copy them on local variables. Unfortunately this is not working. The variable will be null and the execution fails, because the file can't be opened.

private String path;

@BeforeStep
public void beforeStep(StepExecution stepExecution) {
    JobParameters jobParameters = stepExecution.getJobParameters();
    this.path = jobParameters.getString("path");
}

How can I access the JobParameters within my reader? I want to pass in the file path as command line argument and then read this file.

Mahmoud Ben Hassine Mahmoud Ben Hassine · Accepted Answer · 2021-02-11T19:10:42

First issue: Is there another solution to this problem?

You current date is resolved per minute, so if you run your job more than one time during that minute, there would be already an job instance with the same parameters, hence the issue. Your ts parameter should have a precision of a second (or less if needed).

Second issue: How can I access the JobParameters within my reader? I want to pass in the file path as command line argument and then read this file.

You don't need that beforeStep method. You can late-bind the job parameter in your bean definition as follows:

@Bean(name = "inReader")
@StepScope
public FlatFileItemReader<CsvInfile> inReader(@Value("#{jobParameters['path']}") String path) {       
    FlatFileItemReader<CsvInfile> reader = new FlatFileItemReader<CsvInfile>();
    reader.setResource(new FileSystemResource(path));
    // ...
    return reader;
}

This would inject the file path in your reader definition if you pass path as a job parameter, something like:

java -jar myjob.jar path=/absolute/path/to/your/file

This is explained in the Late Binding of Job and Step Attributes section of the reference documentation.

Spring Batch: Job can't be started with different JobParameters and JobParameters can't be accessed

1 Answers