Spring Batch - how to launch job with file name received from rest API request

Question

I am trying to expose a Rest API and use it to receive a file name parameter and launch a Spring Batch job with it.

I want each request to launch the job and want my ItemReader to look for that file name.

I've read some other questions regarding how to launch a job via Rest API, and how the ItemReader can access jobParameters, but could not find a way to receive the parameters from the API request. Below is my code:

@Configuration
public class BatchConfig {

    @Autowired
    public JobBuilderFactory jobBuilderFactory;

    @Autowired
    public StepBuilderFactory stepBuilderFactory;

    private static final String WILL_BE_INJECTED = null;

    @Bean
    public Job processJob() {
        return jobBuilderFactory.get("myJobName")
                .incrementer(new RunIdIncrementer())
                .flow(step1())
                .end()
                .build();
    }

    @Bean
    public Step step1() {
        return stepBuilderFactory.get("step1")
                .<Foo, Foo> chunk(1)
                .reader(excelReader(WILL_BE_INJECTED))
                .processor(new Processor()).faultTolerant().skipPolicy(skip())
                .writer(new Writer())
                .build();
    }

    @Bean
    @StepScope
    ItemReader<Foo> excelReader(@Value("#{jobParameters['fileName']}") String fileName) {
        PoiItemReader<Foo> reader = new PoiItemReader<>();
        reader.setLinesToSkip(3);
        reader.setResource(new ClassPathResource("data/" + fileName));
        reader.setRowMapper(excelRowMapper());
        return reader;
    }

    private RowMapper<Foo> excelRowMapper() {
        return new FooExcelRowMapper();
    }

    @Bean
    public SkipPolicy skip() {
        return new SkipPolicies();
    }

}

And my Controller:

@RestController
public class BatchResource {

    private final JobLauncher jobLauncher;

    private final Job job;

    public BatchResource(JobLauncher jobLauncher, Job job) {
        this.jobLauncher = jobLauncher;
        this.job = job;
    }

    @PostMapping
    public void launchJob(@RequestParam(value = "file_name", required = true) final String fileName) throws Exception {
        final JobParametersBuilder jobParametersBuilder = new JobParametersBuilder();
        jobParametersBuilder.addString("fileName", fileName);

        final JobParameters jobParameters = jobParametersBuilder.toJobParameters();

        jobLauncher.run(job, jobParameters);
    }
}

Unfortunately, it doesn't look like the ItemReader is understanding the fileName to be read. The job runs "successfully" but does not actually read my input file.

Edit: I started a new project with the same intention and noticed that as soon as I use the @StepScope annotation, it stops working and does not work again, even if I remove the @StepScope annotation. Now I'm trying to figure out how to overcome this

Your config LGTM. You just need to make sure that the value of the file name you get from the API is either relative to the classpath or an absolute path and use the correct Resource implementation accordingly (ClassPathResource or FileSystemResource). — Mahmoud Ben Hassine
Actually, if I do not use the parameters received by REST API, and use a hard-coded file name, it works. So I believe this is not a matter of using the correct Resource implementation. — elxapinhon
In that case make sure that the resolved value coming from the rest call has the same base path as the value you hard coded. — Mahmoud Ben Hassine
I started a new project with the same intention and noticed that as soon as I use the @StepScope annotation, it stops working and does not work again, even if I remove the @StepScope annotation. Now I'm trying to figure out how to overcome this. — elxapinhon

elxapinhon elxapinhon · Accepted Answer · 2019-06-16T21:04:38

After some debugging, I noticed the ItemReader implementation I'm using to read excel files (https://github.com/spring-projects/spring-batch-extensions/tree/master/spring-batch-excel) could not read the excel file as soon as I start using @StepScope annotation.

So I found this question PoiItemReader with StepScope Annotation do not read Excel file answered by Niraj Sonawane.

Basically I had to change the return type from ItemReader<Foo> to PoiItemReader<Foo> below:

@Bean
@StepScope
PoiItemReader<Foo> excelReader(@Value("#{jobParameters['fileName']}") String fileName) {
    PoiItemReader<Foo> reader = new PoiItemReader<>();
    reader.setLinesToSkip(3);
    reader.setResource(new ClassPathResource("data/" + fileName));
    reader.setRowMapper(excelRowMapper());
    return reader;
}

Spring Batch - how to launch job with file name received from rest API request

2 Answers