0
votes

I need to process input file bi-monthly using Spring batch. The file name is coming with timestamp so it's going to change in each run.

FileName : NY_Prov_mmddyyyy.csv

I am using below code to read the file, I am trying to give only directory name as there is a pattern in the fileName and the file name is not a fixed one. But, this is not working.

@Bean
    @StepScope
    // FlatfileItemReader to define file properties
    public FlatFileItemReader<Provider> providerItemReader(){
        // create FlatFileItemReader
        FlatFileItemReader<Provider> reader = new FlatFileItemReader<>();
        // skip header
        reader.setLinesToSkip(1);
        reader.setResource(new PathResource(localFileDownloadPath));
        DefaultLineMapper<Provider> customerLineMapper = new DefaultLineMapper<>();
        // tokenizer for delimited file
        DelimitedLineTokenizer tokenizer = new DelimitedLineTokenizer();
        customerLineMapper.setLineTokenizer(tokenizer);
        customerLineMapper.setFieldSetMapper(new ProviderFieldSetMapper());
        customerLineMapper.afterPropertiesSet();
        reader.setLineMapper(customerLineMapper);
        return reader ;
    }

application.properties :

local.file.download.path =C:/Users/Desktop/data/in/

Error :

Caused by: java.lang.IllegalStateException: Input resource must be readable (reader is in 'strict' mode): path [C:\Users\Desktop\data\in]

I also tried to use MultiResourceItemReader but there also, I am having issues as listed in this thread :

No resources to read error while using MultiResourceItemReader

UPDATE :

If I pass the parameter at the reader level then what should I pass at the Step level ?

  @Bean
    public Step step2(){
        // step 2 : Read csv files and dump it into a custom table
        return stepBuilderFactory.get("stepReadLoad")
                .<Provider, Provider>chunk(1000)
                .reader(providerItemReader()) <-- looking for parameter here
                .writer(providerJdbcBatchItemWriter())
                .build();
    }

Update 2 :

I am trying to use it with MultiResourceItemReader. I may not be passing the job parameters correctly.

application.properties :

local.input.file.path.pattern =file:C:/Users/Desktop/data/in/*.csv

Code :

@StepScope
   @Bean
    // MultiResourceItemReader to read multiple files sequentially
    public MultiResourceItemReader<Provider> providerMultiResourceItemReader(@Value("#{jobParameters[local.file.download.path.pattern]}") Resource[] resources) {
    //public MultiResourceItemReader<Provider> providerMultiResourceItemReader() {
            MultiResourceItemReader<Provider> multiResourceItemReader = new MultiResourceItemReader<>();
            multiResourceItemReader.setResources(resources);
            //multiResourceItemReader.setResources();
            multiResourceItemReader.setDelegate(providerItemReader());
            multiResourceItemReader.setStrict(true);
            return multiResourceItemReader;

Error :

Caused by: org.springframework.beans.NotReadablePropertyException: Invalid property 'local' of bean class [org.springframework.batch.core.scope.context.StepContext]: Bean property 'local' is not readable or has an invalid getter method: Does the return type of the getter match the parameter type of the setter?
    
1

1 Answers

0
votes

Since the file name is dynamic, you should pass it as a job parameter and inject in the reader at runtime using the late binding technique. Here is a quick example:

@StepScope
@Bean
public FlatFileItemReader<Foo> flatFileItemReader(@Value("#{jobParameters[input.file.name]}") String name) {
    return new FlatFileItemReaderBuilder<Foo>()
            .name("flatFileItemReader")
            .resource(new FileSystemResource(name))
            ...
}