I have a spring batch job (v4.3.1,no xml configuration) to load a CSV file in a database. The jobs are started by a rest controller so not automatically at start. It all works fine when the filename is known at startup of the application.
But when I try to pass a jobParameter to the step (and annotate it with @StepScope) I get an exception at the start of the application (at that moment the job is not started yet). The exception I get is
Caused by: java.lang.IllegalArgumentException: Path must not be null
at org.springframework.util.Assert.notNull(Assert.java:201) ~[spring-core-5.3.2.jar!/:5.3.2]
at org.springframework.core.io.FileSystemResource.<init>(FileSystemResource.java:80) ~[spring-core-5.3.2.jar!/:5.3.2]
at TaskImportJobConfiguration.importTasksReader(TaskImportJobConfiguration.java:66) ~[classes!/:na]
at TaskImportJobConfiguration.step1(TaskImportJobConfiguration.java:109) ~[classes!/:na]
at TaskImportJobConfiguration$$EnhancerBySpringCGLIB$$b686b37a.CGLIB$step1$3(<generated>) ~[classes!/:na]
at TaskImportJobConfiguration$$EnhancerBySpringCGLIB$$b686b37a$$FastClassBySpringCGLIB$$7987bf2f.invoke(<generated>) ~[classes!/:na]
at org.springframework.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:244) ~[spring-core-5.3.2.jar!/:5.3.2]
at org.springframework.context.annotation.ConfigurationClassEnhancer$BeanMethodInterceptor.intercept(ConfigurationClassEnhancer.java:331) ~[spring-context-5.3.2.jar!/:5.3.2]
at TaskImportJobConfiguration$$EnhancerBySpringCGLIB$$b686b37a.step1(<generated>) ~[classes!/:na]
The step1() method calls .reader(importTasksReader(null)). This should work as the parameters are bound at runtime. But I notice this is not what is happening. At the moment the step1() method is called (so at startup of the application) the FileSystemResource passed to my ItemReader checks the parameter which is null at that time and thus throws an exception.
What am I doing wrong here ?
Here is my code:
import java.beans.PropertyEditor;
import java.beans.PropertyEditorSupport;
import java.time.LocalDate;
import java.time.format.DateTimeFormatter;
import java.util.HashMap;
import java.util.Map;
import javax.sql.DataSource;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.batch.core.Job;
import org.springframework.batch.core.Step;
import org.springframework.batch.core.configuration.annotation.JobBuilderFactory;
import org.springframework.batch.core.configuration.annotation.StepBuilderFactory;
import org.springframework.batch.core.configuration.annotation.StepScope;
import org.springframework.batch.core.launch.support.RunIdIncrementer;
import org.springframework.batch.item.database.BeanPropertyItemSqlParameterSourceProvider;
import org.springframework.batch.item.database.JdbcBatchItemWriter;
import org.springframework.batch.item.file.FlatFileItemReader;
import org.springframework.batch.item.file.builder.FlatFileItemReaderBuilder;
import org.springframework.batch.item.file.mapping.BeanWrapperFieldSetMapper;
import org.springframework.batch.item.support.PassThroughItemProcessor;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.core.io.FileSystemResource;
import org.springframework.jdbc.core.namedparam.NamedParameterJdbcTemplate;
@Configuration
public class TaskImportJobConfiguration {
private static final Logger logger = LoggerFactory.getLogger(TaskImportJobConfiguration.class);
private static final String QUERY_INSERT_TASK = "INSERT INTO TASK (N_I_IDF, C_I_TASK_CODE, C_TASK_TYPE_CODE) "
+ "VALUES (NEXTVAL FOR TASK_SEQ, concat('BC',varchar_format(NEXTVAL FOR TASK_SEQ,'000000000')), :taskType)";
@Autowired
private JobBuilderFactory jobBuilderFactory;
@Autowired
private StepBuilderFactory stepBuilderFactory;
@Bean
public Job importTasksJob(Step step1) {
return jobBuilderFactory.get("importTasksJob").incrementer(new RunIdIncrementer())
.flow(step1).end().build();
}
private static final DateTimeFormatter DATE_FORMATTER=DateTimeFormatter.ofPattern("dd/MM/yyyy");
@StepScope
public FlatFileItemReader<CsvTask> importTasksReader(@Value("#{jobParameters['inputfile']}") String inputFile) {
logger.info("IN-ImportTasksReader ("+inputFile+")");
return new FlatFileItemReaderBuilder<CsvTask>()
.name("taskItemReader")//
.resource(new FileSystemResource(inputFile))//
.linesToSkip(1)//
.delimited()//
.delimiter(";")
.names(new String[] { "taskType"})
.fieldSetMapper(new BeanWrapperFieldSetMapper<CsvTask>() {
{
setTargetType(CsvTask.class);
// 8< some custom editors >8
}
}).build();
}
@Bean
public JdbcBatchItemWriter<CsvTask> taskWriter(DataSource datasource, NamedParameterJdbcTemplate jdbcTemplate) {
JdbcBatchItemWriter<CsvTask> itemwriter=new JdbcBatchItemWriter<>();
itemwriter.setDataSource(datasource);
itemwriter.setJdbcTemplate(jdbcTemplate);
itemwriter.setSql(QUERY_INSERT_TASK);
itemwriter.setItemSqlParameterSourceProvider(new BeanPropertyItemSqlParameterSourceProvider<>());
return itemwriter;
}
@Bean
public Step step1(JdbcBatchItemWriter<CsvTask> taskWriter) {
return stepBuilderFactory //
.get("importTasksJob.step1")//
.<CsvTask, CsvTask>chunk(10) //
.reader(importTasksReader(null)) //
.processor(new PassThroughItemProcessor<CsvTask>())
.writer(taskWriter) //
.build();
}
}
UPDATE: when I add @bean annotation to importTasksReader like :
@Bean
@StepScope
public FlatFileItemReader<CsvTask> importTasksReader(@Value("#{jobParameters['inputfile']}") String inputFile) {
I get following exception:
2021-02-15 15:10:06.531 WARN 1 --- [ main] ConfigServletWebServerApplicationContext : Exception encountered during context initialization - cancelling refresh attempt: org.springframework.beans.factory.support.BeanDefinitionOverrideException: Invalid bean definition with name 'scopedTarget.importTasksReader' defined in BeanDefinition defined in class path resource [TaskImportJobConfiguration.class]:
Cannot register bean definition [Root bean: class [org.springframework.aop.scope.ScopedProxyFactoryBean]; scope=; abstract=false; lazyInit=null; autowireMode=0; dependencyCheck=0; autowireCandidate=false; primary=false; factoryBeanName=null; factoryMethodName=null; initMethodName=null; destroyMethodName=null; defined in BeanDefinition defined in class path resource [TaskImportJobConfiguration.class]] for bean 'scopedTarget.importTasksReader':
There is already [Root bean: class [null]; scope=step; abstract=false; lazyInit=null; autowireMode=3; dependencyCheck=0; autowireCandidate=false; primary=false; factoryBeanName=taskImportJobConfiguration; factoryMethodName=importTasksReader; initMethodName=null; destroyMethodName=(inferred); defined in class path resource [TaskImportJobConfiguration.class]] bound.
2021-02-15 15:10:06.577 ERROR 1 --- [ main] o.s.b.d.LoggingFailureAnalysisReporter :
***************************
APPLICATION FAILED TO START
***************************
Description:
The bean 'scopedTarget.importTasksReader', defined in BeanDefinition defined in class path resource [TaskImportJobConfiguration.class], could not be registered. A bean with that name has already been defined in class path resource [TaskImportJobConfiguration.class] and overriding is disabled.
Action:
Consider renaming one of the beans or enabling overriding by setting spring.main.allow-bean-definition-overriding=true
For completeness: my application @Configuration class also instantiates the StepScope:
@SpringBootApplication
@Configuration
@EnableBatchProcessing
public class Application {
@Bean
public static StepScope scope() {
return new StepScope();
}
// ... other stuff hidden here
}
