I created simple job which reads all files from folder(D:\\chunk
), does empty procesing and empty writing and I registered listener to remove file after processing.
On Windows(On Linux and MacOs it does not happen) mashine I experince following error:
2019-09-09 12:08:13.752 WARN 4028 --- [ main] c.b.m.b.RemovingListener : Failed to remove chunk 0b9a2623-b4c3-42b2-9acf-373a2d81007c.csv
java.nio.file.FileSystemException: D:\chunk\1.csv: The process cannot access the file because it is being used by another process.
at java.base/sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:92)
at java.base/sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:103)
at java.base/sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:108)
at java.base/sun.nio.fs.WindowsFileSystemProvider.implDelete(WindowsFileSystemProvider.java:270)
at java.base/sun.nio.fs.AbstractFileSystemProvider.deleteIfExists(AbstractFileSystemProvider.java:110)
at java.base/java.nio.file.Files.deleteIfExists(Files.java:1180)
at my.super.project.batch.RemovingListener.afterStep(RemovingListener.java:31)
at my.super.project.batch.RemovingListener$$FastClassBySpringCGLIB$$e695a1e2.invoke(<generated>)
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:750)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
at org.springframework.aop.support.DelegatingIntroductionInterceptor.doProceed(DelegatingIntroductionInterceptor.java:136)
at org.springframework.aop.support.DelegatingIntroductionInterceptor.invoke(DelegatingIntroductionInterceptor.java:124)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:689)
at my.super.project.batch.RemovingListener$$EnhancerBySpringCGLIB$$10d47ff9.afterStep(<generated>)
Let me show you source:
I start job like this(java 11 syntax):
Path location = Path.of("D:\\chunk");
var parameters = new JobParametersBuilder()
.addString("files.location", location.toUri().resolve("*").toString()).toJobParameters();
jobLauncher.run(job, parameters);
job configuration:
@Bean
public Job fileProcessingJob(
MultiResourcePartitioner partitioner,
Step slaveStep
) {
return jobBuilderFactory
.get("read-file-job")
.start(stepBuilderFactory.get("master-step")
.partitioner("processChunk", partitioner)
.step(slaveStep)
.build())
.build();
}
partitioner:
@Bean
@JobScope
public MultiResourcePartitioner filesPartitioner(
ResourcePatternResolver resolver,
@Value("#{jobParameters['files.location']}") String location
) throws IOException {
var partitioner = new MultiResourcePartitioner();
partitioner.setResources(resolver.getResources(location));
return partitioner;
}
slave step:
@Bean
public Step slaveStep(
FlatFileItemReader<String> reader,
RemovingListener listener
) {
return stepBuilderFactory.get("processChunk")
.<String, String>chunk(10)
.reader(reader)
.processor((Function<String, String>) s -> s) //empty
.writer(items -> { //empty
})
.listener(listener)
.build();
}
RemovingListener:
@StepScope
@Component
public class RemovingListener extends StepExecutionListenerSupport {
private final Resource resource;
public RemovingListener(@Value("#{stepExecutionContext['fileName']}") Resource resource) {
this.resource = resource;
}
@Override
public ExitStatus afterStep(@NonNull StepExecution stepExecution) {
try {
Files.deleteIfExists(resource.getFile().toPath());
} catch (IOException e) {
log.warn("Failed to remove chunk {}", resource.getFilename(), e);
}
return stepExecution.getExitStatus();
}
}
What is it going on?
Why does it happen ? how to fix it ?
Full source could be found here: https://github.com/gredwhite/spring-batch-hello-world/tree/master/src/main/java/spring/boot/hello/process_cannot_access
afterStep
. It is step scope to deal with those streams – Artem Bilan