Apache Camel route hangs when file is deleted during processing

Question

The program i am trying to build, consumes files on a Windows file share using Camel 2.18.5 and other programs occasionally add/modify/delete files in the share folder.

Building a simple test route

        from("file:///C:/Temp/from?readLock=changed&readLockCheckInterval=10000&readLockTimeout=0")
            .routeId("SimpleFileRoute")
            .to("file:/C:/Temp/to");

then placing a file, test.txt, in the from folder, waiting for the route to make a test.txt.camelLock file and then manually deleting test.txt before the readLockCheckInterval has passed, will cause the route to hang indefinitely without throwing an exception or processing any other files.

"changed" seems to be the only appropriate readLock for the situation.

Is there any way to make the route throw an exception, clean up the lock file and continue consuming files?

Edit: The stack trace of the thread that seems to be the route is this:

Daemon Thread [Camel (camel-1) thread #2 - file:///C:/Temp/from] (Suspended)    
Thread.sleep(long) line: not available [native method]  
FileChangedExclusiveReadLockStrategy.sleep() line: 104  
FileChangedExclusiveReadLockStrategy.acquireExclusiveReadLock(GenericFileOperations<File>, GenericFile<File>, Exchange) line: 90    
GenericFileRenameProcessStrategy<T>(GenericFileProcessStrategySupport<T>).begin(GenericFileOperations<T>, GenericFileEndpoint<T>, Exchange, GenericFile<T>) line: 64    
GenericFileRenameProcessStrategy<T>.begin(GenericFileOperations<T>, GenericFileEndpoint<T>, Exchange, GenericFile<T>) line: 37  
FileConsumer(GenericFileConsumer<T>).processExchange(Exchange) line: 362    
FileConsumer(GenericFileConsumer<T>).processBatch(Queue<Object>) line: 223  
FileConsumer(GenericFileConsumer<T>).poll() line: 187   
FileConsumer(ScheduledPollConsumer).doRun() line: 174   
FileConsumer(ScheduledPollConsumer).run() line: 101 
Executors$RunnableAdapter<T>.call() line: 511   
ScheduledThreadPoolExecutor$ScheduledFutureTask<V>(FutureTask<V>).runAndReset() line: 308   
ScheduledThreadPoolExecutor$ScheduledFutureTask<V>.access$301(ScheduledThreadPoolExecutor$ScheduledFutureTask) line: 180    
ScheduledThreadPoolExecutor$ScheduledFutureTask<V>.run() line: 294  
RejectableScheduledThreadPoolExecutor(ThreadPoolExecutor).runWorker(ThreadPoolExecutor$Worker) line: 1142   
ThreadPoolExecutor$Worker.run() line: 617   
Thread.run() line: 745

Can you try with a newer Camel version. And are you saying that "change" is the only read lock you have tested that has a problem and causing it to "hang". And when it hangs can you try to take a thread dump and see where Camel is hanging? — Claus Ibsen
Thanks for asking. I am pretty bound to that Camel version, but an upgrade has been on my wish list for a long time - perhaps next year. "change" is the only read lock I have tested - the others seem to have other potential problems for me. I updated the question with the stack trace of a suspension of the route when experiencing the problem. — Jan Larsen
Ah okay your problem is that you have a read lock timeout of 0 which you should not have as then it will never timeout, so set that to some sensitive value, like 30-60 seconds or something since the check interval is 10 seconds. — Claus Ibsen

Claus Ibsen Claus Ibsen · Accepted Answer · 2018-12-20T22:26:26

Okay so the issue is that you use readLockTimeout=0 which then prevents the read lock to timeout if a lock on the file cannot be granted, for whatever reason. You should not really do this (bad idea) so set it to a higher timeout value.

On the other hand we can also improve camel-core to check if the file still exists when it checks for changed and if the file doesnt exists then it can exit the check also.

I have logged a ticket about this: https://issues.apache.org/jira/browse/CAMEL-13025

Apache Camel route hangs when file is deleted during processing

1 Answers