I have a route polling for files from an inbound folder (/tmp/inbound/) and moves them to completed after processing (/tmp/completed) or to an error folder in case of unexpected exceptions (/tmp/error). Files are copied to the inbound folder every 10 mins through a different FTP process.
We see that the camel file component picks up the files in chronological order even though new files are written every 10 mins, but at some point the polling starts for the next day's files even though I have previous day's files left unprocessed. This results in a backlog of data.