I'm using Apache Camel to interact with several SFTP endpoints; for each one, I perform the following pipeline:
- retrieve the list of existing files
- validate those files against a given set of rules
- download remote files, in case of successful validation
Everything works like a charm (for about a hundred different endpoints) and the URI used to retrieve the list of files is something like that: sftp://${HOST}:${PORT}/${DIR}?username=${USER}&download=false&recursive=true&disconnect=true&sendEmptyMessageWhenIdle=true
The problem is that, for one of those SFTP endpoints, the SFTP Camel component behaves, alternatively, as follows:
- immediately return 0 remote files
- takes a couple of minutes to list the remote content (which is composed by around 250 files, from 2KB to 2MB each)
In addition, in the latter case, the download takes around 30 seconds to download only 10KB of data.
Since this is happening on this specific SFTP only, I suppose it doesn't directly depend on Camel, which works fine for all other endpoints.
So, my questions are:
- what can affect such a connection, leading to an unreasonable delay (there are no network issues, nor huge data to fetch)?
- supposing it depends on the remote SFTP endpoint, why should the aforementioned Camel URI immediately return 0 files, since lots of files exist in the SFTP?
Thanks for any feedback.