We are using spring integration sftp inbound channel adapter which poll every few seconds and downloads the zip file to local dir for further processing. Problem starts when there is a big file where client is still uploading that file and this inbound adapter picks up that incomplete file. We are using AcceptAllFileFilter for remote filtering and for local we have custom filter.
Is there a better way to ignore or check if file is completely uploaded and then pick up for processing?
This is a classic problem with (s)ftp - fetching partial files.
There are two common solutions...
foo.done when foo.zip is complete).Spring Integration (on the writing side) uses 1) (writes the file with foo.zip.writing and renames to foo.zip). This is the simplest mechanism because it's easy to filter .writing files - Spring Integration does this automatically.
2) is a little more difficult because you need to write a custom FileListFilter to ignore foo.zip if foo.done is not present.
The underlying (S)FTP protocols have no way to prevent fetching partial files without using a mechanism like these.
EDIT:
If you have no control over the sender you could write a custom filter (FileListFilter) that will only fetch a file if it hasn't changed since some time.
This is not a Spring Integration issue it's a problem that has to be solved for any application using (S)FTP. Spring Integration provides one solution, and all the hooks you need to solve it for your situation.
