Pulling Files Periodically and Recursively via SFTP using Spring Integration

Question

I need to connect periodically to a remote location via SFTP and check for new files. If new files have arrived, I need to download them and rename them. I don't delete the remote files, and I only need to process them once.

So far so good.

I can do all that by using an SFTP Inbound Channel Adapter combined with a Poller and an SftpPersistentAcceptOnceFileListFilter.

The trick is I need to do this recursively, which, I believe, takes me away from the realm of the SFTP Inbound into the SFTP Outbound Gateway.

But, unless I'm missing something, this is not helpful. I need a periodic inbound operation, so I can't use a Poller with the SFTP Outbound Gateway. And ideally I don't want to care about the naming pattern of the remote directory structure.

To be honest, my knowledge of Spring Integration is not very deep, so hopefully I'm missing something obvious. And if it's needed I can extend the SFTP Inbound Channel Adapter to create what I need using JSch library (not sure even if this practical, but I'll be reading about this while I wait for responses to this post).

I just don't want to hack my way around it when there's a more elegant or more standard way to tackle this problem of mine.

Thanks.

Answer 1

I need a periodic inbound operation, so I can't use a Poller with the SFTP Outbound Gateway.

Simply poll a POJO inbound channel adapter that returns the directory you want to poll and send the message to the outbound gateway.

Docs here. It shows XML configuration but it can be done with Java configuration or the Java DSL too.