Spring Integration SFTP: removing or noving multiple files

Question

With Spring Integration I want to move or remove multiple files or non-empty folders at once on a remote SFTP server. But I can't seem to find support for this in the official Spring docs, as it seems that this is unsupported. Although the documentation isn't always correct anyway.

I was thinking by using the int-sftp:outbound-gateway with the rm command with a payload the directory name. But it doesn't seem to work. I haven't tried it with mv yet, but I'm wondering if anyone has any experience with this behaviour in Spring Integration.

Answer 1

It's not very clear from your question : are the files you want to remove local to your application or remote, on the SFTP server ?

Below is an example of what I have in one my app, maybe that can help : incoming messages (with file name in payload) are first sent to remote SFTP server, and then are deleted locally

<integration:publish-subscribe-channel
    id="sftpChannel" />

<!-- The processed file will be sftped to the server -->
<sftp:outbound-channel-adapter id="sftpOutboundAdapter"
    session-factory="sftpSessionFactory" channel="sftpChannel" order="1"
    charset="UTF-8" remote-file-separator="/" remote-directory="${sftp.remote.directory}"
    remote-filename-generator-expression="payload.getName()" mode="REPLACE" />

<!-- sftped file will be removed from the staging folder -->
<integration:service-activator
    input-channel="sftpChannel" output-channel="nullChannel" ref="sftpFileDeleter"
    method="deleteAfterSftpingFile" order="2" />

with SftpFileDeleter being

public class SftpFileDeleter {

private static final Logger LOGGER = Logger
        .getLogger(SftpFileDeleter.class);

@ServiceActivator
public void deleteAfterSftpingFile(Message<File> fileMessage) throws IOException{
    Path fileToDeletePath = Paths.get(fileMessage.getPayload().getAbsolutePath());
    Files.delete(fileToDeletePath);

    LOGGER.info("[SENT]File Sent to Sftp Server and deleted:"+fileToDeletePath.getFileName());


}

}

Answer 2

Did you have a look at this example ?

https://github.com/spring-projects/spring-integration-samples/blob/master/basic/sftp/src/test/resources/META-INF/spring/integration/SftpOutboundGatewaySample-context.xml

It looks like that's exactly what you want to do. One thing you may do wrong though, is that according to the docs (http://docs.spring.io/spring-integration/reference/html/sftp.html , section 26.7), payload of your incoming message doesn't have to contain the filename : you need to be careful with the headers though, and set correct properties (file_remoteDirectory / file_remoteFile in your case) with proper values.

I don't know your current config, but you would probably need a message transformer just before the SFTP Outbound Gateway, to move the information from the payload to the header of the message.

Answer 3

I solved it with writing my own code on top of Spring, as @Vincent_F suggested. First of all you have to autowire an SFTP session factory like so:

@Autowired
private DefaultSftpSessionFactory sftpSessionFactory;

After which the session can be used to, in my case, rename the directory. This should probably also work with the default Spring DSL or XML, but I couldn't get it to work... Here is my own code relevant to this use case:

SftpSession session = sftpSessionFactory.getSession();

try {
    if (!session.exists(sftpConfiguration.getOtherRemoteDirectory())) {
        throw new FileNotFoundException("Remote directory does not exists... Continuing");
    }

    for (ChannelSftp.LsEntry entry : session.list(sftpConfiguration.getRemoteDirectory())) {
        if (entry.getFilename().equalsIgnoreCase(sftpConfiguration.getOtherDirectory())) {
            session.rename(sftpConfiguration.getOtherRemoteDirectory(), String.format("%s%s-%s",
                    sftpConfiguration.getRemoteDirectory(), sftpConfiguration.getReportDirectory(),
                    this.createDate(new Integer(entry.getAttrs().getMTime()).longValue() * 1000)));
        }
    }
} catch (FileNotFoundException e) {
    logger.error(e.getMessage());
} catch (IOException e) {
    logger.error("Could not rename remote directory.", e);
}