Spring Integration (SFTP): problem with downloading files from subdirectories

Question

I am using Spring Integration version 5.4.4 in a Spring Boot application. I need to get all the XML files from subdirectories under the "server_sftp" directory. For this I use SFTP Streaming Inbound Channel Adapter and SFTP Outbound Gateway with mget command. Unfortunately, the application only downloads files from the root directory ("server_sftp"), and does not download files from subdirectories.

Where do I go wrong?

@Bean
@InboundChannelAdapter(channel = "downloadXmlFileInputChannel", poller = @Poller(fixedDelay = "300000"))
public MessageSource<InputStream> sftpXmlFileMessageSource() {
    SftpStreamingMessageSource messageSource = new SftpStreamingMessageSource(template());
    messageSource.setRemoteDirectory("server_sftp/");
    return messageSource;
}

@Bean
public IntegrationFlow xmlFilesReadingFlow() {
    return IntegrationFlows
            .from(sftpXmlFileMessageSource(), e -> e.poller(Pollers.fixedDelay(Duration.ofSeconds(5))))
            .handle(Sftp
                    .outboundGateway(template(), Command.MGET, "'server_sftp/*'")
                    .options(Option.RECURSIVE)
                    .autoCreateLocalDirectory(true)
                    .localDirectoryExpression("'../webapps/event_report_app-1.0/xmlFilesLocalDirectory/' + #remoteDirectory")
                    .localFilenameExpression("#remoteFileName.replaceFirst('sftpSource', 'localTarget')"))
            .channel("downloadXmlFileOutputChannel")
            .get();
}

@Bean
public PollableChannel downloadXmlFileInputChannel() {
    return new QueueChannel();
}

@Bean
public DirectChannel downloadXmlFileOutputChannel() {
    return new DirectChannel();
}

directory structure on the sftp server

Gary Russell Gary Russell · Accepted Answer · 2021-06-07T13:46:15

Use an outbound gateway with a recursive MGET command to fetch the full tree.

Using the mget Command mget retrieves multiple remote files based on a pattern and supports the following options:

-P: Preserve the timestamps of the remote files.

-R: Retrieve the entire directory tree recursively.

-x: Throw an exception if no files match the pattern (otherwise, an empty list is returned).

-D: Delete each remote file after successful transfer. If the transfer is ignored, the remote file is not deleted, because the FileExistsMode is IGNORE and the local file already exists.

The message payload resulting from an mget operation is a List object (that is, a List of File objects, each representing a retrieved file).

Spring Integration (SFTP): problem with downloading files from subdirectories

1 Answers