Download a single file via FTP with Spring Integration

Question

I was reading through the Spring Integration Documentation thinking that a file download would be pretty simple to implement. Instead, the article provided me with many different components that seem to over-qualify my needs:

The FTP Inbound Channel Adapter is a special listener that will connect to the FTP server and will listen for the remote directory events (e.g., new file created) at which point it will initiate a file transfer.

The streaming inbound channel adapter produces message with payloads of type InputStream, allowing files to be fetched without writing to the local file system.

Let's say I have a SessionFactory declared as follows:

@Bean
public SessionFactory<FTPFile> ftpSessionFactory() {
    DefaultFtpSessionFactory sf = new DefaultFtpSessionFactory();
    sf.setHost("localhost");
    sf.setPort(20);
    sf.setUsername("foo");
    sf.setPassword("foo");
    return new CachingSessionFactory<>(sf);
}

How do I go from here to downloading a single file on a given URL?

Answer 1

You can use an FtpRemoteFileTemplate...

@SpringBootApplication
public class So44194256Application implements CommandLineRunner {

    public static void main(String[] args) {
        SpringApplication.run(So44194256Application.class, args);
    }

    @Bean
    public DefaultFtpSessionFactory ftpSessionFactory() {
        DefaultFtpSessionFactory sf = new DefaultFtpSessionFactory();
        sf.setHost("10.0.0.3");
        sf.setUsername("ftptest");
        sf.setPassword("ftptest");
        return sf;
    }

    @Bean
    public FtpRemoteFileTemplate template(DefaultFtpSessionFactory sf) {
        return new FtpRemoteFileTemplate(sf);
    }

    @Autowired
    private FtpRemoteFileTemplate template;

    @Override
    public void run(String... args) throws Exception {
        template.get("foo/bar.txt",
                inputStream -> FileCopyUtils.copy(inputStream, 
                      new FileOutputStream(new File("/tmp/bar.txt"))));
    }

}

Answer 2

following code block might be helpful

@Bean
public SessionFactory<LsEntry> sftpSessionFactory() {
    DefaultSftpSessionFactory factory = new DefaultSftpSessionFactory(true) {
        {
            setHost("localhost");
            setPort(20);
            setUser("foo");
            setPassword("foo");
            setAllowUnknownKeys(true);
        }
    };
    return new CachingSessionFactory<LsEntry>(factory);
}

@Bean
public SftpInboundFileSynchronizer sftpInboundFileSynchronizer() {
    SftpInboundFileSynchronizer fileSynchronizer = new SftpInboundFileSynchronizer(sftpSessionFactory()) {
        {
            setDeleteRemoteFiles(true);
            setRemoteDirectory("/remote");
            setFilter(new SftpSimplePatternFileListFilter("*.txt"));
        }
    };
    return fileSynchronizer;
}

@Bean
@InboundChannelAdapter(channel = "sftpChannel", poller = @Poller(fixedDelay = "600"))
public MessageSource<File> sftpMessageSource() {
    SftpInboundFileSynchronizingMessageSource messageSource = new SftpInboundFileSynchronizingMessageSource(
            sftpInboundFileSynchronizer()) {
        {
            setLocalDirectory(new File("/temp"));
            setAutoCreateLocalDirectory(true);
            setLocalFilter(new AcceptOnceFileListFilter<File>());
        }
    };
    return messageSource;
}

obtained from https://github.com/edtoktay/spring-integraton

Answer 3

To add to @garyrussell's answer:

In FTPS protocol, if you are behind a firewall, you will might encounter Host attempting data connection x.x.x.x is not the same as server y.y.y.y error (as described here). The reason is the FtpSession instance returned from DefaultFtpsSessionFactory by default does remote verification test, i.e. it runs in an "active" mode.

The solution is to disable this verification on the FtpSession instance by setting the "passive mode", when you create the DefaultFtpsSessionFactory.

DefaultFtpsSessionFactory defaultFtpsSessionFactory() {

        DefaultFtpsSessionFactory defaultFtpSessionFactory = new DefaultFtpsSessionFactory(){
            @Override
            public FtpSession getSession() {
                FtpSession ftpSession = super.getSession();
                ftpSession.getClientInstance().setRemoteVerificationEnabled(false);
                return ftpSession;
            }
        };
        defaultFtpSessionFactory.setHost("host");
        defaultFtpSessionFactory.setPort(xx);
        defaultFtpSessionFactory.setUsername("username");
        defaultFtpSessionFactory.setPassword("password");
        defaultFtpSessionFactory.setFileType(2); //binary data transfer

        return defaultFtpSessionFactory;
    }