Spring Integration Streaming Inbound channel adapter in cluster environment

Question

I have a requirement to implement the sftp streaming inbound channel adapter in multi/cluster environment. I am not supposed to store the file in my local directory, i have stream the file and process it immediately. It's working very well in single instance, but If i try to run the poller in multiple nodes am running in to issues like duplicate processing, file not found for the 2nd node.

I tried implementing it with the propertiesmetadatastore as per the docs https://docs.spring.io/spring-integration/reference/html/redis.html#redis-metadata-store
datastore is getting created but still the issue was not yet resolved.

Could some one please help me on this issue.

Thanks

Answer 1

Well, not clear what's going on in your environment, but SftpPersistentAcceptOnceFileListFilter with any possible distributed MetadataStore is the solution for you. Right, RedisMetadataStore is one of them: all your cluster instances have to look into the same Redis server though.