Spring Integration to Iterate through a list of files or records

Question

I am using spring integration to download files and to process them.

<int-sftp:inbound-channel-adapter channel="FileDownloadChannel"
                      session-factory="SftpSessionFactory"
                      remote-directory="/home/sshaji/from_disney/files"
                      filter = "modifiedFileListFilter"
                      local-directory="/home/sshaji/to_disney/downloads" 
                      auto-create-local-directory="true" >                       
       <integration:poller cron="*/10 * * * * *" default="true"/>   
</int-sftp:inbound-channel-adapter>

 <integration:transformer  input-channel="FileDownloadChannel"
                            ref="ErrorTransformer"
                            output-channel="EndChannel"/>

   <integration:router input-channel="FileErrorProcessingChannel"
                        expression="payload.getErrorCode() > 0">   
        <integration:mapping value="true" channel="ReportErrorChannel"/>
        <integration:mapping value="false" channel="FilesBackupChannel"/>
   </integration:router>

The int-sftp:inbound-channel-adapter is used to download files from sftp server. It downloads about 6 files. all xml files.

The transformer iterates all the 6 files and check whether they have an error tag. If there is an error tag then it will set its errorcode as 1. else it will be set a 0.

When it comes out of the transformer and goes to the router, i want to send the files whose errorcode is set to 1 to move to a specific folder (Error) and those which has errorcode set to 0 to move to another folder (NoError).

Currently the transformer returns a " list fileNames " which contains the errorcode and fileNames of all the 6 files.

How can i check the error code for each file using the router? and then map that particular file to a router.

Common C Logic for my problem

for (int i =0; i<fileNames.lenght();i++) {
     if(fileNames[i].getErrorCode == 1) {
     moveToErrorFolder(fileNames[i].getName());
     } else {
     moveToNoErrors(fileNames[i].getName());
     }
}

How can i achieve this using spring integration?. If its not possible, is there any workaround for it?. I hope now its clear. I am sorry for not providing enough details last time.

Also in the int-sftp:inbound-channel-adapter i have hard coded the "remote-directory" and "local-directory" fields to a specific folder in the system. can i refer these from a bean property or from a constant value?. I need to configure these values based on config.xml file, is that possible?.

I am new to Spring Integration. Please help me. Thanks in Advance.

Sorry, I can't understand what you are asking. Please provide many more details, as well as the SI configuration before the router. — Gary Russell
Thanks @GaryRussell for the reply, i have updated the post with more details. please have a look. Please ask me if you need more details. I appreciate the help. — Shamnad P S

Gary Russell Gary Russell · Accepted Answer · 2013-07-31T15:17:44

It's still not clear to me what you mean by "The transformer iterates all the 6 files".

Each file will be passed to the transformer in a single message, so I don't see how it can emit a list of 6.

It sounds like you need an <aggregator/> with a correlation-strategy-expression="'foo'" and release-strategy-expression="size() == 6". This would aggregate each single File into a list of File and pass it to your transformer. It then transforms it to a list of your status objects containing the filename and error code.

Finally, you would add a <splitter/> which would split the list into separate FileName messages to send to the router.

You can use normal Spring property placeholders for the directory attributes ${some.property} or SpEL to use a property of another bean #{someBean.remoteDir}.

Spring Integration to Iterate through a list of files or records

1 Answers