FTP takes more times to consume message when the files are more- Mule ESB

Question

I have 30k to 50 K images present in FTP. Just i need to take the image and place in local folder. It is taking more time just to take 20 messages.

At the same time, if i keep some 1000 images in FTP and tried to connect ,all the 1000 images being processed so fast and placed in folder within 3 to 5 minutes. But if we have 30k images in FTP and when tried the same, just to process 30 images it take 5 minutes or more. Could any one please explain how can we achieve this fast to consume message.

I have tried to reduce the polling frequency.But no help.

    <flow name="testFlow">
    <ftp:inbound-endpoint host="internal" port="21" path="/item/images" user="test" password="test" pollingFrequency="500" responseTimeout="90000" doc:name="FTP"/>
    <logger message="***Logger**" level="INFO" doc:name="Logger"/>
    <file:outbound-endpoint path="C:\test1" responseTimeout="10000" doc:name="File"/>
</flow>

Any one if comes across the scenario, help will be great. Thanks.

Víctor Romero Víctor Romero · Accepted Answer · 2015-10-22T06:51:29

The ftp connector does an ls as the first thing to figure out what is in the server. If you keep that massive amount of files, of course the ls will take quite a while.

If you need to process necesarily that amount of files, try having a composite-source endpoint with different endpoints and splitting the files in directories at the source or if this is no possible just file a ticket to MuleSoft.

FTP takes more times to consume message when the files are more- Mule ESB

1 Answers