Mule collection splitter asynchronus flow

Question

I had used for loop for a db data to upload to cloud, but it is sequential. I was going through the collection splitter link,according to docs given it should process the data asynchronously. Purpose of using is I have huge set of data to be uploaded, collection splitter with VM helps for me according to the document.

1)Is there any different way of handling this?

2) If I make VM as request/response instead of on-way, is my process going to become sequential instead concurrent as described in the docs:

There is, however, one key difference: each fraction of the message will be processed simultaneously rather than in sequence. If you deploy your app to a cluster of servers this will have a big effect on performance.

but when it is request/response, it is waiting for the response, when it is waiting for the response is another process starts or it waits to complete earlier thread to finish?

If I make process as one-way and use resequencer and collection aggregator as described in the document, this steps executes before finishing my db status update. Flow:

<flow name="psi2sfdc-VM">
  <vm:inbound-endpoint exchange-pattern="one-way" path="uploadSfdc" connector-ref="VM" doc:name="UploadSFDC"/>
    <choice doc:name="Choice">
         <when expression="#[flowVars.sfdcId!=null]">        
            <db:update config-ref="PSI_Database_Configuration" doc:name="Processed">
                <db:parameterized-query><![CDATA[UPDATE client SET status = 'done' WHERE client_id = #[flowVars.clientId]]]></db:parameterized-query>
            </db:update>              
          <logger level=INFO, message="Db Update"/>
         </when>
         <otherwise>
             <logger message="else block" level="INFO" doc:name="Logger"/>                
         </otherwise>
     </choice> 
     <resequencer failOnTimeout="false" doc:name="Resequencer"/>
     <collection-aggregator failOnTimeout="false" doc:name="Collection Aggregator"/>
     <logger level=INFO, message="Oncomplete"/>
</flow>

Here on completes prints before db update logger message, this is due to async behavior, but collection aggregator should aggregate once all process completes.

I did not understand the last 2 statements "if I make process as one-way ....", could you elaborate please — Sudarshan
How are you concluding that the resequencer is called even before the update is completed ? — Sudarshan
@Sudarshan, I have tested. Since it is one-way, it will not wait for the response. If I make request-response, it goes in a sequence. — AKB

Sudarshan Sudarshan · Accepted Answer · 2015-06-24T03:14:11

I will try to answer your questions.

1) The approach you have mentioned is the most optimum according to me. Splitting a large piece of work into smaller subsets, where each subset can be processed on a different node (potentially), no other pattern mule will get you there. So bingo nice work

2) When VM is request/response it internally does not use a queue, I found that out rather painfully and you could find the story here, so that by itself will slow things down compared to one-way.

This bit from your question, I don't fully understand,

but when it is request/response, it is waiting for the response, when it is waiting for the response is another process starts or it waits to complete earlier thread to finish?

whether or not the next flow (the one with VM inbound) is executed in a new thread, this current thread is still blocked and will not proceed till it gets the response back.

To conclude for your case, you stick with one-way endpoints.

Mule collection splitter asynchronus flow

1 Answers