Got duplicate message While Aggregate: mule esb

Question

Facing problem while combining multiple files. I store two different files. And combine those two files into one. Some time i got duplicate message. Or some time one file dropped. Following is my flow

<flow name="CombineFiles" >
        <file:inbound-endpoint path="Custom" responseTimeout="10000" doc:name="File" moveToDirectory="BackUp"/>
        <file:file-to-string-transformer doc:name="File to String"/>
        <scripting:component doc:name="Groovy">
            <scripting:script engine="Groovy"><![CDATA[long now = new java.util.Date().getTime(); 
                long mod = now % 2000; 
                message.setCorrelationId(String.valueOf(now - mod));
                message.setCorrelationGroupSize(100);
                return message;
            ]]></scripting:script>
        </scripting:component> 

        <collection-aggregator  failOnTimeout="false" doc:name="Collection Aggregator" timeout="3000"/>
        <combine-collections-transformer doc:name="Combine Collections"/>
        <logger message="FINAL #[payload]" level="INFO" doc:name="Logger"/>
        <object-to-string-transformer doc:name="Object to String"/>
        <file:outbound-endpoint path="Result" outputPattern="result1" responseTimeout="10000" doc:name="File"/>

    </flow>

Some time i got message from file1 two times. or some time got only 1 file message. I want to combine 2 files. What is wrong with my flow???

Why are you setting the correlation group size to 100 if you want to combine 2 files? And also, I think the correlation id is not well set, because it only depends on the file arrival time. Regards, Marcos. — MarcosNC
In 2 sec no.of files come based on that i set correlation id and size. But If i want to integrate 2 files so what way i have to set correlation id ?? — user3855589
If you want to combine 2 files, the group size should be 2. If this value is greater the aggregator will continue waiting after the those 2 files arrived, in fact, the aggregator timeout is 3 seconds, and your correlation id mods by 2 seconds. However, I think the timestamp mod is not a good approach, because the files can arrive when the two seconds window has finished. — MarcosNC

Anirban Sen Chowdhary Anirban Sen Chowdhary · Accepted Answer · 2015-02-19T15:44:27

Just after <combine-collections-transformer/> put

 <set-payload value="#[message.payload[0]] #[message.payload[1]]" doc:name="Set Payload"/>

and check whether you issue got solved. Here what I am trying is to explicitly setting your message from collection aggregator ... But please not this may not work if there is more than 2 files .. but since your reuirement is to use 2 files here as input .. this will work fine

Got duplicate message While Aggregate: mule esb

1 Answers