How to send messages from Mule to ActiveMQ in batches

1
votes

In mule I have a flow that receive an InputStream with a 500mb xml file.

I want to split the xml file following certain rules.

The result is 390000 messages that I need to send to an ActiveMQ queue.

ActiveMQ will give an outofmemory exception if I send all messages in one transaction.

If I don't use transaction it will succeed but it will be much slower.

What's the best way to send the messages in batches of 1000?

Can I use standard components?

I am using ActiveMQ 5.13 and Mule 3.7

Thanks

1
javamuleactivemq

1 Answers

1
votes

I think what you are looking for batch commits doc here. This component will execute the internal flow components once he collected the specified numbers of records. Example:

<batch:commit size="100" doc:name="Batch Commit">
    <!-- Put here all the message processor you want for example the active mq one -->
</batch:commit>

Please note that transaction are allowed only at step level and therefore also in the commit block that I think this is exactly what are you looking for.

Footnote: Batch commit processor is allowed only inside a batch step.

Hope this helps

Regards