Why messages sometimes get lost using Scatter/Gather with default threading profile?

Question

I am using reliable delivery in mule flow. It is very simple case that takes message from JMS queue (ActiveMQ based), invokes several actions depending on it's content and, if everything is fine - delivers it into another JMS queue.

A flow is synchronized, both JMS queues are transactional (first BEGINS, second JOINS transaction), redelivery is used and DLQ for undelivered messages. Literally: I expect that all messages are properly either processed or delivered to DLQ.

For processing orchestration I am using Scatter/Gather flow control which works quite fine until I call external HTTP service using HTTP connector. When I use default threading profile it happens, that some messages are lost (like 3 of 5000 messages). They just disappear. No trace even in DLQ.

On the other hand, when I use custom profile (not utilizing thread) - all messages are getting processed without any problems.

What I have noticed is the fact, default threading profile utilizes 'ScatterGatherWorkManager', while custom uses 'ActiveMQ Session Task' threads.

So my question is: what is the possible cause of loosing these messages?

I am using Mule Server 3.6.1 CE Runtime.

It is unclear why you would be using messages. IMO it is best to create, test and share a minimal example that replicates this. Just some notes: 1) remember that HTTP is not transactional (only JDBC, JMS, VM are in Mule), 2) consider using until-successful around the HTTP requester as a means of controlling failures. — Gabriel Dimech

Utsav Utsav · Accepted Answer · 2015-10-27T15:36:03

by default scatter gather is setup for no failed routes you can define your own aggregation strategy to handle lost message

custom-aggregation-strategy https://docs.mulesoft.com/mule-user-guide/v/3.6/scatter-gather

Why messages sometimes get lost using Scatter/Gather with default threading profile?

1 Answers