Is there a way to determine if processor stuck when run with concurrentConsumers?

Question

At some point one someQueue's size is starting to grow up. The Messages are enqueued, but they are not deqeued. Such queue is consumed with <from uri="activemq:queue:someQueue?concurrentConsumers=5"/> and it seems that parallel Processors are working fine, because data seems to be processed - further routes are triggered.

I suppose that one of the Processor which works concurently somohow stuck. How to check what is the reason? How to check if it really stuck, without adding additional watch dogs/timer threads in Processor? How to check the processor working time? Is there way to find which message caused it?

After some time when queue is to big the route stop processing data, and no other paralell Processors are run.

What I used till now is just keeping the thread code safe, and displaying status with activemq:dstat. I also think about attaching the JPDA to the Karaf to see what is happaning insisde, but maybe there are other, simplier methods to find what is the problem.

<route id="someRoute" startupOrder="7">
        <from uri="activemq:queue:someQueue?concurrentConsumers=5"/>
        <process ref="someProcesor"/>  
        <choice>
            <when>
                <simple>${header.ProcesedSuccesfull}</simple>                    
                    <to uri="activemq:queue:otherQueue"/>                                                               
            </when>
            <otherwise>                    
                <log loggingLevel="ERROR" message="error" loggerRef="myLogger"/>
            </otherwise>
        </choice>
    </route>

Claus Ibsen Claus Ibsen · Accepted Answer · 2015-11-23T18:28:38

You can see the inflight repository, and as well the await manager mbean [1]. The latter can tell you how many threads are blocked waiting for some external condition to trigger before continuing.

You can use hawtio [2] web console to see that from a web browser, and then trigger a stuck thread to unblock and continue and reject/fail the messaging, but that would leave the thread to continue running in your use-case so it can pickup new messages.

Is there a way to determine if processor stuck when run with concurrentConsumers?

1 Answers