Consuming message in the correct order from an Advanced Queue by Camel and JMS

Question

I have a problem by using Apache Camel in combination with Oracle Advanced Queues and JMS.

It's about an application to distribute messages. Messages are received and enqueued in Oracle Advanced Queues with the help of Camel. Then they are consumed with Camel and forwarded to the target system. For the case that the message delivery fails, there is an retry count defined in the Advanced Queue, so that the message delivery is repeated.

If Camel now dequeue a message and sends it to a target system that is not available, either a HttpOperationFailedException or NoSuchEndpointException is thrown. These are caught and there is a rollback performed.

At this point, the expectation is that the message delivery will be retried as often as defined in the Retry Count and then moved to an exception queue. However, what is happening is that the next message in the queue is send.

Because the contents of the messages are partially dependent on each other, they must be processed sequentially.

I think that there is an misconfiguration in the usage of the JMS Library, but I'm not sure and have found nothing I could influence this behavior.

The used JMS library is Oracle AqApi v 11.2.0.3.

Here is the code for the Camel route:

from("jms-camel-component:myComponent.AQ?jmsMessageType=Text").routeId("deliveryToTarget")
        .transacted()                
        .setExchangePattern(ExchangePattern.InOut)                        
        .setHeader(Exchange.HTTP_QUERY, constant("throwExceptionOnFailure=false"))
        .setHeader(Exchange.CONTENT_TYPE, constant("application/xml; charset=UTF-8"))
        .doTry()
            .recipientList(header("endpointTarget"))
            .endDoTry()
            .process(ResponseProzessor.getInstance())
            .log("Message was delivererd.")
        .doCatch(HttpOperationFailedException.class, NoSuchEndpointException.class)    
            .process(ResponseProzessor.getInstance())
            .log("Error occured.")
            .rollback()
        .end();

Here is the JmsComponent configuration:

JmsComponent jmsComponent = new JmsComponent(scc);
jmsComponent.setConnectionFactory(connectionFactory);
jmsComponent.setTransactionManager(tm);
jmsComponent.setMaxConcurrentConsumers(1);            
jmsComponent.setMaxMessagesPerTask(1);                  
jmsComponent.setIncludeSentJMSMessageID(true);

Thank you in advance for your help!

UPDATE

I think, I've found the reason for the described behavior. There is a delay configured on the Advanced Queue. As long as the delay lasts, the next message from the queue is dequeued. The messages are not dequeued randomly, they are dequeued according to the priorities.

I really think this is something that has to be configured on the consumer. Is there any trick to configure the camel-jms-component to consume the first message from queue as long as it's not commited or moved to the exception queue? I didn't find an option to configure it directly on camel...

In JMS in general, message order is not guaranteed. In ActiveMQ, if you have only a single consumer, and client-side redelivery, you're pretty safe, but I don't know about Oracle AQ... — GeertPt
I would be interested in a sollution for Weblogic Application Servers if there is no universal sollution for this problem. — elfwyn
The (clean) solution to this is always server side as the client (should/) doesn't know the messages in the queue. Depending on your QoS you'd need to read all the messages in the queue to figure out which message you want first - which, depending on your options (configuration wise) could still be an option. I'd check my answer below / the configuration on the server side before you do implement this on the client side. HTH — PhilW
In hinsight the problem seems to have been a delay set on the Advanced Queue DB-side. This caused the next message to be consumed by the Camel Framework before the previous transaction was rolled back cleanly in case of an error. It would be nice though to know if that is the right thing to happen under these circumstances or an oversight by the Oracle AQ Implementation. — elfwyn
@PhilW: Does that mean that I have to put a message broker like Apache ActiveMQ between Camel and the database? As far as I understood, those message broker serve the same purpose as the Oracle Advanced Queues. It's not more than another queue. Is that correct? So isn't there one obsolete? — The Dodo

PhilW PhilW · Accepted Answer · 2015-09-23T11:51:05

I'm not a Oracle AQ expert, but as far as I can see this is a setting on the queue, not on the client side.

The sort_list parameter determines the order in which messages are dequeued. You cannot change the message sort order after you have created the queue table

from: http://docs.oracle.com/cd/B19306_01/server.102/b14257/aq_admin.htm

You most likely have ENQ_TIME or COMMIT_TIME set - which might already satisfy your needs.

And, of course, your consumer must be the only one.

Consuming message in the correct order from an Advanced Queue by Camel and JMS

1 Answers