Camel JMS Queue Polling and data recovery

0
votes

Hi I am new to Camel and have a design question related to JMS queues.

I am receiving set of data. These data have a reference date. These data are sent every 15 minutes by a batch process.

I have to process the received data and forward them to another route.

If a given data cannot be processed, I need to reprocess it. And I have to ensure it is processed before the next data set is processed.

So I was thinking of creating a JMS route to receive these data before processing. Then process it. Then send it to another queue.

FTP --> Process data rows (A) --> JMS Queue --> Processor (B) --> direct:call

If processor B fails I want the data to be processed before the next data set is sent by FTP. (because second data set may contain an update of the data of the first dataset)

So I was thinking using a queue, to make sure they are always processed in the order they are being received.

But my experience with JMS, without Camel, is that once the object is consumed from the queue it is not in the queue anymore. Is it also the case with Camel? In this case to I have to retry to process the data, or put them back in the queue?

This "recovery" part is not clear to me and I'd like to understand the patterns that do support this.

Many thanks for your help

Gilles

1
ftpapache-cameljmsrecovery

1 Answers

0
votes

This part "once the object is consumed from the queue it is not in the queue anymore." is not fully correct. Actually, when you are subscribing to the queue and getting a message you need to process it and send acknowledge back to the JMS broker. If acknowledge is successful then the message will be removed from the queue. But if acknowledge will be not successful or if your process will die and connection to the broker will break then the message will not be removed from the queue and will be passed to another consumer.

Often most of the JMS libraries are using mode when acknowledgement is sent right when message was received by consumer but you always have possibility to change this mode and send acknowledgement manually when your processing part will be finished successfully.

What about camel jms (http://camel.apache.org/jms.html) you can use endpoint option "acknowledgementModeName" which has some different possible values like:

  • AUTO_ACKNOWLEDGE (default) - acknowledgement will be sent right after corresponded "from" in your route
  • CLIENT_ACKNOWLEDGE - allows the application to control when the acknowledgment is sent and if there are no exceptions will be thrown during exchange processing then message will be acknowledged and removed from queue.