Discard duplicate message only if they are still queued with ActiveMQ Artemis and JBoss EAP 7.1

Question

We're using ActiveMQ Artemis on JBoss EAP 7.1.

We noticed that once a message with a specific _AMQ_DUPL_ID value is passed through the queue, if the message producer tries to send a message with the same _AMQ_DUPL_ID value to the same queue again it is discarded by the broker. However, our need is to discard duplicate messages only if they are still in queue.

Is there a way to achieve this goal?

We use the primary key from the database as _AMQ_DUPL_ID value. This is the code we use

public void sendMessage(final T msg, final String id) { 
   jmsTemplate.send(destination, new MessageCreator() { 
      @Override
      public Message createMessage(Session session) throws JMSException { 
         Message message = session.createObjectMessage(msg);
         message.setStringProperty("_AMQ_DUPL_ID", id);
         return message;
      }
   });
}

We're looking for a solution because we have a timer that every 30 seconds loads from DB all records with a specific value for status field and puts them into the JMS queue.

Consumers consume JMS messages, processes them, updates their status field, insert/update them into the DB and opens a websocket connection with another application that we don't control. Sometimes the consumer hangs on the websocket call and, consequently, it remains busy while the timer continues to fill the queue.

To solve this problem we thought that something like Artemis duplicate message detection would help. However, when the external app hangs our consumer we need our timer to be able to put the message on the queue again.

we use the DB unique id (primary key) as message-id and this is the code-snipped we use ` public void sendMessage(final T msg, final String id) { jmsTemplate.send(destination, new MessageCreator() { @Override public Message createMessage(Session session) throws JMSException { Message message = session.createObjectMessage(msg); message.setStringProperty("_AMQ_DUPL_ID", id); // messa.setJMSCorrelationID(id); return message; } }); } ` — mandrake_2

Justin Bertram Justin Bertram · Accepted Answer · 2020-02-24T17:35:37

The duplicate message detection on ActiveMQ Artemis is working as designed. Its goal is to avoid any chance that a consumer will receive a duplicate message which means that even though a message may no longer be in the queue (e.g. because it was consumed) any duplicate of that message should still be rejected.

What you're asking for here is akin to asking how you can insert multiple records with the same primary key into a database table. It simply can't because because the entire point of having a primary key is to avoid duplicate records.

I recommend you implement some kind of timeout for the websocket call otherwise your application will be negatively impacted by a resource you have no control over.

Aside from that you may be able to use a last-value queue using the primary key as the value for _AMQ_LVQ_NAME. This will guarantee that only 1 instance of the message will be in a queue at any point. Read the documentation for more details.

Discard duplicate message only if they are still queued with ActiveMQ Artemis and JBoss EAP 7.1

1 Answers