Acknowledge message in SQS queue

6
votes

I am using Amazon SQS with Amazon SQS-JMS java library with Java EE 7. What I want to achieve is after receiving a message, depending on business logic of the application either confirm (consume) the message or resend it to the queue again and after 3 failed retries move it to DLQ.

I though about using CLIENT_Acknowledge mode in JMS and only acknowledging the messages that were successfully processed, but this is from their official documentation:

In this mode, when a message is acknowledged, all messages received before this message are implicitly acknowledged as well. For example, if 10 messages are received, and only the 10th message is acknowledged (in the order the messages are received), then all of the previous nine messages are also acknowledged.

This example also seems to confirm this: http://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/code-examples.html#example-synchronous-receiver-client-acknowledge-mode.

For me this is kind of a strange behavior and opposite what I would expect from a client_acknowledge. Is there a more elegant solution here than just manually sending message throughout the code to main SQS queue or DLQ depending on process status?

2
javaamazon-web-servicesjmsmessage-queueamazon-sqs
Why do you wish to resend a message to the queue? A message should either be consumed and deleted, or left alone. What do you wish to achieve by "acknowledging" a message? – John Rotenstein
@JohnRotenstein due to business logic when I first receive event B, than A, I want to return B to queue and processes it after processing event A. Also I would like to use retry count and move messages to DLQ after 3 fails , so returning message to queue makes sense. – Bojan Trajkovski
SQS can provide its own DLQ logic. Also, if you use an SQS FIFO queue, it can guarantee ordering. Given these built-in capabilities, I would suggest you do not re-process messages -- just pull them off, process them, then delete them. – John Rotenstein

2 Answers

3
votes

You can use:

UNORDERED_ACKNOWLEDGE

SQSSession.UNORDERED_ACKNOWLEDGE

Which comes from 'com.amazon.sqs.javamessaging;' and as it states in the documentation it is a variation of Client_Acknowledge which only acknowledges the message for which it is called.

 /**
 * Non standard acknowledge mode. This is a variation of CLIENT_ACKNOWLEDGE
 * where Clients need to remember to call acknowledge on message. Difference
 * is that calling acknowledge on a message only acknowledge the message
 * being called.
 */

dependency example: "com.amazonaws:amazon-sqs-java-messaging-lib:1.0.3"

3
votes

To handle this case you can use RedrivePolicy attribute for the DLQ that you created. Solution for this case can be:

  • Create a 2 sqs Qs say my_q and my_q_dl (latter one is for DLQ)
  • Set DLQ my_q_dl as the DLQ of my_q by using RedrivePolicy.
  • Here, care should be taken to specify deadLetterTargetArn and maxReceiveCount. This maxReceiveCount is the number of times you want to process any message without acknowledging before sending it to the DLQ. If you set maxReceiveCount=3 then, the msg will remain in my_q up to 3rd pull by the consumer with no ack. 2 cases here:
    • Normal case: msg gets deleted as soon as ack is received.
    • If no ack (msg delete) for that msg upto third time then the msg gets deleted from my_q and pushed to my_q_dl itself.

*RedrivePolicy - The string that includes the parameters for the deadletter queue functionality of the source queue.

deadLetterTargetArn - The Amazon Resource Name (ARN) of the dead-letter queue to which Amazon SQS moves messages after the value of maxReceiveCount is exceeded.

maxReceiveCount - The number of times a message is delivered to the source queue before being moved to the dead-letter queue.

Note The dead-letter queue of a FIFO queue must also be a FIFO queue. Similarly, the dead-letter queue of a standard queue must also be a standard queue.*