How keep messages on SQS after triggering lambda

2
votes

In my application we are using a SQS to queue messages to be processed by another module. SQS doesn't send notification that a message has come and I don't want to make my application to go to check on it every "X times". So I'm trying to use a lambda trigger to make a http request to my module and make it pool messages from SQS when a message got there.

The problem is SQS deletes the sent messages if there is no error on the lambda function (as far I know). Forcing an error just to keep the messages on the pool can't be right. So I need a way to keep messages on the SQS after the lambda was triggered.

Maybe I should move the code that process the message to the lambda function, but I'm looking for ways to keep it there.

Anyone could give some guidance?

Thanks in advance

1
amazon-web-servicesaws-lambdaamazon-sqs
It's a queue and queues have certain behaviors. If you want to explicitly re-queue a message so that you can process it later, then you can do that, with an optional delay. Also note that Lambda can process SQS messages in batches. - jarmod

1 Answers

3
votes

SQS is built to be a single producer to consumer for its queues so the intended functionality is happening.

However, there is a solution available for this exact scenario but it will require you to update your architecture.

The solution is to use a fanout architecture.

fanout

You would instead publish to an SNS topic, which has your SQS queue subscribed to it. Then create additional SQS queues for parallel channels (1 per each unique Lambda).

Add each Lambda function as consumer of its own SQS queue, each with their own processing.