AWS SNS message lifetime

6
votes

I have a use case where I need Amazon SNS to send a notification until my application (let's call it APP) has successfully received it, but the documentation says that the maximum lifetime of a message can be 1 hour.

Let's say that the APP crashes and it's not possible to get it live in 1 hour. I still need to somehow receive these messages.

There are multiple ways to implement it:

  1. APP polls from SQS. I do not like this option because it produces too much network traffic between APP and AWS.
  2. SNS sends a notification to both: APP and SQS. If APP is able to receive the message it will instantly remove it from the SQS. If the APP is not able to receive the message (crashed), it can load the messages from SQS on startup and clean the queue.
  3. AWS Lambda code as messaging service. If Lambda code fails it can push the message to SQS Dead Letter Queue, otherwise keeps the queue clean. Handling Lamba code updates is too much overhead, would be cool to solve this problem with pure AWS if possible.

The perfect solution would be to set endless timeout for SNS message, but looks like Amazon does not support it.

What do you think is the best solution to solve this problem? Have I missed something?

2
amazon-web-servicesaws-lambdaamazon-sqsamazon-sns
If you wish to rely on non-aws solution, you could use pubnub w/ message history (retention) plugin. – hjpotter92
Thanks hjpotter92. As it is highly secured solution, I prefer not to send any data to third party except Amazon. – Madis Pukkonen
Any update on this? – Chamin Wickramarathna

2 Answers

6
votes

One option might be to have SNS deliver messages to a Lambda that calls your app. If the Lambda can't deliver the message to your app then fail so that SNS will retry the Lambda. You can then configure your Lambda with a dead letter queue (SQS) so that if it fails too many times the message will go onto the queue. Finally you can have another Lambda running on a schedule that checks the dead letter queue and retries the Lambda invocation. It would just keep putting the message back onto the dead letter queue if it fails.

This way if your app is available the message would be delivered immediately. If the app isn't available then it would retry delivery later.

0
votes

I believe the easiest solution for you is to set up an SNS dead-letter queue to the SNS subscription that delivers messages to the App. More information:

https://aws.amazon.com/blogs/compute/designing-durable-serverless-apps-with-dlqs-for-amazon-sns-amazon-sqs-aws-lambda/