At the moment I'm are pooling AWS SQS from our back-end and doing business logic once payload is received.
I would like to move this to AWS Lambda and start automating business logic via SQS/SNS.
As I can not subscribe to AWS SQS events, what is the best practice in implementing SQS pooling with Lambda (node.js)?
SQS doesn't really work well with Lambda since you cannot automatically trigger Lambda functions from SQS queues messages.
I would rather remove the SQS/SNS logic and go for a DynamoDB Streams based solution that would cover the queueing, archiving & Lambda triggering tasks natively: your producer puts messages in a DynamoDB table while your Lambda is triggered for any new entry with Streams (it's an AWS native mechanism)
Of course a Kinesis based solution may be considered as well.
It is possible to even simplify the whole polling process by using the built-in SQS event source for lambda.
Lambda will automatically scale out horizontally consume the messages in my queue. Lambda will try to consume the queue as quickly and effeciently as possible by maximizing concurrency within the bounds of each service. As the queue traffic fluctuates the Lambda service will scale the polling operations up and down based on the number of inflight messages.
see AWS Blog
