Scaling of Lambda functions for SNS trigger

2
votes

I have a Lambda function that is triggered by an SNS topic. What would happen to the messages being published to the SNS topic if Lambda reaches its limit of maximum concurrent executions and is not able to scale further? For example, consider a situation where my SNS topic is receiving 1000 messages per second but Lambda is able to scale only up to processing 600 messages per second. From what I understand about SNS, it is a pub/sub mechanism and there can be no backlog in it (unlike SQS, Kinesis etc.). So what will happen to the extra 400 messages per second?

Also, how can I monitor if my Lambda is able to process at the rate at which SNS is receiving messages?

1
amazon-web-servicesaws-lambdaamazon-sns
Possible duplicate of How to limit the consuming rate from a topic? – Kannaiyan

1 Answers

2
votes

To answer your first question you need to understand the retry behavior of AWS Lambda. Please see the following quote out of the documentation.

Asynchronous invocation – Asynchronous events are queued before being used to invoke the Lambda function. If AWS Lambda is unable to fully process the event, it will automatically retry the invocation twice, with delays between retries. If you have specified a Dead Letter Queue for your function, then the failed event is sent to the specified Amazon SQS queue or Amazon SNS topic. If you don't specify a Dead Letter Queue (DLQ), which is not required and is the default setting, then the event will be discarded. For more information, see Dead Letter Queues.

To answer your second question:

You could use AWS CloudWatch.

There are two metrics interesting for you:

  • AWS/Lambda - Invocations
  • AWS/SNS - NumberOfMessagesPublished