Why is Lambda throttled when invoking from SQS?

2
votes

I have an SQS queue that is used as an event source for a Lambda function. Due to DB connection limitations, I have set a maximum concurrency to 5 for the Lambda function.

Under normal circumstances, everything works fine, but when we need to make changes, we deliberately disable the SQS trigger. Messages start to back up in the SQS queue as expected.

When the trigger is re-enabled, 5 Lambda functions are instantiated, and start to process the messages in the queue, however I also see CloudWatch telling me that the Lambda is being throttled.

Please could somebody explain why this is happening? I expect the available Lambda functions to simply work through the backlog as fast as they can, and wouldn't expect to see throttling due to the queue.

1
amazon-web-servicesaws-lambdaamazon-sqs
Rampant speculation: The Lambda SQS pollers are reacting to the queue backlog, and increasing the worker count beyond the base 5. "The Lambda service monitors the number of inflight messages, and when it detects that this number is trending up, it will increase the polling frequency by 20 ReceiveMessage requests per minute and the function concurrency by 60 calls per minute. As long as the queue remains busy it will continue to scale until it hits the function concurrency limits." – Michael - sqlbot
I have an issue where the reserved concurrency count is 30. The Lambda listens to an SQS queue. I see high throttling in the Lambda. The question is why does throttling happen when messages can stay in the queue waiting to be processed ? – Amrith M

1 Answers

2
votes

This is expected behaviour.

"On reaching the concurrency limit associated with a function, any further invocation requests to that function are throttled, i.e. the invocation doesn't execute your function. Each throttled invocation increases the Amazon CloudWatch Throttles metric for the function"

https://docs.aws.amazon.com/lambda/latest/dg/concurrent-executions.html