Invoke an aws lambda function n number of times (concurrently) with different parameters

Question

I would like to schedule an aws lambda function to execute every 5 minutes. I want to have 100 invocations of the lambda function (concurrently) but each invocation will pass a different parameter.

Example: consider the below lambda function which takes a number as an input.

def lambda_handler(event, context):
    number = event['number']
    # some logic

Every five minutes, I want this function to be called with lets say these numbers [1, 5, 6, 7, 8, 19, 20, ...]. The list can contain 1000 elements.

I have looked into cron expressions in scheduled CloudWatch events but that works for scheduling the execution every 5 minutes, how do i pass these 1000 elements to 1000 different invocations of the lambda function? Do I need to integrate some other AWS feature to achieve this?

What is about to have 2 different Lambdas? One is invoked through AWS CloudWatch Schule-Rule and responsible to invoke the second Lambda 1000x which is doing your desired job. (You could also put this logic into one Lambda) — MaiKaY

thun thun · Accepted Answer · 2018-03-28T17:48:54

I would recommend using a single 'controller' lambda running at 5 minute intervals (AWS events) that creates 1000 SNS messages. Each SNS message would have it's own unique parameter. The SNS topic would have a 'worker' lambda subscribed. It would automatically fire for each of the created messages. SNS and Lambda both have error handling to deal with any failures gracefully (retry).

Invoke an aws lambda function n number of times (concurrently) with different parameters

2 Answers