AWS Lambda as a social media post scheduler? Single function with unique data per scheduled event trigger?

Question

I would like to use AWS Lambda as a social media post scheduler, but I can't find an elegant way to do so. In our app, users create social media posts and set a time. We then post them via the social network's API at the time specified.

I need to be able to schedule a Lambda to run once at a scheduled time and with unique data (being the user's token and the body of the post) in order to accomplish it. Here's an example:

John wants to post to Twitter next Thursday at 2pm. He's scheduled a post with the body "Hello world!" for that time via our web app. The app will talk to AWS Lambda via the API and set a Lambda function to fire one time next Thursday at 2pm. That function would fire a request to the Twitter API with John's token and the body ("Hello world!").

Would love to be able to do this serverless with Lambda, but I can't find a great way. If you could pair a Cloudwatch scheduled event trigger with a unique payload, that might work, but I don't see that it's possible. Otherwise, it seems this would require creating a new Lambda function for each post with the data hard-coded or having the Lambda hit the database to look for the scheduled post. Creating potentially hundreds of bespoke Lambda functions seems like a huge mess, and hitting the database at Lambda runtime seems like undue stress on the database since we have all the data we need in-hand at the time we schedule.

Any suggestions for how I might accomplish this with Lambda? Is there another AWS service that is better suited to the task? Should I give up on serverless and just set up another EC2 instance to handle the scheduler?

Answer 1

You definitely don't want to be creating a function + event per scheduled task.

The scalable way to do this would be to schedule a single function to run regularly (e.g. hourly) and check a database to see if any posts where scheduled for the last hour (i.e. since the last run), and perform them if so.

The reason I am suggesting a database is because you need to manage your state (that is, the post payload/details) somewhere, and relying on CloudWatch Events for this is not the right way, for all the reasons you've listed in your question.

An alternative to a database would be to put the payload in S3, and have the scheduled function check a specific location/bucket for the payloads that need processing. Lambda to S3 communication is very fast, and you don't need to worry about load or network transfers.

Answer 2

You could use AWS Step Functions for this task. With these you can model a state machine which waits for the exact timestamp to trigger.

https://aws.amazon.com/step-functions/

The only drawback of those is, that documentation is still pretty scarce, but if you log into the AWS console, they provide some samples how to implement those wait processes.