AWS API Gateway POST Request for daily data load

Question

I am someone who is totally new to REST APIs, pardon the newbie-ish mistakes.

My requirement is:

The source Database people wants to send JSON data on an hourly basis to an API endpoint which I publish. I am not sure of what all do I need to build to make sure it happens seamlessly. My target is to receive the data and create CSV files and save in it AWS S3 for further downstream processing.

My plan is, creating an AWS API Gateway endpoint which will accept POST requests and whenever anyone sends data through POST, the API Gateway will trigger AWS Lambda Function which will run Python to parse the JSON data to CSV and store in AWS S3. Is this thought valid? What all am I missing out? Are there best practices which needs to be implemented?

Answer 1

This architecture seems to be what you wanna do.

You wanna make sure that your API is secured with a key or via Cognito (more complex) and that your Lambda have the IAM permissions needed in order to access your bucket.

This post will help you understand the Lambda blueprint that is triggered when an object is upload to s3. Just change the Lambda trigger and a little bit the Python code and you're done.

Answer 2

Yes,this is a simple, typical serverless stack and it works perfectly fine.

Additionally, you may also focus on the authentication on the API Gateway end point to make it secure.