AWS Lambda: Clarification on retrieving data from event object

7
votes

At the moment I think I have a foundational misunderstanding of how to extract/pass data from lambda functions & I'm seeking clarification

Example: Let's say I want to pass some data to a lambda function ie {"hello":"world"} and do so by passing data to that Lambda function by creating a REST endpoint in AWS API Gateway

At the moment I understand there are three ways to extract data:

1) event.queryStringParameters (makes sense)

ex. We can attach query parameters to the request URL: https://fakefakefake.execute-api.us-west-2.amazonaws.com/test/myapi?hello=world and in the lambda function:

const data = event.queryStringParameters.hello; // 'world'

2) event.body (makes sense & this is possible because of "Lambda Proxy Integration")

ex. If we attach the data in the body of the POST/PUT/etc request using Lambda Proxy Integration (ie forward all the data), we can access it via event.body & within the lambda function (but making sure to JSON.parse the event.body since Lambda Proxy Integration will pass through stringified JSON & not valid/"real" JSON):

const parsedBody = JSON.parse(event.body); // should wrap in try/catch
const data = parsedBody.hello; // 'world'

3) Directly on the event object (Unclear)

ex. This case is unclear at the moment-- pass data to the lambda function from a REST endpoint setup in API Gateway where it is then accessible directly from the event object?

const data = event.hello; // 'world'

What is an example of how to pass data "directly" on the event object in a Lambda function like in case #3? I THINK this case requires that I create a "mapping template" when setting up the API/Lambda but I'm still unclear.

For a simple Node script, case 2 appears to have the "overhead" of parsing event body from stringified JSON so that's an understandable downside, but in addition to how to do it why or when would Case 3 be a more desirable approach?

2
javascriptnode.jsamazon-web-servicesaws-lambdaaws-api-gateway

2 Answers

9
votes

Lambda is standalone service that doesn't need to be integrated with API Gateway. queryStringParameters, body, body mapping templates, all of this is specific not to Lambda, but to Lambda - API Gateway integration.

If you are using Lambda with other services then the data is usually passed directly via event object and there is not much of a reason to pass it in some other way.

For example, you can subscribe Lambda function to S3 bucket and use it to programatically process events such as file being uploaded to your bucket. In this case, information such as bucket name, object key, object data, metadata, ... will be passed directly via event object.

And, when using Lambda with API Gateway, why would you want to use body mapping templates to pass data to your Lambda function directly via event object? Because you can reuse that function much easier for other purposes (if viable in your scenario), because your Lambda function will have much simpler interface, instead of one that is tight to API Gateway integration.

For example, you can have a function that performs some calculations on passed in numbers which you can call via API Gateway as well as to call it directly from your application. It will be much easier to work with such function if it expects event.x and event.y instead of some event.queryStringParameter.x which might make zero sense outside of API Gateway.

7
votes

Building on insights from Matus, I now can better answer the rest of the questions from above:

  • "What is an example of how to pass data "directly" on the event object in a Lambda function like in case #3?"

When setting up API Gateway with Lambda to send-through specific pieces of data you need to use a Mapping Template which gets setup in AWS API Gateway.

That mapping template itself is written in the Velocity Template Language (VTL) from the Apache Foundation. Ex for attaching only the "hello" data so it is accessible as const data = event.hello; // world, ex:

    { 
      "hello": $input.params('$hello')
    }

Note: VTL is very powerful, the above is not a realistic example of usage but smallest amount of VTL just to communicate the idea, see here and here for more detail

Jumping through the interface to set that Mapping Template is a bit of an ordeal for a starter example so here's illustrated steps:

API Steps

Warning: Be sure to double-check what is actually displayed in the editor, since there might occasionally occur some funky/unexpected behavior on the Mapping Template when you change the dropdown. Sometimes you also reach unrecoverable issues involving the mapping template, it's best when starting out to go ahead and delete the method and restart from API Gateway.

  • "For a simple Node script, case 2 appears to have the "overhead" of parsing event body from stringified JSON so that's an understandable downside, but why or when would Case 3 be a more desirable approach?

It all depends on how your Lambda is receiving its data.

The key insight from Matus is that this is all ultimately an implementation detail. Lambdas in this contrived example (ie set up a REST API via AWS Gateway, send a GET/POST to that API and have lambda do something with the data) can retrieve data 3 ways:

(1) URL parameters const data = event.queryStringParameters.hello; // world

(2) request body const data = event.body.hello; // world (see note below)

(3) directly on the event object const data = event.hello; // world

Note on 2: This requires selecting (a) Lambda Proxy Integration instead of using a mapping template and in your code you'll need JSON.parse the event body before accessing the data, see this answer for further details

All this depends on what it is being fed from API Gateway. In this specific example, I'm talking about doing a REST request to pass data on an API Gateway endpoint that then is processed by Lambda-- but lots of other services/triggers can send data to a Lambda script for analysis.

Other helpful resources:

Ex. a toy Lambda function echos back whatever is POST'd:

let client = null; // Data outside function handler (ex a DB connection or an incrementing value) can change within the handler code below and persist between Lamba invocations so long as the container is still "warm", see more for details: https://docs.aws.amazon.com/lambda/latest/dg/running-lambda-code.html

exports.handler = async (event, context) => {
    let data = {};

    // Lambda Proxy Integration
    if (event && event.body) {
        try {
            data = JSON.parse(event.body);
        } catch(e) { 
            return {
                statusCode: 400,
                body: JSON.stringify({message: 'There was an error parsing the JSON data posted to this endpoint', error:e})
            }
        }
    }

    try {

        // Echo back the data or perform transformations, pass on to another service, etc
        const response = {
            statusCode: 200,
            body: JSON.stringify({message: 'Data submitted', data})
        };

        return response;

    } catch(e) {
        // Report errors related with connection, auth, DB write, etc
        return {
          statusCode: 409,
          body: JSON.stringify({message: 'There was some type of catastrophic error', error:e})
      }
   }
};