AWS Lambda continue execution after callback

Question

I'm using API Gateway + AWS Lambda to handle Slack API requests. Slack requires that the Lambda functions respond with a 2XX within 3 seconds so I need to respond quickly and then trigger some long-running db operations. The problem is that AWS Lambda either (1) waits until the event loop is empty before returning the response to API Gateway or (2) returns the response immediately when callback function is called, but it then immediately freezes the Lambda execution and the long-running operations with it.

Reading this and other posts, it seems like the usual way to do this is to have 2 lambdas: one that sends the 2XX response immediately and then triggers another lambda to carry out the long-running operations. That seems fine but I'm curious whether that can be done with just one lambda invocation?

Option #1 from this excellent answer by Mark B seems to suggest it is possible with an AWS Service integration type. But following these instructions to set up an AWS Service integration and setting the invocation type header to 'Event', I am still seeing that the Lambda execution gets frozen immediately after the callback is run.

My little test handler looks like this. I suspect Cloudwatch does not log after the callback is run so I have the handler make a request out to ngrok after a 5 sec timeout. I expect the callback response immediately and then 5 seconds later, I receive a GET request via ngrok.

const https = require('https');

exports.handler = function(event, context, callback) {
    context.callbackWaitsForEmptyEventLoop = false;

    const options = {
      hostname: 'f3e7f2cc.ngrok.io',
      port: 443,
      path: '/',
      method: 'GET'
    };

    setTimeout(function() {
        console.log(options);
        const req = https.request(options, (res) => {
            console.log('statusCode:', res.statusCode);
            console.log('headers:', res.headers);
        });

        req.end();
    }, 5000);

    const end = new Date().toUTCString();
    callback(null, {body: "OK", timeEnd: end});
};

However, I can see that this is not working as expected. I get an immediate callback response, but no GET request is received after 5 seconds. Are there additional configurations that need to be tweaked to get this going? Or should I just go with the 2 lambda approach?

why not use two lambdas, as you mentioned, its easier to do with two lambdas, one calling the other using Event invocation type. — Arun K

nickolay.laptev nickolay.laptev · Accepted Answer · 2019-12-23T20:47:40

As I understand your problem is that Lambda is executed in a sync way and its execution is too long.

Solution is to invoke Lambda in an async way. In this case you will get an immediate response from Gateway and execution of your Lambda will be scheduled using internal Lambda queue.

How to do this is described here. It works for me.

AWS Lambda continue execution after callback

1 Answers