Awaiting http request in AWS Lambda / Pulumi

Question

I have an AWS Lambda function which triggers https request to Google API. I want the function to be awaitable, so that it does not end immediately, but only after getting response from Google API. Yes, I know I pay for the execution, but this will not be called often, so it is fine.

The problem is that the http request does not seem to fire correctly. The callback is never executed.

I have made sure that the async/await works as expected by using setTimeout in a Promise. So the issue is somewhere in the https.request.

Also note that I am using Pulumi to deploy to AWS, so there might be some hidden problem in there. I just can't figure out where.

---

The relevant code:

AWS Lambda which calls the Google API

import config from '../../config';
import { IUserInfo } from '../../interfaces';
const https = require('https');

function sendHttpsRequest(options: any): Promise<any> {
    console.log(`sending request to ${options.host}`);
    console.log(`Options are ${JSON.stringify(options)}`);

    return new Promise(function (resolve, reject) {
        console.log(` request to ${options.host} has been sent A`);

        let body = new Array<Buffer>();
        const request = https.request(options, function (res: any) {
            console.log('statusCode:', res.statusCode);
            console.log('headers:', res.headers);

            if (res.statusCode != 200) {
                reject(res.statusCode);
            }

            res.on('data', (data: any) => {
              console.log(`body length is ${body.length}`);
              console.log('data arrived', data);
              body.push(data);
              console.log('pushed to array');
              console.log(data.toString());
            });
        });

        request.on('end', () => {
            console.error('Request ended');
            // at this point, `body` has the entire request body stored in it as a string
            let result = Buffer.concat(body).toString();
            resolve(result);
        });

        request.on('error', async (err: Error) => {
          console.error('Errooooorrrr', err.stack);
          console.error('Errooooorrrr request failed');
          reject(err);
        });

        request.end();

      console.log(` request to ${options.host} has been sent B`);
    });
}

/**
 * AWS Lambda to create new Google account in TopMonks domain
 */
export default async function googleLambdaImplementation(userInfo: IUserInfo) {

    const payload = JSON.stringify({
        "primaryEmail": userInfo.topmonksEmail,
        "name": {
            "givenName": userInfo.firstName,
            "familyName": userInfo.lastName
        },
        "password": config.defaultPassword,
        "changePasswordAtNextLogin": true
    });

    const resultResponse: Response = {
        statusCode: 200,
        body: 'Default response. This should not come back to users'
    }     

    console.log('Calling google api via post request');

    try {
        const options = {
            host: 'www.googleapis.com',
            path: '/admin/directory/v1/users',
            method: 'POST',
            headers: {
                'Content-Type': 'application/json',
                'Content-Length': payload.length.toString()
            },
            form: payload
        }

        const responseFromGoogle = await sendHttpsRequest(options);
        console.log('responseFromGoogle', JSON.stringify(responseFromGoogle));
    }
    catch (err) {
        console.log('Calling google api failed with error', err);
        resultResponse.statusCode = 503;
        resultResponse.body = `Error creating new Google Account for ${userInfo.topmonksEmail}.`;
        return resultResponse;
    }

    console.log('request to google sent');
    return resultResponse;
 }

Answer 1

The problem is that the http request does not seem to fire correctly. The callback is never executed.

I believe this part of the issue is related to some combination of (a) potentially not actually sending the https request and (b) not using the correct callback signature for https.request. See the documentation at https://nodejs.org/api/https.html#https_https_request_options_callback for details on both of these.

Use node-fetch package

The following example works for me using node-fetch:

import * as aws from "@pulumi/aws";
import fetch from "node-fetch";

const api = new aws.apigateway.x.API("api", {
    routes: [{
        method: "GET", path: "/", eventHandler: async (ev) => {
            const resp = await fetch("https://www.google.com");
            const body = await resp.text();
            return {
                statusCode: resp.status,
                body: body,
            }
        },
    }],
})

export const url = api.url;

Pulumi complains, it something like "Can not serialize native function" or something like that. The problematic part is that node-fetch relies on Symbol.iterator

As noted in the comments, some of the conditions that can lead to this are documented at https://pulumi.io/reference/serializing-functions.html. However, I don't see any clear reason why this code would hit any of those limitations. There may be details of how this is used outside the context of the snippet shared above which lead to this.