Puppeteer slow execution on Google Cloud Functions

Question

I am using Puppeteer on Google Cloud Functions.

After a few tests, I noticed that my code takes about 56 seconds on average when deployed on Google Cloud Functions infrastructure, while the same function tested locally takes only 13 seconds.

index.js

const chromium = require('chrome-aws-lambda');
const puppeteer = require('puppeteer-core');
const functions = require('firebase-functions');

exports.check = functions.https.onRequest(async (req, res) => {
    const License = req.query.License;

    browser = await puppeteer.launch({
        args: chromium.args,
        defaultViewport: chromium.defaultViewport,
        executablePath: await chromium.executablePath,
        headless: chromium.headless,
      });
    const page = await browser.newPage();

    await page.goto('http://www.example.com', {waitUntil: 'networkidle2'});
    await page.focus('#txtUserName');
    await page.keyboard.type('testUsername');
    await page.focus('#txtPassword');
    await page.keyboard.type('123123');
    await page.click('#btnLogin');
    await page.waitForSelector('#ctl00_400_header_400')
    //console.log("[✓]login successfully.")
    await page.evaluate(() => document.querySelector('#ctl00_400_header_400').click());
    await page.waitForSelector('#__tab_ctl00_ContentPlaceHolder1_tabQuickSearch_vehicleSerachClaim')
    //console.log("[✓]Enquriy page loaded successfully")
    await page.evaluate(() => document.querySelector('#__tab_ctl00_ContentPlaceHolder1_tabQuickSearch_vehicleSerachClaim').click());
    await page.waitForSelector('#ctl00_ContentPlaceHolder1_tabQuickSearch_vehicleSerachClaim_rdvehicleSearchLicense')
    //console.log("[✓]Claim section loaded successfully")
    await page.evaluate(() => document.querySelector('#ctl00_ContentPlaceHolder1_tabQuickSearch_vehicleSerachClaim_rdvehicleSearchLicense').click());
    //console.log("[✓]License tap loaded successfully")
    await page.waitForSelector('#ctl00_ContentPlaceHolder1_tabQuickSearch_vehicleSerachClaim_txtclaimSearchPersonLicNo');
    await page.focus('#ctl00_ContentPlaceHolder1_tabQuickSearch_vehicleSerachClaim_txtclaimSearchPersonLicNo');
    await page.keyboard.type(License);
    await page.evaluate(() => document.querySelector('#ctl00_ContentPlaceHolder1_tabQuickSearch_vehicleSerachClaim_btnVheicleSearchButtonClaim').click());    

    try {
        await page.waitForSelector('#ctl00_ContentPlaceHolder1_lblErrMessage')
        const textContent = await page.evaluate(() => document.querySelector('#ctl00_ContentPlaceHolder1_lblErrMessage').textContent);
        res.status(200).send( 'Result => ' + textContent );
        await browser.close();
    } catch (error) {
        //console.log("The element didn't appear.")
    }    

    try {
        await page.waitForSelector('#ctl00_ContentPlaceHolder1_tabQuickSearch_vehicleSerachClaim_grdClaimDraftSp > tbody > tr:nth-child(3) > td')
        const textContent = await page.evaluate(() => document.querySelector('#ctl00_ContentPlaceHolder1_tabQuickSearch_vehicleSerachClaim_grdClaimDraftSp > tbody > tr:nth-child(3) > td').textContent);
        res.status(200).send( 'Result => ' + textContent );
        await browser.close();
    } catch (error) {
        //console.log("The element didn't appear.")
    }   

});

Package.json

{
    "name": "functions",
    "version": "0.0.1",
    "description": "Cloud Functions for Firebase",
    "dependencies": {
      "chrome-aws-lambda": "1.14.0",
      "firebase-functions": "2.2.0",
      "iltorb": "2.4.2",
      "puppeteer-core": "1.14.0",
      "firebase-admin": "7.2.0"
    },
    "engines": {
      "node": "8"
    },
    "private": true
  }

Deployed with Firebase Functions using NodeJS 8 & 2 GB Memory allocated.

How I can improve my code to speed up the execution time?

You cannot compare a cloud function to local execution. How long is the start of the function vs. execution? Also checkout this issue on github. You should also measure which parts of your script are slow. — Thomas Dondorf

Doug Stevenson Doug Stevenson · Accepted Answer · 2019-04-04T16:51:32

I don't think it's good to expect that any give code should run as fast in Cloud Functions as on any modern desktop, especially not something as complex as Puppeteer (which is essentially running Chrome).

GCF only allocates a single CPU to any given server instance. It doesn't have a GPU. GCF is meant for simple work that doesn't require heavy computation. Desktops often have 4-8 cores (or more) and a GPU that helps Chrome run quickly. There is really no comparison that can be made between these two situations.

Bottom line is that, for this code, there is not much you can do to speed it up to be in line with a desktop experience.

Puppeteer slow execution on Google Cloud Functions

1 Answers