GCS on Google Cloud Functions slower than localhost

Question

I have a file in Google Cloud Storage. I need to access it from a Google Cloud Function that uses it to respond to an http request.

The code runs like a charm with "functions framework" on my PC and retrieves the file in about half a second.

When I put the code on Google Cloud Functions in the same region as the storage, I expected it to run faster (at least saving the network trip from my house in Milan to Google's locations in Belgium).To my surprise the same code running on GCF takes on average around 5 seconds (sometimes much longer).

I tried everything I could imagine without any significant effect. Some things I already tried:

changing the version of Node runtime
importing fast-crc32c dependency
requesting a ReadableStream and piping it directly to the res object

Now I am running out of ideas...

The relevant code is (simplified for testing and for clarity):

const { Storage } = require("@google-cloud/storage");

exports.helloWorld = (req, res) => {
  const storage = new Storage({ projectId: "my project ID" });
  const bucketName = "my-bucket-name";
  const srcFilename = "my-file-name.json";

  let file = storage.bucket(bucketName).file(srcFilename);
  file
    .download()
    .then(function(data) {
      const d = data;
    })
    .then(() => {
      res.status(200).send("Ciao!");
    });
};

Any help or idea is much appreciated!

Thanks,

Filippo

PS I add an instrumented version of the code I used to measure the timing:

const { Storage } = require("@google-cloud/storage");

exports.helloWorld = (req, res) => {
  var hrstart = process.hrtime();
  var timedEvents = [];
  function addTimedEvent(msg) {
    const event = { msg: msg, hrend: process.hrtime(hrstart) };
    timedEvents = [...timedEvents, event];
  }
  function printTimeEvents() {
    for (var e of timedEvents) {
      console.info(
        "Execution time (hr): %ds %dms Message:%s",
        e.hrend[0],
        e.hrend[1] / 1000000,
        e.msg
      );
    }
  }
  addTimedEvent("gcs 1");
  const storage = new Storage({ projectId: "my project ID" });
  const bucketName = "my-bucket-name";
  const srcFilename = "my-file-name.json";

  addTimedEvent("gcs 2");
  let file = storage.bucket(bucketName).file(srcFilename);
  addTimedEvent("gcs 3");
  file
    .download()
    .then(function(data) {
      addTimedEvent("gcs 4");
      const d = data;
      addTimedEvent("gcs 5");
    })
    .then(() => {
      printTimeEvents();
      res.status(200).send("Ciao world!");
    });
};

The typical execution on my PC:

Execution time (hr): 0s 0.0059ms Message:gcs 1
Execution time (hr): 0s 0.1911ms Message:gcs 2
Execution time (hr): 0s 0.7464ms Message:gcs 3
Execution time (hr): 0s 338.6312ms Message:gcs 4
Execution time (hr): 0s 338.6361ms Message:gcs 5

The typical execution on GCF:

Is it slow on every invocation or just the first? Cloud functions endure a cold start penalty if a new instance needs to be created to serve a request (frequently immediately after deployment, for example). — robsiemb
@robsiemb: on every invocation. To be sure, I measured the timing of the file.download() in order to sterilize any effect of the cold-start, but no... it is not a cold-start problem unfortunately. — filippopanelli
Please edit the question show exactly what you're using to benchmark the time. If you're not doing it in the code itself, you're probably measuring much more than just the transfer itself. — Doug Stevenson
@DougStevenson I added the instrumented code, I hope it helps — filippopanelli

Jak Jak · Accepted Answer · 2019-11-27T18:00:22

Set up whatever can be shared between invocations outside of the function, for example (if the bucket does not change):

const { Storage } = require("@google-cloud/storage");
const storage = new Storage({ projectId: "my project ID" });
const bucketName = "my-bucket-name";
const bucket = storage.bucket(bucketName);

exports.helloWorld = (req, res) => {
  const srcFilename = "my-file-name.json";

  let file = bucket.file(srcFilename);
  file
    .download()
    .then(function(data) {
      const d = data;
    })
    .then(() => {
      res.status(200).send("Ciao!");
    });
};

GCS on Google Cloud Functions slower than localhost

1 Answers