Google Cloud Storage (Js), Async/Await & Cloud Functions Errors

Question

THIS IS SOLVED ON GOOGLE SIDE NOW

Google Cloud NodeJs library now fix integrated. Keeping this question just for reference.

ORIGINAL QUESTION

I wanted my code to be clean(er) and used Typescript & async/await writing Cloud Functions accessing Google Cloud Storage.

Some important parts of my tsconfig.json:

{
  "compilerOptions": {
    "declaration": false,
    "target": "es6",
    "module": "commonjs",
    "noImplicitAny": true,
    "removeComments": true,
    "noUnusedLocals": true,
    "moduleResolution": "node",
    "sourceMap": false,
    "typeRoots": [
      "node_modules/@types"
    ],
    "lib": [
      "es2016"
    ],
    "emitDecoratorMetadata": true,
    "experimentalDecorators": true,
    "outDir": "./"
  }
}

First error I had was ECONNRESET and I was making a call like this which probably caused it:

await bucket.file(path).download({
  destination: tempFilePath
})

I've figured rest of the function did not wait till this line to be finished and whole function execution ends before file is downloaded from GCS to temp path. So I've put that section into try/catch block:

  try { 
    await bucket.file(path).download({
      destination: tempFilePath
    })
  } catch (error) {
    console.log(error);
  }

It worked fine till today. Today I had this error:

convert: Empty input file '/temp/img-file.jpg'

which again made me think that next line (converting image size) is executed before file is downloaded from bucket to temp folder.

Am I missing something ?

@m_callens adding a console.log just before that call now, will let you know soon. — Bogac
Each time it prints correct path and I've made 6 tests, all worked now. Is there a time I need to wait after I deploy my functions? I've tested and had error within a minute after deployment (when that time I had error), now all seems to work fine. — Bogac
Today I had this error: code: 'ECONNRESET', errno: 'ECONNRESET', syscall: 'read' I'm going back to Promise-Land and ditch async/await — Bogac

Bogac Bogac · Accepted Answer · 2017-05-04T10:10:29

Apparently I've made some errors due to fact that there are still many libraries out there not having proper type definitions, or not at all, including Google ones, and me making wrong guesses.

Be careful using libraries without Type Definitions and open their source to see return types.

While searching a solution I ran into this ansfer as well: GCS permissions related error and a bash script to fix. If you read this answer maybe your problem is related to that, so I share it here too.

UPDATE: Again I've seen an error, so to double make sure I've included both async calls in try/catch:

  let myBucket : any = {};

  try {
    myBucket = await googlecloudstorage.bucket('<bucket-id>');
    await myBucket.file(path).download({
      destination: tempFilePath
    })
  } catch (error) {
    console.log(error);
  }

UPDATE 2: Still having errors, trying o figure it out within Typescript but I plan to go back to plain javascript. Also I've tried updating ACLs with a batch file as explained in issue I've provided link above, didn't seem to help.

UPDATE 3 (Hopeful to be last): Well, for calls related to Google Cloud Storage I use plain javascript promises now and ditch async/await, like ones used in Firebase-Functions Samples repo, so far so good.

UPDATE 4: I kept getting errors, after changing my code, upon uploads/reads inconsistently. Made further searches and...

Read this answer from another question describing same problem. Basically it's not us, it's them, but they don't admit. It's because the way google-cloud-node package uses sockets. Will post it here if some sane way of fixing this comes up.

UPDATE 5:

Here is how not to keep sockets open forever:

var gcs = require('@google-cloud/storage')(...)
gcs.interceptors.push({
  request: function(reqOpts) {
    reqOpts.forever = false
    return reqOpts
  }
})

...seems to work, for now.

Google Cloud Storage (Js), Async/Await & Cloud Functions Errors

1 Answers