Google Cloud Function - Error: Cannot find module google-cloud/bigquery

Question

I am working to create a cloud function that moves avro files from GCS to BigQuery, anytime a new file lands in GCS. I am using the cloud function ui in GCP. I have 512 MB for memory allocated. Trigger is Google Storage. Event Type if Finalize/Create. Source Code is Inline Editor.

Below is my code. I can successfully deploy, but I am receiving the below error post deployment, and nothing moves to BigQuery.

Additionally, I am attempting to move avro files from a folder WITHIN a bucket, so I am not pulling directly from the top parent bucket. That is the purpose of the below code, as I attempt to get into the folder, which is called "example_spend/"

error: Cannot find module google-cloud/bigquery

'use strict';

exports.createExampleTableFromFile = function(event, callback) {
const file = event.data;
if (file.resourceState === 'exists' && file.name && 
    file.name.indexOf('example_spend/') !== -1) {

console.log('Processing file: ' + event.data.name);


const BigQuery = require('@google-cloud/bigquery');
const Storage = require('@google-cloud/storage');
const assert = require('assert');


 const filename = event.data.name;
 const bucketName = event.data.bucket;


const projectId = "gcp-pilot-192921";
const datasetId = "example_etl";
const tableId = filename.slice(0,filename.indexOf(".")).toLowerCase();  


const bigquery = new BigQuery({
  projectId: projectId,
});

const storage = Storage({
  projectId: projectId
});  


const metadata = {
  sourceFormat: 'AVRO', 
  autodetect: true, 
  writeDisposition: 'WRITE_TRUNCATE'
};

bigquery
  .dataset(datasetId)
  .table(tableId)
  .load(storage.bucket(bucketName).file(filename), metadata)
  .then(results => {
    const job = results[0];


    assert.equal(job.status.state, 'DONE');
    console.log(`Job ${job.id} to load table ${tableId} completed.`);  


    const errors = job.status.errors;
    if (errors && errors.length > 0) {
      throw errors;
    }

  })
  .catch(err => {
    console.error('Error during load job:', err);
  });

callback();
}};

You have a low rate. Important on SO, you have to mark accepted answers by using the tick on the left of the posted answer, below the voting. This will increase your rate. See how this works by visinting this link: meta.stackoverflow.com/questions/5234/… — Pentium10
Thank you! I didn't realize how much it mattered. Thank you for letting me know. — Bava
put braces around "BigQuery". The api requires it (note Storage does NOT tho)... const {BigQuery} = require('@google-cloud/bigquery'); — smoore4

Ben P Ben P · Accepted Answer · 2019-05-02T15:03:51

It looks like you haven't added any dependencies to your function:

When using the inline editor, click on to "requirements.txt" for python or "package.json" for javascript, where you can enter the required packages that your function needs to run, these will then be imported when your function spins up. Note that you can also specify versions if required, for example in python: requests==2.19.0.

Google Cloud Function - Error: Cannot find module google-cloud/bigquery

1 Answers