Connect Cloud Storage to Cloud Function

Question

I'm having issues executing a Cloud Function on GCP which tries to update some google sheets of mine. I got this script working in Jupyter but have struggled to deploy it virtually as a Cloud Function. My issue seems to be authorizing the CF to post to google sheets.

I've tried many things over 6+ hours, most questions on stackoverflow, medium articles github but haven't been able to find a working solution for me. I don't think it's a roles/permissions issues. I understand how some of these may work when you are outside cloud functions but not inside of it.

Ultimately, I think from what I've seen the best way is to host my JSON secret key inside of a storage bucket and call that, I've tried this to no success and this does seem somewhat convoluted as everything is from a google service.

I've honestly gone back to my orignal code so am back to the first error which is simply that my JSON key cannot be found as when I was running it in Jupyter it was in the same directory...hence why I created a google storage bucket to try to link to.

import pandas as pd
import gspread
from oauth2client.service_account import ServiceAccountCredentials
import google.cloud
from df2gspread import df2gspread as d2g
from df2gspread import gspread2df as g2d
import datetime
import time
import numpy as np


def myGet(event, context):
    scope = ['https://spreadsheets.google.com/feeds', 'https://www.googleapis.com/auth/drive']
    credentials = ServiceAccountCredentials.from_json_keyfile_name('my-key-name.json', scope)
    gc = gspread.authorize(credentials)

    spreadsheet_key = '--removed actual key/id--'

ERROR: File "/env/local/lib/python3.7/site-packages/oauth2client/service_account.py", line 219, in from_json_keyfile_name with open(filename, 'r') as file_obj: FileNotFoundError: [Errno 2] No such file or directory: 'my-key-name.json'

Thanks very much for any guidance and support on this. I have thouroughly looked and tried to solve this on my own. EDIT: Please keep in mind, this is not a .py file living in a directory, that's part of my issue, I don't know where to link to as its an isolated "Cloud Function" as far as I can tell.

Some links I've looked at in my 20+ attempts to fix this issue just to name a few:

How can I get access to Google Cloud Storage using an access and a secret key

Accessing google cloud storage bucket from cloud functions throws 500 error

https://cloud.google.com/docs/authentication/getting-started#auth-cloud-implicit-python

https://cloud.google.com/docs/authentication/getting-started#setting_the_environment_variable

UPDATE: I realized you could upload a zip of your files to show three files in the inline editor. At the beginning I was not doing this so could not figure out where to put the JSON key. Now I have it viewable and need to figure out how to call it in the method.

When I do a test run of the CF, I get a non-descript error which doesn't show up in the logs and can't test it from the Cloud Schedular like I could previously. I found this on stack overflow and feel like I now need the same version but for python and figure out what calls to make from the google docs.

Cloud Functions: how to upload additional file for use in code?

guillaume blaquiere guillaume blaquiere · Accepted Answer · 2020-01-08T06:13:26

My advice is to not use JSON key file in your Cloud Functions (and on all GCP product). With Cloud Function, like with other GCP product, you have the capability to load automatically the service account during your deployment.

The advantage of Cloud Function Identity is that you haven't a key file to store secretly, you don't have to rotate your key file for increasing the security, you don't have risk of leak of key file,...

By the way, use the default service account in your code.

If you need to get the credential object, you can use the oauth2 python library for this.

import google.auth

credentials, project_id = google.auth.default()

Connect Cloud Storage to Cloud Function

3 Answers