Access BigQuery DataStore of Project1 from Python Cloud Function in Project 2 using Service Account Json Key of Project 1

0
votes

I have 2 different projects in GCP.

I need to access BigQuery DataStore of Project-1. Have created the required Service Account Json Key.

Have written a Python Cloud Function in Project-2. I need to access BigQuery Data store of Project-1. as I understand, i need to retrieve the credentials from the service account json key file (from Project-1). Have uploaded the service account json key file in bucket storage of Project-2.

Am trying to retrieve the credentials from this Json key file. Have tried following different methods:
credentials = service_account.Credentials.from_service_account_file(filename,scopes='https://www.googleapis.com/auth/cloud-platform') or client = bigquery.Client.from_service_account_json(filename) or client = bigquery.Client.from_service_account_file(filename) or credentials = service_account.Credentials.from_service_account_info(service_account_info)

and I keep getting error,

[Errno 2] No such file or directory: filename

I have tried to give filename as URL or LinkURL or Public URL of the bucket Json file.

Please advise how we can access BigQuery of Project-1 from Cloud Function of Project-2 and I do have Service Account Json Key from Project-1.

client = bigquery.Client(credentials=credentials,project=credentials.project_id,)

#client = bigquery.Client.from_service_account_json(filetest)

from Python Cloud Function in Project 2 using Service Account Json Key of Project 1

3
jsongoogle-bigquerygoogle-cloud-functionscredentials

3 Answers

1
votes

Thanks for your help. We are able to make it working.

It is as following:

  1. Create Service Account in Project 2

  2. In Project 1, Add new IAM role with new member as Project 2 Service Account ID and give access to BigQuery of Project 1

  3. In Project 2, Add Service Account ID to execute the Cloud Function

  4. In Cloud Function, just refer to Project 1 like client = bigquery.Client(project='Project 1) and we are able to retrieve the data from BigQuery. –

1
votes

Service accounts have mail addresses like user accounts.

So you may give required permissions to [email protected] on project1 dataset/tables? Do you really need another service account on project2?

0
votes

I need to clarify first:

  1. I notice that you store the service account in project 2 how to you pass this to your cloud function? Do you use some gcs client to read the content of the service account and then set is as GOOGLE_APPLICATIONS_CREDENTIALS environment variable?
  2. If you already create a service account and download the key from project 1 then you only need to make sure you can query the data by acquired necessary access, e.g BigQuery viewer if you only need to query the data. After that passed the credentials and you can proceed with the step that you already mentioned before.