BigQuery authorization

1
votes

I am trying to use the bq command line tool to load data into BigQuery from GCS bucket and I receive the following error message:

BigQuery error in load operation: Access Denied: Job mythical-maxim-293:bqjob_r11765e0cd9ceb52b_000001427694f0e1_1: RUN_JOB

I was using service account (with private key) for authentication. I followed the following links for granting the service account access level:

https://developers.google.com/bigquery/loading-data-into-bigquery https://developers.google.com/bigquery/access-control

The service account email was granted WRITE access with the BigQuery dataset and READ access with the GCS bucket.

Note: Adding the service account email as a writer to the project, solved the issue but this is not feasible for my case. I am not allowed to request project level write access but BigQuery and GCS (readonly).

Thanks!

2
google-bigquerygoogle-cloud-storage
Can you copy and paste the exact command you were trying to execute? – vegemite4me
bq \ --service_account 692299006198-o3sg7i6nqp3dc3sa4kjj5voc290t5f77@developer.gserviceaccount.com \ --service_account_credential_file ~/.bigquery/identity.xplenty.cred \ --service_account_private_key_file ~/.bigquery/identity.xplenty.p12 \ --service_account_private_key_password notasecret \ --project_id mythical-maxim-293 \ load my_data_set_1.my_table_1 gs://xptest/test.csv – Moty Michaely
Did you give your service account read access to the objects in the bucket? Bucket read permission doesn't necessarily imply object read permission. – Brandon Yarbrough
Yes, I have. For the same service account, I was able to use gsutil for downloading the csv file: gsutil cp gs://xptest/test.csv ./test.csv – Moty Michaely

2 Answers

4
votes

In order to run the job, the service account must be given at least READ permissions on the project. This is because whoever runs a job in the project can do things that cost the project owner money (e.g. run queries).

To add the service account to the project, go to https://cloud.google.com/console, then click on "Permissions", then "Add member".

0
votes

You must provide the WRITE permission on the dataset.

https://cloud.google.com/bigquery/loading-data-into-bigquery#access

This is bad, as WRITE permission imply that you have READ permission. But, for bigquery READ is paid and Load is free. For doing free task, access to paid service should be necessary.

Google must correct this.