Login failed during python Insert job with BigQuery API

Question

I am trying load local file to bigquery by setting up a server-server auth. I've done following steps

Created service account
Create JSON key file for this account
Activated service acount with

gcloud auth activate-service-account command
Logged in with

gcloud auth login

Trying to execute python script to upload file to BigQuery

scopes =

['https://www.googleapis.com/auth/bigquery',
         'https://www.googleapis.com/auth/bigquery.insertdata']

credentials = ServiceAccountCredentials.from_json_keyfile_name(
    '/path/privatekey.json', scopes)
# Construct the service object for interacting with the BigQuery API.
service = build('bigquery', 'v2', credentials=credentials)

# Load configuration with the destination specified.
load_config = {
    'destinationTable': {
        'projectId': "project id",
        'datasetId': "data set id",
        'tableId': "table name"
    }
}

# Setup the job here.
# load[property] = value
load_config['schema'] = {
    'fields': [
        <several field>
    ]
}


upload = MediaFileUpload('/path/to/csv/file',
                         mimetype='application/octet-stream',
                         # This enables resumable uploads.
                         resumable=True)
# End of job configuration.

run_load.start_and_wait(service.jobs(),
                        "my project id",
                        load_config,
                        media_body=upload)

The result is

   "error": {
    "errors": [
   {
    "domain": "global",
    "reason": "required",
    "message": "Login Required",
    "locationType": "header",
    "location": "Authorization"
   }
  ],
  "code": 401,
  "message": "Login Required"
 }
}

But I have enough rights to create query jobs

query_request = service.jobs()
query_data = {
    'query': (
        'SELECT COUNT(*) FROM [dmrebg.testDay];')
}

query_response = query_request.query(
    projectId=project_id,
    body=query_data).execute()

print('Query Results:')
for row in query_response['rows']:
    print('\t'.join(field['v'] for field in row['f']))

What did I miss? I thought that I've already logged in.

dmreshet dmreshet · Accepted Answer · 2016-06-15T16:32:30

The problem is that any call to https://www.googleapis.com/bigquery/v2/projects/project_id/jobs/* will cause the same problem

{
  "error": {
    "errors": [
   {
    "domain": "global",
    "reason": "required",
    "message": "Login Required",
    "locationType": "header",
    "location": "Authorization"
   }
  ],
  "code": 401,
  "message": "Login Required"
 }
}

So it is a problem with my browser auth, python auth is good.

And root cause is that I my CSV Schema and data do not match.

Errors:
Too many errors encountered. (error code: invalid)

Login failed during python Insert job with BigQuery API

1 Answers