BigQuery API query job return 404 for existing dataset

Question

I am running into a problem while using BigQuery API in Python.

I have set up a bigquery Client bigquery.Client(credentials=credentials, project=project_id, location=location) and it works like expected. datasets = list(client.list_datasets()) Returns the list of datasets in the project.

%%bigquery --project project_id df
SELECT *
FROM `cached.csv_reports_sessions_report`

Magics request works and creates the required dataframe.

However, if I try to create the same dataframe from a query job like so:

sql = """
    SELECT *
    FROM `cached.csv_reports_sessions_report`
    LIMIT 100
"""
df = client.query(sql, project=project_id).to_dataframe()

I get the following error: 404 Not found: Dataset project_id:cached was not found in location europe-west3. The location is correct location for the project and dataset.

I have tried using the full path including the project_id like so: project_id.cached.csv_reports_sessions_report Still get the same error.

What am I doing wrong?

Eldar Vagapov Eldar Vagapov · Accepted Answer · 2020-06-11T07:38:51

I found out the problem. I have declared my location as 'europe-west3' which is the actual location, but it seems for the query job it was too specific. It started working fine after declaring location as simply 'EU'

BigQuery API query job return 404 for existing dataset

1 Answers