Dataflow needs bigquery.datasets.get permission for the underlying table in authorized view

Question

In a dataflow pipeline, I'm reading from a BigQuery Authorized View:

beam.io.Read(beam.io.BigQuerySource(query = "SELECT col1 FROM proj2.dataset2.auth_view1", use_standard_sql=True))

This is the error which I'm getting:

Error: Message: Access Denied: Dataset proj1:dataset1: The user [email protected] does not have bigquery.datasets.get permission for dataset proj1:dataset1.

proj1:dataset1 has the base table for the view auth_view1.

According to this issue in DataflowJavaSDK, dataflow seems to be directly executing some metadata query against the underlying table.

Is there a fix available for this issue in Apache Beam SDK?

Answer 1

Explicitly setting the Query location is also a solution in the Apache Beam Java SDK, using the withQueryLocation option of BigQueryIO.

It looks like setting the query location is not possible in the Python SDK yet.