I have two GCP projects. Project A and Project B. Under project A, I have a Bigquery data set and an IAM user - IAM-BQ-PROJ-A with roles BigQuery Data Viewer and BigQuery User.
Project B hosts a Kubernetes cluster. There is a Rails application in project B that is executing queries against the Bigquery dataset in project A. I have the credentials for IAM user - IAM-BQ-PROJ-A accessible to the Rails app. However, these queries fail with the following error -
Google::Cloud::PermissionDeniedError: accessDenied: Access Denied: Project B: The user IAM-BQ-PROJ-A does not have bigquery.jobs.create permission in Project B.
These queries run successfully when the Rails application is running in the local development environment outside of GCP.
If I create an IAM user - IAM-BQ-PROJ-A under Project B with roles BigQuery Data Viewer and BigQuery User then these queries execute successfully.
Why is this the case? Shouldn't these queries fire successfully without having an IAM user under project B provided that the credentials are accessible (similar to the local dev environment functionality)?