Create BigQuery materialized view using Cloud Deployment Manager

Question

I can create native and external tables, as well as views in BigQuery using Deployment Manager, however, there doesn't seem to be any way to create a materialized view. Page at: https://cloud.google.com/bigquery/docs/materialized-views-intro says its still in beta - is it typical for GCP beta features to have no support in Deployment Manager?

Is there a workaround for this? Only thing I can think of is using a Job resource https://cloud.google.com/bigquery/docs/reference/rest/v2/jobs/insert and specifying a DDL query like: CREATE MATERIALIZED VIEW ... but I don't think Deployment Manager will track and manage the resource correctly.

Edit: for anyone else, this works even though not documented:

- name: test-mv
  type: bigquery.v2.table
  properties:
    datasetId: experiments
    tableReference:
      datasetId: experiments
      tableId: test_mv
    materializedView:
      query: "select test, COUNT(*) as counter from experiments.test group by test"

Nick_Kh Nick_Kh · Accepted Answer · 2020-06-02T11:16:01

According to Google Cloud Deployment Manager documentation bigquery.v2.table resource type is actually leveraging tables REST object as a part of BigQuery API inventory.

Having said this, I assume that you can fully rely on tables.insert method as per Bigquery guidelines here:

Call the tables.insert method with a defined materializedView resource as part of your API request. The materializedView resource contains a query field.

"materializedView": {
    "query": "select product_id,sum(clicks) as
                sum_clicks from project-id.my_dataset.my_base_table
                group by 1"
  }

Defining appropriate materializedView property in the initial deployment configuration or Python/Jinja template.

I haven't tested yet the mentioned above API functionality, but once you have any further doubts with the implementation I'll try to dig deeper.

Create BigQuery materialized view using Cloud Deployment Manager

1 Answers