Truncate BigQuery table through spark

1
votes

Is it possible to truncate Google BigQuery table through Spark/Scala program?

I tried examples of wordcount given on the https://cloud.google.com/hadoop/examples/bigquery-connector-spark-example

But could not find any way to truncate table.

Any insight on this?

1
apache-sparkgoogle-bigquery

1 Answers

0
votes

The default write disposition in BigQueryConfiguration is WRITE_APPEND. You need to specify WRITE_TRUNCATE for the BigQuery output configuration. So adding this line of code should work:

[..]
conf.set(BigQueryConfiguration.OUTPUT_TABLE_WRITE_DISPOSITION_KEY, "WRITE_TRUNCATE")
[..]