How can I load parquet data to spark from Google cloud storage and save it to BigQuery?
0
votes
1 Answers
2
votes
To load data into Spark from Google Cloud Storage:
df=spark.read.parquet("gs://<path to parquet file>")
And you can write after doing necessary transformations in the similar way(add gs to the start of the file name)
You can parquet load data from cloud storage in parquet to BigQuery , by following the below link:
https://cloud.google.com/bigquery/docs/loading-data-cloud-storage-parquet
