I'm working on a project that consists of receiving data in BigQuery, do some aggregations and export it to Qlik Sense to make dashboards. I'm still on a exploratory phase with only a few Gb of data but I want to calculate how much I will pay to export data from BigQuery to an external platform in a production phase. When I pass data from Cloud Storage buckets (where the data comes from) to BigQuery, data compression occurs, i.e., the tables in bigQuery occupy less space than in the buckets of Cloud Storage. Thus, my question is, when I then export the data from BigQuery to an external paltform, what volume of traffic will be considered? The data's original size (before compression), or the size (compressed) it has on bigQuery? Will the egress(export) volume depend on the format that Qlik uses to export the data? If yes, what is the format Qlik uses to export the data?
1 Answers
0
votes
If you follow BigQuery Exporting Data and export BigQuery table data to GCS. Exporting itself is free of cost.
Although free, exporting has a limit of 10 TB of data per day per project. You are suggested to use Storage API if exporting quota doesn't work for you.
In case that you are trying to use Qlik/BigQuery connector. I noticed that the maximum response size is 128MB compressed, it might not be able to export a few GB of data for you. From the documentation, the pricing model on BigQuery side is simple, since the connector is just placing a query and read response:
- Query cost: see BigQuery query pricing, which is based on the size of the table that the query scanned
- Transfer cost: free
