BigQuery CSV Export to GCS: set Content-Type header to “text/csv”

1
votes

I'm exporting a bigquery table as a CSV to a public Google Cloud bucket. From there I can host it publicly over HTTP. The BigQuery export (aka extract) feature doesn't seem to set the file's Google Cloud Storage metadata correctly.

The exporter sets the Content-Type field to be application/octet-stream. Because it's a CSV file, I want it to be set to text/csv (or maybe text/plain). Why? As google outlines here, that's a best practice for a variety of reasons.

I don't see any option in the api documentation to control this.

Another issue: when I set configuration.extract.compression to be GZIP, the resulting file in GCS doesn't have the Content-Encoding property set to 'gzip'. It would be nice if the exporter did that too!

1
google-bigquery
Is this a question or a feature request? As a feature request, you can file and track them in the following url: issuetracker.google.com/issues/… – Felipe Hoffa
Hey @Felipe Hoffa -- it's a question to ask whether this is possible in the current system. If the answer is 'no' then I'll file a feature request (or possible it's a bugfix, because the default behavior seems a bit wrong IMHO). – conradlee
I now take it that there is no way to set these fields (but please confirm this with an answer if you know for sure), so I've created a ticket in the issue tracker here: issuetracker.google.com/issues/111662658 – conradlee

1 Answers

1
votes

As the documentation [1] says, application/octet-stream is set when the uploader doesn't specify the content type. As in this case, BQ export is the uploader and you cannot specify it, it's going to be set as that. I saw that you created a feature request asking for that.

A workaround would be editing the metadata of each file in GCS bucket. You can achieve this by using the UI, console or coding [2].