Error while downloading Google storage blob : (Received response with content-encoding: gzip, but failed to decode it.)

Question

I am using python to download file, however getting error while downloading file from google cloud storage.

File metadata is as follows: content_type : text/csv content_encoding: gzip file extension: *.csv.gz

Getting following exception: exceptions.ContentDecodingError: 'Received response with content-encoding: gzip, but failed to decode it.'

Using following python API to download file. blob.download_to_filename(filename, start=100)

Question: 1. How to download if source file has above metadata attributes? (gzip as content encoding) 2. How to uncompress file while copying (gsutil cp)?

Answer 1

1. Received response with content-encoding: gzip, but failed to decode it.

You can work around this for now by setting the Accept-Encoding header to identity:

import requests

requests.get('http://XXXXX/', headers={'Accept-Encoding': 'identity'})

An alternative solution The solution is to normalize the header:

if (req.http.Accept-Encoding ~ “gzip”) {

set req.http.Accept-Encoding = “gzip”;

} else {

unset req.http.Accept-Encoding;

}

2. It doesn't look like there's a way to disable the auto-decompression behavior for gsutil cp, for one-off use cases, gsutil cat will skip the decompression:

$ gsutil cat gs://bucket/obj.gz > /destination/path/obj.gz