2
votes

Just started using google cloud storage. I am having problem receiving gzip files from it. Stored file is gzip'ed but it gets decompressed on request. I have read all documentation, forums and have tried dozens of different combinations of headers and still fetched file in browser is not compressed. Could some one point out what I am doing wrong?

Article about gzip, headers, transcoding https://cloud.google.com/storage/docs/transcoding

I move my file to storage with this command

gsutil -h "Cache-Control:public,max-age=10,no-transform" cp  -Z bundle.js gs://lol-champs.tomdid.com 

File gets gziped before uploading file and size decreases by 5 times once it gets on storage, I have checked in console.

Browser request headers

Accept:*/*
Accept-Encoding:gzip, deflate, sdch
Accept-Language:en-US,en;q=0.8,lt;q=0.6
Cache-Control:no-cache
Connection:keep-alive
Host:lol-champs.tomdid.com
Pragma:no-cache
Referer:http://lol-champs.tomdid.com/

Response headers

Accept-Ranges:bytes
Access-Control-Allow-Origin:*
Access-Control-Expose-Headers:Content-Encoding
Access-Control-Expose-Headers:Content-Length
Access-Control-Expose-Headers:Content-Type
Cache-Control:public,max-age=10,no-transform
Content-Language:en
Content-Type:application/javascript
Date:Sat, 15 Oct 2016 21:40:14 GMT
ETag:"8505f976c2ffbf1e69c3ee9fdf2f04d5"
Expires:Sat, 15 Oct 2016 21:40:24 GMT
Last-Modified:Sat, 15 Oct 2016 21:40:09 GMT
Server:UploadServer
Transfer-Encoding:chunked
x-goog-generation:1476567609681000
x-goog-hash:crc32c=Gik6aA==
x-goog-hash:md5=hQX5dsL/vx5pw+6f3y8E1Q==
x-goog-metageneration:1
x-goog-storage-class:STANDARD
x-goog-stored-content-encoding:gzip
x-goog-stored-content-length:290207

So google sets custom header that content is gziped in storage but it sends back response to me decompressed and without Content-Encoding header.

Not sure is the way how I upload file is wrong, or it is missing some headers. feeling lost. Link to File -> http://lolchamps.tomdid.com/bundle.js

Any help would be much appreciated.

UPDATE: Found something interesting, if I am requesting file with googles URL- https://storage.googleapis.com/lolchamps.tomdid.com/bundle.js it returns gziped file, if the same file is requested via my domain it returns decompressed version. WHY??? I smell dark magic.

1
Can I ask you sth. When you upload the file, the file is compressed at google cloud storage servers or at client side?Is there such option? - curious
@curious it is compressed using gsutils when uploading. Check cloud.google.com/storage/docs/gsutil/commands/cp -Z option - Oozhaa
I have the exact same problem here with 2 behaviors depending of the URL as mentioned in your update. This is dark magic indeed. Did you solve this? - probitaille

1 Answers

0
votes

I can't seem to reproduce this...?

$ curl -v http://lolchamps.tomdid.com/bundle.js
[...]
< Content-Length: 290667
< Content-Encoding: gzip
[...]

Edit: Same with Chrome, full request as follows:

GET /bundle.js HTTP/1.1
Host: lolchamps.tomdid.com
Connection: keep-alive
User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.2840.59 Safari/537.36
Accept: */*
DNT: 1
Referer: http://lolchamps.tomdid.com/
Accept-Encoding: gzip, deflate, sdch
Accept-Language: en-US,en;q=0.8,de;q=0.6,de-DE;q=0.4

And the response:

HTTP/1.1 200 OK
Date: Tue, 18 Oct 2016 17:27:43 GMT
Cache-Control: no-transform
Expires: Wed, 18 Oct 2017 17:27:43 GMT
Last-Modified: Sun, 16 Oct 2016 22:53:39 GMT
ETag: "7c79e6fd21a30bf99f0344a7fb9f6105"
x-goog-generation: 1476658419217154
x-goog-metageneration: 1
x-goog-stored-content-encoding: gzip
x-goog-stored-content-length: 290667
Content-Type: application/javascript
Content-Encoding: gzip
Content-Language: en
x-goog-hash: crc32c=vZKmgQ==
x-goog-hash: md5=fHnm/SGjC/mfA0Sn+59hBQ==
x-goog-storage-class: STANDARD
Accept-Ranges: bytes
Content-Length: 290667
Access-Control-Allow-Origin: *
Access-Control-Expose-Headers: Content-Type
Access-Control-Expose-Headers: Content-Length
Access-Control-Expose-Headers: Content-Encoding
Server: UploadServer