Google Cloud Storage: Setting incorrect MIME-type

3
votes

I have a Node.js server running on a Google Compute Engine virtual instance. The server streams incoming files to Google Cloud Storage GCS. My code is here: Node.js stream upload directly to Google Cloud Storage

I'm passing Content-Type in the XML headers and it's working just fine for image/jpeg MIME-types, but for video/mp4 GCS is writing files as application/octet-stream.

There's not much to this, so I'm totally at a loss for what could be wrong ... any ideas are welcome!

Update/Solution

The problem was due to the fact that the multiparty module was creating content-type: octet-stream headers on the 'part' object that I was passing into the pipe to GCS. This caused GCS to receive two content-types, of which the octet part was last. As a result, GCS was using this for the inbound file.

2
google-cloud-storagegoogle-compute-engine
Could you please post what your http request looks like? Are you sure that video/mp4 is really set as content type in the http request? - markovuksanovic
@markovuksanovic thank you for the help; please see above - Dan
Can you confirm that this is multipart upload? In that case there should be multiple http requests sent out? Can you also provide http response headers and body for this request (and the other ones if there are any)? - markovuksanovic
@markovuksanovic My level-of-knowledge with a lot of HTTP is kind of low -- how can I verify that it is multipart? Other info included above (console.log of the response from Google). - Dan
You can use wireshark to trace http. Make sure that you don't use HTTPS requests but HTTP only (just for testing purposes). You can also send me/share the capture file so I can have a look at what's happening. - markovuksanovic

2 Answers

1
votes

Ok, looking at your HTTP request and response it seems like content-type is specified in the URL returned as part of the initial HTTP request. The initial HTTP request should return the endpoint which can be used to upload the file. I'm not sure why that is specified there but looking at the documentation (https://developers.google.com/storage/docs/json_api/v1/how-tos/upload - start a resumable session) it says that X-Upload-Content-Type needs to be specified, along some other headers. This doesn't seem to be specified in HTTP requests that were mentioned above. There might be an issue with the library used but the returned endpoint does not look as what is specified in the documentation.

Have a look at https://developers.google.com/storage/docs/json_api/v1/how-tos/upload, "Example: Resumable session initiation request" and see if you still have the same issue if you specify the same headers as suggested there.

0
votes

Google Cloud Storage is content-type agnostic, i.e., it treats any kind of content in the same way (videos, music, zip files, documents, you name it).

But just to give some idea,

First I believe that the video () you are uploading is more or less size after it being uploded. so , it falls in application/<sub type>. (similar to section 3.3 of RFC 4337)

To make this correct, I believe you need to fight with storing mp4 metadata before and after the file being uploaded.

please let us know of your solution.