Cloud Storage - can retrieve Object via API, but strange errors when trying to access the MediaLink

3
votes

I am using golang on appengine (go version go1.2.1 (appengine-1.9.3) linux/386)

My new app is having trouble when trying to read a cloud storage file's contents. I can get the Object info via google-api-go-client, and the MediaLink gives me a URL like this:

https://www.googleapis.com/storage/v1beta2/b/bucket/o/profile%2Fpath_to_the_file.jpeg?generation=1402107955298000&alt=media

On dev_app_server or appengine, when I try to urlfetch this jpeg, I get "404 Not Found". When I paste the same URL into my browser, I get a 307 redirect, and then a 200 for a new URL that looks likes this:

https://storage.googleapis.com/bucket/profile%2Fpath_to_the_file.jpeg?generation=1402107955298000

I tried a test and just did a urlfetch request for first a www.googleapis.com URL, and then a storage.googleapis.com URL. The first URL gives a 404, but the second URL gives 200 OK. I have nearly identical code working in another app - why am I getting the false 404s in this instance?

Thanks, Tim.

1
google-app-enginegogoogle-cloud-storagegoogle-api-go-client
I don't use those services so I can't be sure, but %2F (an escaped /) looks suspicious. Does it make any difference if you unescape the /? – twotwotwo
I'm not sure about the problem, but I would be interested in knowing what would happen if you tried using API version "v1" instead of "v1beta2." – Brandon Yarbrough
It seems to be an issue with how urlfetch reacts to the %2F in the URL. When I test with curl, I have to use a %2F in the file path. If I change it to / I get a 404. Both URLs give a 404 in urlfetch though. – tgreiser

1 Answers

1
votes

Apparently this is an issue with how golang parses a URL string and translates it to a net/url struct. You can use the URL.Opaque field to work around this behavior. I didn't write this code, but here is an example of how googleapi addresses this same issue in order to preserve the %2F separators in the path.

http://play.golang.org/p/TkKRROJTfb