Download image data then upload to Google Cloud Storage

Question

I have a Flask web app that is running on Google AppEngine. The app has a form that my user will use to supply image links. I want to download the image data from the link and then upload it to a Google Cloud Storage bucket.

What I have found so far on Google's documentation tells me to use the 'cloudstorage' client library which I have installed and imported as 'gcs'.
found here: https://cloud.google.com/appengine/docs/python/googlecloudstorageclient/read-write-to-cloud-storage

I think I am not handling the image data correctly through requests. I get a 200 code back from the Cloud Storage upload call but there is no object when I look for it in the console. Here is where I try to retrieve the image and then upload it:

img_resp = requests.get(image_link, stream=True)
objectName = '/myBucket/testObject.jpg'

gcs_file = gcs.open(objectName,
                    'w',
                    content_type='image/jpeg')
gcs_file.write(img_resp)
gcs_file.close()

edit: Here is my updated code to reflect an answer's suggestion:

image_url = urlopen(url)
content_type = image_url.headers['Content-Type']
img_bytes = image_url.read()
image_url.close()

filename = bucketName + objectName
options = {'x-goog-acl': 'public-read',
           'Cache-Control': 'private, max-age=0, no-transform'}

with gcs.open(filename,
              'w',
              content_type=content_type,
              options=options) as f:
    f.write(img_bytes)
    f.close()

However, I am still getting a 201 response on the POST (create file) call and then a 200 on the PUT call but the object never appears in the console.

GAEfan GAEfan · Accepted Answer · 2016-08-25T00:17:52

Try this:

from google.appengine.api import images
import urllib2

image = urllib2.urlopen(image_url)

img_resp = image.read()
image.close()

objectName = '/myBucket/testObject.jpg'
options = {'x-goog-acl': 'public-read', 
         'Cache-Control': 'private, max-age=0, no-transform'}

with gcs.open(objectName,
              'w',
              content_type='image/jpeg', 
              options=options) as f:

    f.write(img_resp)
    f.close()

And, why restrict them to just entering a url. Why not allow them to upload a local image:

if isinstance(image_or_url, basestring):    # should be url
    if not image_or_url.startswith('http'):
        image_or_url = ''.join([ 'http://', image_or_url])
    image = urllib2.urlopen(image_url)
    content_type =  image.headers['Content-Type']
    img_resp = image.read()
    image.close()
else:
    img_resp = image_or_url.read()
    content_type = image_or_url.content_type

If you are running on the development server, the file will be uploaded into your local datastore. Check it at:

http://localhost:<your admin port number>/datastore?kind=__GsFileInfo__

and

http://localhost:<your admin port number>/datastore?kind=__BlobInfo__

Download image data then upload to Google Cloud Storage

1 Answers