Django Tweepy can't access Amazon S3 file

Question

I'm using Tweepy, a tweeting python library, django-storages and boto. I have a custom manage.py command that works correctly locally, it gets an image from the filesystem and tweets that image. If I change the storage to Amazon S3, however, I can't access the file. It gives me this error:

 raise TweepError('Unable to access file: %s' % e.strerror)

I tried making the images in the bucket "public". Didn't work. This is the code (it works without S3):

filename = model_object.image.file.url
media_ids = api.media_upload(filename=filename)  # ERROR

params = {'status': tweet_text, 'media_ids': [media_ids.media_id_string]}
api.update_status(**params)

This line:

model_object.image.file.url

Gives me the complete url of the image I want to tweet, something like this:

https://criptolibertad.s3.amazonaws.com/OrillaLibertaria/195.jpg?Signature=xxxExpires=1467645897&AWSAccessKeyId=yyy

I also tried constructing the url manually, since it is a public image stored in my bucket, like this:

filename = "https://criptolibertad.s3.amazonaws.com/OrillaLibertaria/195.jpg"

But it doesn't work.

¿Why do I get the Unable to access file error?

The source code from tweepy looks like this:

def media_upload(self, filename, *args, **kwargs):
    """ :reference: https://dev.twitter.com/rest/reference/post/media/upload
        :allowed_param:
    """
    f = kwargs.pop('file', None)
    headers, post_data = API._pack_image(filename, 3072, form_field='media', f=f)  # ERROR
    kwargs.update({'headers': headers, 'post_data': post_data})


def _pack_image(filename, max_size, form_field="image", f=None):
        """Pack image from file into multipart-formdata post body"""
        # image must be less than 700kb in size
        if f is None:
            try:
                if os.path.getsize(filename) > (max_size * 1024):
                    raise TweepError('File is too big, must be less than %skb.' % max_size)
            except os.error as e:
                raise TweepError('Unable to access file: %s' % e.strerror)

Looks like Tweepy can't get the image from the Amazon S3 bucket, but how can I make it work? Any advice will help.

@BurhanKhalid how should I call it? If I do model_object.image.file.url() I get the error: 'S3BotoStorageFile' object has no attribute 'url' — Alejandro Veintimilla
@BurhanKhalid it works as an attribute like this: model_object.image.url . That gives me the complete url of the image. But I still get the error: Unable to access file. — Alejandro Veintimilla

tutuDajuju tutuDajuju · Accepted Answer · 2016-07-04T20:01:52

The issue occurs when tweepy attempts to get file size in _pack_image:

if os.path.getsize(filename) > (max_size * 1024):

The function os.path.getsize assumes it is given a file path on disk; however, in your case it is given a URL. Naturally, the file is not found on disk and os.error is raised. For example:

# The following raises OSError on my machine
os.path.getsize('https://criptolibertad.s3.amazonaws.com/OrillaLibertaria/195.jpg')

What you could do is to fetch the file content, temporarily save it locally and then tweet it:

import tempfile


with tempfile.NamedTemporaryFile(delete=True) as f:
    name = model_object.image.file.name
    f.write(model_object.image.read())
    media_ids = api.media_upload(filename=name, f=f)
    params = dict(status='test media', media_ids=[media_ids.media_id_string])
    api.update_status(**params)

For your convenience, I published a fully working example here: https://github.com/izzysoftware/so38134984

Django Tweepy can't access Amazon S3 file

1 Answers