Django-compressor / django-storages links being wrongly cached; expiring

Question

I have a Heroku setup with django-compressor compressing my CSS and JS files. My default cache backend is set up with memcached (actually Memcachier when on Heroku). My static files are being served on an Amazon S3 instance (handled by django-storages and boto). Generally my setup is in line with this one from the django-compressor docs.

In my pages, the links to my compressed files have an "expires" query that seems to be an hour from when the link is generated. I don't want to set there to be no expiration for all CSS or JS requests because I have some CSS and JS that I don't compress (it gives me errors when I try... probably a different question).

However, once an hour the links break and the site has no more styles or JS. It looks like the link, or at least the expires header, is being cached and not regenerated, so after the time in the expires argument passes, Amazon no longer returns the file. If I flush the memcache, it changes the expires header in the link to an hour from then, which fixes the problem for an hour, until it expires again.

Here is an example of a link that was generated around 1:39 PM EST today, Tuesday, September 18: https://zen180-static.s3.amazonaws.com/CACHE/css/68d31794a646.css?Signature=u%2FTxeF7LBBZTDV79YovOjoK2tcw%3D&Expires=1347993542&AWSAccessKeyId=AKIAIJ7VKLEX7HIMJZCA. After it is first generated, the page continues to serve that link without alteration. Around 2:39 EST (i.e. the Unix time 1347993542, from the expires argument in the URL), that link stopped working and sent back an "Expired" XML message (the same one it does now). But the bad link was still there on my page until I flushed memcache.

Here are the relevant settings:

COMPRESS_ENABLED = True

STATICFILES_STORAGE = '[my app's name].storage.CachedS3BotoStorage' (a subclass of S3BotoStorage that is almost identical to the one from django-compressor's docs linked above)

COMPRESS_STORAGE = STATICFILES_STORAGE

COMPRESS_URL = STATIC_URL

CACHES = {
  'default': {
    'BACKEND': 'django_pylibmc.memcached.PyLibMCCache',
    'LOCATION': 'localhost:11211',
    'TIMEOUT': 500,
    'BINARY': True,
  }
}

Here is the error:

<Error>
<Code>AccessDenied</Code>
<Message>Request has expired</Message>
<RequestId>81A63F24378ECB5E</RequestId>
<Expires>2012-09-18T18:39:02Z</Expires>
<HostId>lIr5l9Fna95DUfk6hUsWqhO5EQNn6Ayu8BatpEavis8YzLvsaJRru4O8P/50pgMy</HostId>
<ServerTime>2012-09-18T18:59:51Z</ServerTime>
</Error>

have you considered enabling public access to your static files on S3? — Julian
Yes, that's what I ultimately did. But I'd like to have the expires headers so the browser can cache the page. — Daniel Robinson

patrys patrys · Accepted Answer · 2012-09-27T18:04:23

If you want the browser to cache your static content, set the settings.AWS_HEADERS accordingly (as outlined in Amazon's best practices):

AWS_HEADERS = {
    'Expires': 'Sun, 19 Jul 2020 18:06:32 GMT'
}

If you really need the link to be regenerated, find the place it is stored in memcache and set the same cache time you send as link expiration time to Amazon (judging from django-compressor docs it should be one of settings.COMPRESS_*_DELAY).

Django-compressor / django-storages links being wrongly cached; expiring

1 Answers