How to prevent python urllib3 from caching response

3
votes

I am polling the same url several times in a row with urllib3. I have profiled the requests and I am getting the header:

Cache-Control: no-transform, max-age=120

On requests after the first I am getting the cached version of the page rather than the request being run again. I have no control over the headers returned from the server, how can I prevent the caching?

I am on Debian Squeeze and this is running python 2.6.6

pool = urllib3.HTTPConnectionPool('itunes.apple.com')
request = pool.request('GET', '/webObjects/MZStore.woa/wa/viewTop?
                       selected_tab_index=0&startIndex=0&genreId=36', 
                       headers = {'Host': 'itunes.apple.com', 
                       'Accept-Encoding': 'gzip, deflate', 'X-Apple-Tz': -18000, 
                       'X-Apple-Store-Front': '143441-1,2')
2
pythoncachinghttp-headersdebian
Please show the call to urllib3 with parameters. - James Thiele

2 Answers

3
votes

urllib3 has no native caching built in. Are you sure it's not the server responding with cached results?

2
votes

Try adding the following headers:

Cache-Control: no-cache

Pragma: no-cache

Expires: Thu, 01 Jan 1970 00:00:00 GMT