python - opening websites using urllib2 from behind corporate firewall - 11004 getaddrinfo failed

6
votes

I am trying to access a website from behind corporate firewall using below:- 

password_mgr = urllib2.HTTPPasswordMgrWithDefaultRealm()
password_mgr.add_password(None, url, username, password)
auth_handler = urllib2.HTTPBasicAuthHandler(password_mgr)
opener = urllib2.build_opener(auth_handler) 
urllib2.install_opener(opener) 
conn = urllib2.urlopen('http://python.org')

Getting error 

URLError: <urlopen error [Errno 11004] getaddrinfo failed>

I have tried with different handlers (tried ProxyHandler also in slightly different way), but doesn't seem to work.

Any clues to what could be the reason for error and any different ways to supply the credentials and make it work?

2
pythonurllib2firewall
What kind of auth does the proxy want? If it demands Proxy-Authenticate: NTLM then it's impossible to make it work with Python, I'm sorry to say. - Colonel Panic

2 Answers

5
votes

If you are using Proxy and that proxy has Username and Password (which many corporate proxies have), you need to set the proxy handler with urllib2.

  proxy_url = 'http://' + proxy_user + ':' + proxy_password + '@' + proxy_ip
  proxy_support = urllib2.ProxyHandler({"http":proxy_url})
  opener = urllib2.build_opener(proxy_support,urllib2.HTTPHandler)
  urllib2.install_opener(opener)

HTTPBasicAuthHandler is used to provide credentials for the site which you are going to access and not for going through the proxy. The above snippet might help you.

4
votes

On Windows, I observed that python uses the IE Internet Options-> LAN Settings settings. So even if we use urllib2 to install opener and specify the proxy_url, it would continue to use the IE settings.

It worked fine finally, when I exported a system variable:

http_proxy=http://userid:[email protected]:port