15
votes

I'm trying to open a website (I am behind a corporate proxy) using urllib.request.urlopen() but I am getting the error:

urllib.error.HTTPError: HTTP Error 407: Proxy Authentication Required

I can find the proxy in urllib.request.getproxies(), but how do I specify a username and password to use for it? I couldn't find the solution in the official docs.

2
Yeah, but that's for Python2.7 Didn't they restructure the entire urllib package in Python3 3? - Lanaru
They didn't fundamentally change the interface -- just moved things around a bit. ProxyHandler now lives in urllib.request.ProxyHandler - Katriel

2 Answers

27
votes
import urllib.request as req

proxy = req.ProxyHandler({'http': r'http://username:password@url:port'})
auth = req.HTTPBasicAuthHandler()
opener = req.build_opener(proxy, auth, req.HTTPHandler)
req.install_opener(opener)
conn = req.urlopen('http://google.com')
return_str = conn.read()
1
votes

You can set the proxy server authentication with your credentials(username and password) to connect to the website using requests. This worked for me. To get your proxy server name: use

import urllib.request as req
import os
#get your proxy server url details using below command
req.getproxies() 

#user your credentials and url to authenticate

os.environ['http_proxy'] = "http://username:pwd@url:80"
os.environ['https_proxy'] = "http://username:pwd@url:80"

#replace username, pwd and url with your credentials.

conn = req.urlopen('https://Google.com')
return_str = conn.read()