1
votes

I ran into this problem for a couple of days and I don't know how to solve it: Let's pretend that I have to connect to the https://targetwebsite.com which is only accessible through proxy server with authentication and that I would like to make python requests to this website. What I have done so far has been:

import requests
from requests.auth import HTTPProxyAuth

session = requests.Session()
session.proxies = {'http': 'proxy_url:port'}
session.auth = HTTPProxyAuth('proxy_username', 'proxy_password')
session.get('https://targetwebsite.com')
>> <Response [404]>

It doesn't use the proxy, in fact if I try to get the IP through API the result is that I get back my current IP address and not the IP of the proxy

...
session.get('https://api.ipify.org/?format=json')
>> <Response [200]>
>> {"ip":"XXX.XXX.XXX.XX"} // IP address of my machine

However if I make a request to the same website but via http it works:

session.get('http://targetwebsite.com')
>> <Response [200]>

I have to say that I'm using a proxy server that I created and it is hosted on a raspberry pi via Squid, so maybe I was wrong to configure it, however I don't think it's the problem because when I use the same proxy with the same credentials with Firefox I can access the websites also via https.

As someone suggested I tried also different configurations with this results:

...
session.proxies = {'http': 'proxy_url:port', 'https': 'proxy_url:port'}
session.get('https://targetwebsite.com')
>> <407 Proxy Authentication Required>
...
session.proxies = {'https': 'proxy_url:port'}
session.get('https://targetwebsite.com')
>> <407 Proxy Authentication Required>
1
I guess you cant access the https site because you are using a http proxy. Try to change the proxy to https and see if it works - Aaron

1 Answers

0
votes

You need to specify HTTPS proxy:

session.proxies = {'https': 'proxy_url:port'}

Then the session can be used to access HTTPS targets, which is all targets nowadays.