1
votes

I am trying to make some HTTP request using a HTTP proxy server over Tor. So far, I have experimented with Privoxy, Tinyproxy and Polipo. But whenever I make a request through any of these proxy servers, they somehow fetch themselves, and get stuck in a loop.

To check if there is any problem with DNS or network, I tried to fetch URLs without proxy and it works fine. I am using Ubuntu 12.04 64bit servers, and using python with requests module to test the setup.

Moreover, I am experiencing this problem on a server with two static IPv4 Addresses, on a server with one static IPv4 address, it works fine. I tried to simulate this on an EC2 server, there also it works on server with Elastic IP address, but on the one without Elastic IP Address, I experience the same problem.

I tried to dig deeper by editing the source code of Tinyproxy to see why the error happens, and its because the proxy server thinks the "Host" is 127.0.0.1 for the request. If I specify "Host" in the request headers, then it works somewhat fine. But that isn't normal behaviour.

If, it could have been only a single Proxy server, then I could have thought of it as a software bug. But the problem is present with all of the proxy servers, and it makes me wonder if there is something wrong with network setup

Here is how I am trying to make request

import requests
# tinyproxy listens on Port 8888
r = requests.get("http://google.com", proxies={'http': '127.0.0.1:8888'}) 

And I get the following response

 400 Bad Request <p>You tried to connect to the machine the proxy is running on </p>
1
can you tag the program language you used ? - zb'
I not sure about tiniproxy, but Privoxy and Polipo are socks proxy, not http/https. - zb'
Privoxy and Polipo are very much HTTP Proxy servers, they can forward to a socks proxy though, and thats why they used to come bundled with TOR. Thanks for the above link though - Pankaj
Problem solved! Turns out it is an issue with the latest version of requests library which I am using. (github.com/kennethreitz/requests/pull/1060) . Thanks @eicto for the indirect help - Pankaj

1 Answers

0
votes

Turns out it is an issue with the latest version of requests library. https://github.com/kennethreitz/requests/pull/1060