1
votes

In my script i am trying to make a request with q proxy server. I simply do:

import requests

response = requests.get('https://websiteiwhantget', proxies={"http": '176.36.111.9:56323', "https": '176.36.111.9:56323'})

the proxies ip addres i get from https://free-proxy-list.net/ but when i run the sript, with every website i put in my get call i have:

raise ProxyError(e, request=request) requests.exceptions.ProxyError: HTTPSConnectionPool(host='www.moma.com', port=443): Max retries exceeded with url: / (Caused by ProxyError('Cannot connect to proxy.', RemoteDisconnected('Remote end closed connection without response',)))

if i remove proxies directive in requests.get all was done. Why with proxy my script does not work? are the proxies listed in the free-proxy-list wrong or i have to change my python call? i use python 3.6

So many thanks in advance AM

1
If you don't mind paid proxy solution, you can try using gimmeproxy.com, it will verify all the proxies in their database. - Andrey E

1 Answers

3
votes

Read this: https://www.scrapehero.com/how-to-rotate-proxies-and-ip-addresses-using-python-3/

And try this:

    import requests
    import random
    from lxml.html import fromstring

    url = 'https://free-proxy-list.net/anonymous-proxy.html'
    response = requests.get(url)
    parser = fromstring(response.text)
    proxies = []
    for i in parser.xpath('//tbody/tr')[:20]:
        if i.xpath('.//td[7][contains(text(),"yes")]'):
            proxy = ":".join([i.xpath('.//td[1]/text()')[0], i.xpath('.//td[2]/text()')[0]])

        try:
            t = requests.get("https://www.google.com/", proxies={"http": proxy, "https": proxy}, timeout=5)
            if t.status_code == requests.codes.ok:
                proxies.append(proxy)
        except:
            pass

    proxy = proxies[random.randint(0, len(proxies)-1)]

    response = requests.get('https://websiteiwhantget', proxies={"http": proxy, "https": proxy})