3
votes

I have follow this tutorial but I still fail to get output. Below is my code in view.py

def index(request):

 #html="a"
 #url= requests.get("https://www.python.org/")
 #page = urllib.request.urlopen(url)
 #soup = BeautifulSoup(page.read())
 #soup=url.content
 #urllib3.disable_warnings()
 #requests.packages.urllib3.disable_warnings(InsecureRequestWarning)
 #url=url.content
 #default_headers = make_headers(basic_auth='myusername:mypassword')
 #http = ProxyManager("https://myproxy.com:8080/", headers=default_headers)

 r = urllib.request.urlopen('http://www.aflcio.org/Legislation-and-Politics/Legislative-Alerts').read()
 soup = BeautifulSoup(r)
 url= type(soup)

 context={"result":url,}
 return render (request, 'index.html',context)

Output:

urlopen error [WinError 10060] A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond

1
It means that there is a network issue and doesn't have anything to do with Django. It could be that the web site in question (aflcio.org) is actively preventing you from scraping it. - Selcuk
if I replace with google.com it show me the same error, any way to overcome this? - Shi Jie Tio
Edit your question to include the full error message with traceback, but I highly doubt that you have a networking issue. Is this a virtual machine? - Selcuk
in future I will include in virtual machine, but now I running on my pc - Shi Jie Tio
so any suggestion? - Shi Jie Tio

1 Answers

2
votes

If you are sitting behind a firewall or similar you might have to specify a proxy for the request to get through.

See below example using the requests library.

import requests

proxies = {
  'http': 'http://10.10.1.10:3128',
  'https': 'http://10.10.1.10:1080',
}

r = requests.get('http://example.org', proxies=proxies)