data get requests from a website with unsupported browser error

0
votes

i have basically tried everything but still i get the unsupported browser error. i use requests module. i have tried with headers, other modules but still the same. 

import requests
url = "https://www.ratemyagent.com.au/"
response = requests.get(url)
html_icerigi = response.text
soup = BeautifulSoup(html_icerigi, "html.parser")

https://www.ratemyagent.com.au/ this is the adres. so please if you have any idea to get rid of this error let me know.

thanks a lot in advance

edit: this is what i have tried as headers :

user_agent = 'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.101 Safari/537.36' headers = { 'User-Agent' : user_agent } response = requests.get(url, headers=headers)

versions // python 3. BeautifulSoup4: 4.6.0 requests:2.20 [this is the output with soup.text or reponse.text][1]

1
pythonweb-scrapingpython-requests
Can't reproduce error, receiving HTTP 200 from the request - kstullich
but when you print the text just unsupported browser - uestadeus
What headers have you tried? Specifically, what User-Agent headers? - John Gordon
user_agent = 'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.101 Safari/537.36' headers = { 'User-Agent' : user_agent } response = requests.get(url, headers=headers) - uestadeus
this is what i have tried - uestadeus

1 Answers

0
votes

it's not a error

when you normally search https://www.ratemyagent.com.au/ by chrome browser, the page resource your get also has the info like you said:

enter image description here


while the resource your get by requests module is same as what you get from a web browser I don't think it is a error.