Problems with chromedriver on Selenium with python: page never loads

0
votes

I originally wanted to get a headless browser running to dynamically parse a web page. I've been having a series of problems with this, even after adding in waits to allow website to load. After commenting out the headless option, I used an xming server as a display for the web page to see what it actually looks like after the browser is created.
Here's a screenshot of what I see: a new tab in chrome, and the browser seems to never get to the .get(). I also get a popup saying "Chrome Automation Extension has crashed..."
I've been searching a lot, some suggest not running as admin/root, so I didn't use sudo before running it. That didn't change message.

Below is my code and the terminal output:

options = webdriver.ChromeOptions()
# options.add_argument('--headless')
options.add_argument('--no-sandbox')
options.add_argument('--disable-dev-shm-usage')
options.add_argument('--ignore-certificate-errors')
options.add_argument('--lang=en_US')
options.add_argument('--user-data-dir')
options.add_argument("--disable-gpu")
# options.add_argument("--remote-debugging-port=9222")

browser = webdriver.Chrome(executable_path='/usr/bin/chromedriver', options=options)
browser.get("http://www.google.com")

Ubuntu terminal:

Traceback (most recent call last):
File "practice3.py", line 40, in
browser = webdriver.Chrome(executable_path='/usr/bin/chromedriver', options=options)
File "etc/anaconda3/lib/python3.8/site-packages/selenium/webdriver/chrome/webdriver.py", line 76, in init
RemoteWebDriver.init(
File "etc/anaconda3/lib/python3.8/site-packages/selenium/webdriver/remote/webdriver.py", line 157, in init
self.start_session(capabilities, browser_profile)
File "etc/anaconda3/lib/python3.8/site-packages/selenium/webdriver/remote/webdriver.py", line 252, in start_session
response = self.execute(Command.NEW_SESSION, parameters)
File "etc/anaconda3/lib/python3.8/site-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute
self.error_handler.check_response(response)
File "etc/anaconda3/lib/python3.8/site-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: unknown error: DevToolsActivePort file doesn't exist
(Driver info: chromedriver=2.41.578700 (2f1ed5f9343c13f73144538f15c00b370eda6706),platform=Linux 4.4.0-18362-Microsoft x86_64)

Also:
Ubuntu 20.04 LTS
Selenium version 3.141.0
Chrome version 84.0.4147.105

~$ /usr/bin/chromedriver
Starting ChromeDriver 2.41.578700 (2f1ed5f9343c13f73144538f15c00b370eda6706) on port 9515
Only local connections are allowed.
1
pythonseleniumparsingselenium-webdriverselenium-chromedriver
I would run without the options just to troubleshoot a bit. Seems like you've got some options set there that are normally used with headless, but you commented out the headless part. Btw, you won't need waits after a get... Selenium will already wait for the page to load (ready state) after a get method is called. Also unless there's a specific reason you should not use the "no-sandbox" tag. – pcalkins
also the "--user-data-dir" argument seems to be setting this to nothing? Why are you setting that one? – pcalkins
I just tried no options and I get "chrome failed to start: crashed" and "DevToolsActivePort file doesn't exist". When I run with just "--no-sandbox" the browser get the page link, but the page never loads anything (can't interact with elements). Also I think ur right about not needing majority of options. Those I collected sorta randomly from other answers when I was really lost. Thanks! – etcTryAgain
that's a really old version of chromedriver there...(didn't notice that before) Eduardo's answer seems spot on. – pcalkins

1 Answers

1
votes

I've seen lots of chrome crashes when there was version mismatch between chromedriver and chrome. Since you're using Chrome version 84.0.4147.105, you should be using nothing but ChromeDriver 84.0.4147.30 Second thing, are you launching the script from an IDE ? if positive that is also another common root cause. Finally I'd stop using XMimg since it adds another variable to the equation. Instead, troubleshoot taking screenshots directly from selenium code regardless you are headless or not.