Python WebDriver how to print whole page source (html)

19
votes

I'm using Python 2.7 with Selenium WebDriver. My question is how to print whole page source with print method. There is webdriver method page_source but it returns WebDriver and I don't know how to convert it to String or just print it in terminal

2
pythonselenium-webdriverwebdriver

2 Answers

39
votes

.page_source on a webdriver instance is what you need:

>>> from selenium import webdriver
>>> driver = webdriver.Firefox()
>>> driver.get('http://google.com')
>>> print(driver.page_source)
<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml" lang="en" itemtype="http://schema.org/WebPage" itemscope=""><head><meta name="descri
...
:before,.vscl.vslru div.vspib{top:-4px}</style></body></html>
-3
votes

You can also get the HTML page source without using a browser. The requests module allows you to do that.

 import requests

 res = requests.get('https://google.com')
 res.raise_for_status()  # this line trows an exception if an error on the 
                         # connection to the page occurs. 
 print(res.text)