Whats the difference between getting text and innerHTML when using selenium. Even though we have text under particular element, when we perform .text we get empty values. But doing .get_attribute("innerHTML") works fine.
Can someone point out the difference between two? When someone should use '.get_attribute("innerHTML")' over .text?
To start with, text is a property where as innerHTML is an attribute. Fundamentally there are some differences between a property and an attribute.
get_attribute(innerHTML) gets the innerHTML of the element.
This method will first try to return the value of a property with the given name. If a property with that name doesn’t exist, it returns the value of the attribute with the same name. If there’s no attribute with that name, None is returned.
Values which are considered truthy, that is equals true or false, are returned as booleans. All other non-None values are returned as strings. For attributes or properties which do not exist, None is returned.
Args:
innerHTML - Name of the attribute/property to retrieve.
Example:
# Extract the text of an element.
my_text = target_element.get_attribute("innerHTML")
text gets the text of the element.
Definition:
def text(self):
"""The text of the element."""
return self._execute(Command.GET_ELEMENT_TEXT)['value']
Example:
# Extract the text of an element.
my_text = target_element.text
Still sounds similar? Read below ...
When the browser loads the page, it parses the HTML and generates DOM objects from it. For element nodes, most standard HTML attributes automatically become properties of DOM objects.
For instance, if the tag is:
<body id="page">
then the DOM object has body.id="page".
Note: The attribute-property mapping is not one-to-one!
In HTML, tags may have attributes. When the browser parses the HTML to create DOM objects for tags, it recognizes standard attributes and creates DOM properties from them.
So when an element has id or another standard attribute, the corresponding property gets created. But that doesn’t happen if the attribute is non-standard.
Note: A standard attribute for one element can be unknown for another one. For instance,
typeis standard attribute for<input>tag, but not for<body>tag. Standard attributes are described in the specification for the corresponding element class.
So, if an attribute is non-standard, there won’t be a DOM-property for it. In that case all attributes are accessible by using the following methods:
elem.hasAttribute(name): checks for existence.elem.getAttribute(name): gets the value.elem.setAttribute(name, value): sets the value.elem.removeAttribute(name): removes the attribute.An example of reading a non-standard property:
<body something="non-standard">
<script>
alert(document.body.getAttribute('something')); // non-standard
</script>
</body>
When a standard attribute changes, the corresponding property is auto-updated, and (with some exceptions) vice versa. But there are exclusions, for instance input.value synchronizes only from attribute -> to property, but not back. This feature actually comes in handy, because the user may modify value, and then after it, if we want to recover the "original" value from HTML, it’s in the attribute.
As per Attributes and Properties in python when we reference an attribute of an object with something like someObject.someAttr, Python uses several special methods to get the someAttr attribute of the object. In the simplest case, attributes are simply instance variables.
In a broader perspective:
someObj.name.__dict__ of an object.someObj.name, the default behavior is effectively someObj.__dict__['name']In Python we can bind getter, setter (and deleter) functions with an attribute name, using the built-in property() function or @property decorator. When we do this, each reference to an attribute has the syntax of direct access to an instance variable, but it invokes the given method function.
For instance, <div><span>Example Text</span></div>
.get_attribute("innerHTML") gives you the actual HTML inside the current element. So theDivElement.get_attribute("innerHTML") returns "<span>Example Text</span>"
.text gives you only text, not include HTML node. So theDivElement.text returns "Example Text"
Please note that the algorithm for .text depends on webdriver of each browser. In some cases such as element is hidden, you might get different text when you use different webdriver.
I usually get text from .get_attribute("innerText") instead of .text so I can handle the all the case.
Chrome (i'm not sure about other browsers) ignores the extra spaces within the HTML code and displays as a single space.
<div><span>Example Text</span></div> # notice the two spaces
.get_attribute('innerHTML') will return the double-spaced text, which is what you would see when you inspect element), while .text will return the string with only 1 space.
>>> print(element.get_attribute('innerHTML'))
'Example Text'
>>> print(element.text)
'Example Text'
This difference is not trivial as the following will result in a NoSuchElementException.
>>> arg = '//div[contains(text(),"Example Text")]'
>>> driver.find_element_by_xpath(arg)
Similarly, .get_attribute('innerHTML') for the following returns Example Text, while .text returns Example Text.
<div><span>Example Text</span></div>
I have just selected the css selector and used below code:
from selenium import webdriver
driver = webdriver.Chrome()
driver.maximize_window()
driver.get("http://www.costco.com/Weatherproof%C2%AE-Men's-Ultra-Tech-Jacket.product.100106552.html")
print driver.find_element_by_css_selector(".product-h1-container.visible-xl-block>h1").text
and it prints:
Weatherproof® Men's Ultra Tech Jacket
The problem is h1[itemprop='name'] selector on chrome or firefox are returning 2 matching nodes while .product-h1-container.visible-xl-block>h1 is returning only one matching node thats why its prining what is expected
To prove my point run below code:
from selenium import webdriver
driver = webdriver.Chrome()
driver.maximize_window()
driver.get("http://www.costco.com/Weatherproof%C2%AE-Men's-Ultra-Tech-Jacket.product.100106552.html")
x= driver.find_elements_by_css_selector("h1[itemprop='name'] ")
for i in x:
print "This is line " , i.text
It will print
This is line
This is line Weatherproof® Men's Ultra Tech Jacket
Because select_element_by_css_selector selects the first element with matching selector and that does not contain any text so it does not print. Hope you understand now
<p>This is demo</p>"while .text will only retrieve all text content of its descendants without any HTML tags.example: "This is demo" – thebadguydriver.find_element_by_css_selector("p").textwill yield nothing. but doing driver.find_element_by_css_selector("p").get_attribute("innerHTML") will result in extractingThis is demo....why is that behavior? – Vivek Srinivasan"http://www.costco.com/Weatherproof%C2%AE-Men's-Ultra-Tech-Jacket.product.100106552.html"Tried getting product title using the following linedriver.find_element_by_css_selector("h1[itemprop='name']").textyielded nothing....but driver.find_element_by_css_selector("h1[itemprop='name']").get_attribute("innerHTML") gets me the product title"Weatherproof\xae Men's Ultra Tech Jacket"– Vivek Srinivasan