Selenium - Identifying an HTML element without an ID or CSS class name

0
votes

I'm using Selenium WebDriver in Python 3 to scrape this webpage/relevant HTML (Pastebin) and am trying to get a list of prices in the "Departing flights" div. This div does not have an ID nor does it have a CSS class name, so I tried identifying it via XPath. On the Pastebin, to find the div that I'm talking about, Ctrl + F <div class="">.

Here is my code:

outbound_fares = browser.find_elements_by_xpath("//*[@id='air-booking-product-0']/div[4]")
outbound_prices = outbound_fares.find_elements_by_class_name("currency_dollars")

The first line should identify the div and the second line should get a list of all the prices within that div. However, I find that outbound_prices is empty. My guess is that the first line is wrong, but I'm not sure how to properly identify the div. Thanks.

2
htmlpython-3.xseleniumxpathweb-scraping
I see "Access denied" message on the link that you have provided. Could you include the relevant part of HTML in your question? - K. B.
@K.B. Sure. I have edited the OP with a Pastebin link: pastebin.com/Bj8w4J4v - Warren Crasta

2 Answers

0
votes

You can use this cssSelector and get the all prices from Depart flights :

cssSelector : 

div[class='fare-button--hybrid'] span>span[class*='screen-reader']

try out this cssSelector and let me know the status. Please let me know if you have any concerns related to this.

0
votes

The element you want doesn't have an ID but I don't know what you mean by "does not have a CSS selector." Another locator may be more efficient at finding it, etc. but every element on the page can be found by a CSS selector.

You can use this CSS selector to return all the prices on the page.

span.fare-button--value-total

Here are some references to read to learn more about CSS selectors and how to use them.

https://www.w3.org/TR/selectors/#overview

https://saucelabs.com/resources/articles/selenium-tips-css-selectors

https://www.smashingmagazine.com/2009/08/taming-advanced-css-selectors/