I am trying to scrape items from page using selenium. Currently I am stuck at scraping text from item tooltip.
Html that I see using developer tools (f12) for one item in a page:
<div class="offer-page-sale-item offer-page-sale-item__actual" data-real-id="12345" data-id="12345-4">
<!-- Render product -->
<div class="item ">
<div class="discount price bare">
<div class="t1">
<span class="value">1</span>
<span class="cents">69</span>
<span class="eur">€</span>
</div>
</div>
<div class="tags tags_primary">
<div class="tag i tooltipstered"></div>
</div>
<div class="img"></div>
<div class="text">
<div class="title">[name of the product]</div>
<div class="pack">
<span class="ptitle"></span>
<span class="price"> </span>
</div>
</div>
<div class="infopop">
<div class="in"><span></span>
<div class="arrow_top"></div>
<div class="arrow_bottom"></div>
</div>
</div>
</div>
</div>
If I look into html from page source (ctrl+u) for the same item I see:
<div class="offer-page-sale-item offer-page-sale-item__actual" data-real-id="12345" data-id="12345-4">
<!-- Render product -->
<div class="item ">
<div class="discount price bare">
<div class="t1">
<span class="value">1</span>
<span class="cents">69</span>
<span class="eur">€</span>
</div>
</div>
<div class="tags tags_primary">
<div title="[product price valid from]-[product price valid until]" class="tag i"></div>
</div>
<div class="img"></div>
<div class="text">
<div class="title">[name of the product]</div>
<div class="pack">
<span class="ptitle"></span>
<span class="price"> </span>
</div>
</div>
<div class="infopop">
<div class="in"><span></span>
<div class="arrow_top"></div>
<div class="arrow_bottom"></div>
</div>
</div>
</div>
</div>
So the only difference is inside <div class="tags tags_primary">
tag.
Because I can actually can see the text when using page source, I imagine I should be able to capture it?
However, Selenium driver gives me only class="tag i tooltipstered"
tag, and not class="tag i"
which has title
attribute I need.
I have tried to:
- use
Actions class MoveToElement()
but still could not find tooltip title. - use
IJavaScriptExecutor
to getinnerHtml
of<div class="tags tags_primary">
without luck.
If someone has any ideas it would be greatly appreciated.
UPD: Rewrited @PDHide phyton code to c#. Created extension for IWebDriver:
public static IWebElement WaitUntilVisible(this IWebDriver driver, By itemSpecifier, int secondsTimeout = 10)
{
var wait = new WebDriverWait(driver, new TimeSpan(0, 0, secondsTimeout));
var element = wait.Until<IWebElement>(driver =>
{
try
{
var elementToBeDisplayed = driver.FindElement(itemSpecifier);
if (elementToBeDisplayed.Displayed)
{
return elementToBeDisplayed;
}
return null;
}
catch (StaleElementReferenceException)
{
return null;
}
catch (NoSuchElementException)
{
return null;
}
});
return element;
}
Then used it to collect text from tooltip:
using(IWebDriver _driver = new ChromeDriver())
{
_driver.Navigate().GoToUrl("https://www.maxima.lt/akcijos");
_driver.WaitUntilVisible(By.CssSelector("#CybotCookiebotDialogBodyLevelButtonLevelOptinAllowallSelectionWrapper #CybotCookiebotDialogBodyLevelButtonLevelOptinAllowAll")).Click();
_driver.WaitUntilVisible(By.ClassName("close")).Click();
var tool = _driver.WaitUntilVisible(By.CssSelector("[class='tag i tooltipstered']"));
Actions actions = new Actions(_driver);
actions.MoveToElement(tool).Build().Perform();
var tooltip = _driver.WaitUntilVisible(By.ClassName("tooltipster-content"));
Console.WriteLine(tooltip.Text);
}