HtmlAgilityPack - XPath: Select Sequence of Nodes After Specific Node until A Node with Specific Descendant

Question

I want to use XPath to select a p tag which contains a strong child tag and place it as the key in a key value pair. The value of which I want to be following p tags until it hits the next strong tag.

Unfortunately the HTML I am dealing with is not my own so I cannot modify its structure to make this simpler. I see several examples of using XPath in this way if the text is known, but in this case the specific text is variable.

Here is the relevant part of the HTMl simplified:

<div class="div_1"> 
 <div class="div_2">
   <p><em><strong>Title 1</strong></em> Some Text</p>
   <p>Some Text <a class="tooltip">Some Text</a></p>
   <p>Some Text <a class="tooltip">Some Text</a></p>
   <p>Some Text <a class="tooltip">Some Text</a></p>
   <p><em><strong>Title 2</strong></em> Some Text.</p>                
  </div>
</div>

Here is the approach I was trying in VB:

For Each trait_head As HtmlAgilityPack.HtmlNode In content.DocumentNode.SelectNodes(
        "//div[@class='div_1']/div[@class='div_2']/p/em/strong")
            trait_heading = trait_head.InnerText
            trait_heading = trait_heading.Trim().Replace(vbCr, "").Replace(vbLf, "")
            For Each trait_bod As HtmlAgilityPack.HtmlNode In content.DocumentNode.SelectNodes(
            "//div[@class='div_1']/div[@class='div_2']/p")
                If trait_body Is Nothing Then
                    trait_body = trait_bod.InnerText
                Else
                    trait_body = trait_body & vbCr & vblf & trait_bod.InnerText
                End If
            Next
trait_value.add(New KeyValuePair(Of String, String)(trait_heading, trait_body))
Next

So what I need to modify is the second XPath statement so that the for loop breaks once it hits that second p tag with the strong.

Looking for this result:
trait_value = "Title 1" => "Some text vbcr vblf Some text vbcr vblf Some text vbcr vblf Some text vbcr vblf","Title 2" => "Some text"

Hopefully what I am asking here is possible just using XPath, but if anyone has suggestions on a different approach entirely I would be happy to hear them.

fpsthirty fpsthirty · Accepted Answer · 2019-01-08T18:04:34

Final result:

For Each trait_head As HtmlAgilityPack.HtmlNode In content.DocumentNode.SelectNodes(
        "//div[@class='div_1']/div[@class='div_2']/p/em/strong")
            trait_heading = trait_head.InnerText
            trait_heading = trait_heading.Trim().Replace(vbCr, "").Replace(vbLf, "")
            For Each trait_bod As HtmlAgilityPack.HtmlNode In content.DocumentNode.SelectNodes(
            "//div[@class='div_1']/div[@class='div_2']/p[em/strong]")
                If trait_body Is Nothing Then
                    trait_body = trait_bod.LastChild
                Else
                    trait_body = trait_body & vbCr & vblf & trait_bod.LastChild
                End If
            Next
trait_value.add(New KeyValuePair(Of String, String)(trait_heading, trait_body))
Next

HtmlAgilityPack - XPath: Select Sequence of Nodes After Specific Node until A Node with Specific Descendant

1 Answers