Html Agility Pack, SelectNodes from a node

Question

Why does this pick all of my <li> elements in my document?

HtmlWeb web = new HtmlWeb();
HtmlDocument doc = web.Load(url);

var travelList = new List<Page>();
var liOfTravels = doc.DocumentNode.SelectSingleNode("//div[@id='myTrips']")
                     .SelectNodes("//li");

What I want is to get all <li> elements in the <div> with an id of "myTrips".

Answer 1

It's a bit confusing because you're expecting that it would do a selectNodes on only the div with id "myTrips", however if you do another SelectNodes("//li") it will performn another search from the top of the document.

I fixed this by combining the statement into one, but that would only work on a webpage where you have only one div with an id "mytrips". The query would look like this:

doc.DocumentNode.SelectNodes("//div[@id='myTrips'] //li");

Answer 2

var liOfTravels = doc.DocumentNode.SelectSingleNode("//div[@id='myTrips']")
                 .SelectNodes(".//li");

Note the dot in the second line. Basically in this regard HTMLAgitilityPack completely relies on XPath syntax, however the result is non-intuitive, because those queries are effectively the same:

doc.DocumentNode.SelectNodes("//li");
some_deeper_node.SelectNodes("//li");

Answer 3

Creating a new node can be beneficial in some situations and lets you use the xpaths more intuitively. I've found this useful in a couple of places.

var myTripsDiv = doc.DocumentNode.SelectSingleNode("//div[@id='myTrips']");
var myTripsNode = HtmlNode.CreateNode(myTripsDiv.InnerHtml);
var liOfTravels = myTripsNode.SelectNodes("//li");

Answer 4

You can do this with a Linq query:

HtmlWeb web = new HtmlWeb();
HtmlDocument doc = web.Load(url);

var travelList = new List<HtmlNode>();
foreach (var matchingDiv in doc.DocumentNode.DescendantNodes().Where(n=>n.Name == "div" && n.Id == "myTrips"))
{
    travelList.AddRange(matchingDiv.DescendantNodes().Where(n=> n.Name == "li"));
}

I hope it helps

Answer 5

This seems counter intuitive to me aswell, if you run a selectNodes method on a particular node I thought it would only search for stuff underneath that node, not in the document in general.

Anyway OP if you change this line :

var liOfTravels = 
doc.DocumentNode.SelectSingleNode("//div[@id='myTrips']").SelectNodes("//li");

TO:

var liOfTravels = 
doc.DocumentNode.SelectSingleNode("//div[@id='myTrips']").SelectNodes("li");

I think you'll be ok, i've just had the same issue and that fixed it for me. Im not sure though if the li would have to be a direct child of the node you have.