HtmlAgilityPack DocumentNode.SelectNodes returns null, shouldn't

Question

I'm trying to scrape content from an example page using the HTML agility pack. The DocumentNode.SelectNodes is returning null for an XPath query when I think it shouldn't. Could someone tell me why? The code is:

HtmlDocument doc = new HtmlDocument();
string xpath = "//h1[@class='product-title fn']"; // note, it still returns 
                                                  // null even with "//div"
doc.OptionFixNestedTags = true;
HtmlNode.ElementsFlags.Remove("form");
HtmlNode.ElementsFlags.Remove("option");

HtmlNodeCollection coll = doc.DocumentNode.SelectNodes(xpath);

if (coll != null)
{
    // do stuff
}
else
{
    // not expecting it to be null unless no matches
}

Doh, forgot to load in the doc.Load url in my UnitTest, hence the error. If someone could delete this question that'd be much appreciated. Thanks, Colin. — Colin Brown
You can delete it. See that little delete link under the tags? — Oded
Side note: When no nodes are (legitimately) found, then (as you know) you get null instead of an empty enumerable. This is annoying when iterating over the result, but can be worked around like this: foreach (var node in doc.DocumentNode.SelectNodes("...") ?? Enumerable.Empty<HtmlNode>()) ... — Cameron

Tim Abell Tim Abell · Accepted Answer · 2012-04-24T13:58:36

According to the upstream bug comments it is for consistency:

DarthObiwan wrote Jan 11 2011 at 9:27 PM

This has been covered before, this function is written to mimic the way the System.XML works. Doing so will cause a major breaking change and thus will probably be slated for 2.0

HtmlAgilityPack DocumentNode.SelectNodes returns null, shouldn't

1 Answers