The weird thing here is that somehow HtmlAgilityPack does not recognize the id attribute of the "Next" link.
This could be a bug in HtmlAgilityPack; you can post it in the HAP Issue Tracker.
However, in the meantime I found this workaround:
- find the table that contains the paging elements (the table with
id="nav"). For this element the id is correctly recognized
- take the first (and only
tr) in the table and the last td of it (using the XPath last() function)
- take the
a element inside the td we obtained at the previous step.
Long story short, here's the code:
var doc = new HtmlWeb().Load("http://www.google.com/search?q=seo");
var nextLink = doc.DocumentNode
.SelectSingleNode("//table[@id='nav']/tr/td[last()]/a");
Console.WriteLine(nextLink.GetAttribute("href", "err"));
Update
After Simon's comment I checked this again and the conclusion is that this is not a bug in HTML Agility Pack; the id="pnnext" attribute is only present when the request is made by a browser (perhaps depending on the UserAgent header value). When doing an HttpWebRequest from code, this is how the "Next" link appears in the output:
<a href="/search?q=seo&hl=en&ie=UTF-8&[...]" style="text-align:left">
??- MiMo