HTML Agility Pack Question (Attempting to parse string from source)

Question

I am attempting to use the Agility pack to parse certain bits of info from various pages. I am kind of worried that using this might be overkill for what I need, if that is case feel free to let me know. Anyway, I am attempting to parse a page from motley fool to get the name of a company based on the ticker. I will be parsing several pages to get stock info in a similar way.

The HTML that I want to parse looks like:

<h1 class="subHead"> 
    Microsoft Corp <span>(NASDAQ:MSFT)</span>
</h1>

Also, the page I want to parse is: http://caps.fool.com/Ticker/MSFT.aspx

So, I guess my question is how do I simply get the Microsoft Corp from the html and should I even be using the agility pack to do things like this?

Edit: Current code

public String getStockName(String ticker)
{
    String text ="";
    HtmlAgilityPack.HtmlWeb web = new HtmlAgilityPack.HtmlWeb();
    HtmlAgilityPack.HtmlDocument doc = web.Load("http://caps.fool.com/Ticker/" + ticker + ".aspx");

    var node = doc.DocumentNode.SelectSingleNode("/h1[@class='subHead']");
    text = node.FirstChild.InnerText.Trim();
    return text;
}

I find the HTML Agility Pack is great for situations like this - i.e. where there is no data source provided, but you would still like to parse 3rd-party data for some reason. I've done something quite similar to what you're working on in the past. Next step is to parse out the price, daily change etc. :) — Chris McAtackney
@C.McAtackney Thanks! Yep, this one is just the start, but I think once I get my feet wet it will not be as rough. I appreciate you letting me know that I am on the right track. — PFranchise

BrokenGlass BrokenGlass · Accepted Answer · 2011-04-10T21:03:13

This would give you a list of all stock names, for your sample Html just of Microsoft:

HtmlAgilityPack.HtmlDocument doc = new HtmlAgilityPack.HtmlDocument();
doc.Load("test.html");

var nodes = doc.DocumentNode.SelectNodes("//h1[@class='subHead']");
foreach (var node in nodes)
{
    string text = node.FirstChild.InnerText; //output: "Microsoft Corp"
    string textAll = node.InnerText; //output: "Microsoft Corp (NASDAQ:MSFT)"
}

Edit based on updated question - this should work for you:

string text = "";
HtmlWeb web = new HtmlWeb();

string url = string.Format("http://caps.fool.com/Ticker/{0}.aspx", ticker);
HtmlAgilityPack.HtmlDocument doc = web.Load(url);

var node = doc.DocumentNode.SelectSingleNode("//h1[@class='subHead']");
text = node.FirstChild.InnerText.Trim();
return text;

HTML Agility Pack Question (Attempting to parse string from source)

2 Answers