Extract content using xpath from an HTML doc using pure Java

Question

I want to extract content from an HTML using xpaths using Java. In ruby I can do this using nokogiri as shown here.

xpath = '/html/body/div/div[2]/div[2]/div/div[2]/div[3]/p'  
doc = Nokogiri::HTML(open('test_001_html64.html'))  
doc.xpath().each do |link|  
puts link.content  
end  

I want to do it in pure Java. I looked at Jsoup but I couldn't find any documentation or example that uses an xpath to do this. Can someone suggest a way?

Thanks

Answer 1

Here's how you can do it with JAXP (bundled in Java): JAXP Manual

Answer 2

You can use HtmlUnit for that task.

Here's a simple example:

final WebClient webClient = new WebClient();
final HtmlPage startPage = webClient.getPage("http://www.google.com/");
List<DomNode> nodes = page.getByXPath("/html/body/div/div[2]/div[2]/div/div[2]/div[3]/p");
for (DomNode node : nodes) {
    System.out.println(node.getNodeName());
}

Answer 3

You can easily do this in jsoup.

Document doc = Jsoup.connect("test_001_html64.html").get();
Elements info = doc.getElementsByTag("html");
//iterate recursively to the desired location in the dom tree.

For faster parsing, you can use specific tags/ ids.

The documentation for jsoup (jsoup.org/apidocs) also exists.