Library to query HTML with XPath in Java?

5
votes

Can anyone recommend me a java library to allow me XPath Queries over URLs? I've tried JAXP without success.

Thank you.

5
javahtmlxhtmlxpath
See stackoverflow.com/questions/9022140/… - not quite a duplicate as it asks about specific XPath functionality but there are better answers there. – Mark Butler
@Reonarudo I am in the same situation as you were when you asked this question. There are many possible suggestions/solutions in the answers, but I would like to know which solution(library) you used and did it work out the way you wanted it ? – Uther Pendragon
@UtherPendragon I'm sorry but this was a long time ago and I cannot recall which project was this. Anyway there should be newer/better libraries available nowadays. – Leonardo Marques

5 Answers

8
votes

There are several different approaches to this documented on the Web:

Using HtmlCleaner

Using Jericho

I have tried a few different variations of these approaches, i.e. HtmlParser plus the Java DOM parser, and JSoup plus Jaxen, but the combination that worked best is HtmlCleaner plus the Java DOM parser. The next best combination was Jericho plus Jaxen.

6
votes

jsoup, Java HTML Parser Very similar to jQuery syntax way.

1
votes

You could use TagSoup together with Saxon. That way you simply replace any XML SAX parser used with TagSoup and the XPath 2.0 or XSLT 2.0 or XQuery 1.0 implementation works as usual.

0
votes

I've used JTidy to make HTML into a proper DOM, then used plain XPath to query the DOM.

If you want to do cross-document/cross-URL queries, better use JTidy with XQuery.

0
votes

Use Xsoup. According to the docs, it's faster than HtmlCleaner. Example

 @Test
    public void testSelect() {

        String html = "<html><div><a href='https://github.com'>github.com</a></div>" +
                "<table><tr><td>a</td><td>b</td></tr></table></html>";

        Document document = Jsoup.parse(html);

        String result = Xsoup.compile("//a/@href").evaluate(document).get();
        Assert.assertEquals("https://github.com", result);

        List<String> list = Xsoup.compile("//tr/td/text()").evaluate(document).list();
        Assert.assertEquals("a", list.get(0));
        Assert.assertEquals("b", list.get(1));
    }

Link to Xsoup - https://github.com/code4craft/xsoup