ANTLR 4 - Tree pattern matching

Question

I am trying to understand parse tree matching in ANTLR 4, so for that I have the following java code:

package sampleCodes;

public class fruits {
  public static void main(String[] args){
    int a = 10;
    System.out.println(a);
  }
}

I am using ANTLR 4 to create a parse tree of this code. Now, I want to use tree pattern matching function to find "int a = 10;". There is a doc on GitHub: https://github.com/antlr/antlr4/blob/master/doc/tree-matching.md which explains this(something like this) by an example:

ParseTree t = ...; // assume t is a statement
ParseTreePattern p = parser.compileParseTreePattern("<ID> = <expr>;", MyParser.RULE_statement);
ParseTreeMatch m = p.match(t);
if ( m.succeeded() ) {...}

From reading through this doc and few other resources, what I understood was that in:

ParseTreePattern p = parser.compileParseTreePattern("<ID> = <expr>;", MyParser.RULE_statement);

The rule to be passed as second argument must be able to correctly parse the pattern provided as first argument. Now the grammar I am using is of java given here: https://github.com/antlr/grammars-v4/tree/master/java

JavaLexer.g4, JavaParser.g4

I cannot get much info on how to structure your pattern string and its corresponding rule from the above GitHub doc. So I have tried few combinations to get the match, but none of them seems to work.For example:

ParseTreePattern p = parser.compileParseTreePattern("<variableDeclaratorId> = <variableInitializer>", parser.RULE_variableDeclarator);
ParseTreeMatch m = p.match(tree);
System.out.println(m);

This gives:

Match failed; found 0 labels

I know i am certainly doing something wrong in my string pattern. Can anyone please help me with explaining this pattern matching function, and tell what should be the correct arguments to be used in this case. Also, it will will be really helpful to provide links to some useful resources where I can learn more about this and work on complex patterns.(I could not find it in ANTLR4 reference)

A part of parse tree for this code

Answer 1

I think what you want is described in Combining XPath and tree pattern matching.

Something like this perhaps:

import org.antlr.v4.runtime.*;
import org.antlr.v4.runtime.tree.ParseTree;
import org.antlr.v4.runtime.tree.pattern.ParseTreeMatch;
import org.antlr.v4.runtime.tree.pattern.ParseTreePattern;

import java.util.List;

public class Main {

  public static void main(String[] args) {

    String source = "package sampleCodes;\n" +
            "\n" +
            "public class fruits {\n" +
            "\n" +
            "  static { int q = 42; }\n" +
            "\n" +
            "  public static void main(String[] args){\n" +
            "    int a = 10;\n" +
            "    System.out.println(a);\n" +
            "  }\n" +
            "}\n";

    JavaLexer lexer = new JavaLexer(CharStreams.fromString(source));
    JavaParser parser = new JavaParser(new CommonTokenStream(lexer));
    ParseTree tree = parser.compilationUnit();

    ParseTreePattern p = parser.compileParseTreePattern("<IDENTIFIER> = <expression>", JavaParser.RULE_variableDeclarator);
    List<ParseTreeMatch> matches = p.findAll(tree, "//variableDeclarator");

    for (ParseTreeMatch match : matches) {
      System.out.println("\nMATCH:");
      System.out.printf(" - IDENTIFIER: %s\n", match.get("IDENTIFIER").getText());
      System.out.printf(" - expression: %s\n", match.get("expression").getText());
    }
  }
}

resulting in the following output:

MATCH:
 - IDENTIFIER: q
 - expression: 42

MATCH:
 - IDENTIFIER: a
 - expression: 10

Answer 2

regarding the grammar you used, your string pattern is correct.

the reason match() is not finding anything, is that probably your passing the whole tree to it (i.e. the tree with rule compilationUnit in root) and probably you're expecting it to search the whole tree, while match() only tries to match the pattern to the given ParseTree object. match() does NOT try to find the given pattern in subtrees of the given ParseTree. for it to work, you first need to find all nodes of VariableDeclaratorContext (by overriding the enterVariableDeclarator() method in BaseListener) and then try to match the pattern on each of them. e.g.

import org.antlr.v4.runtime.CharStreams;
import org.antlr.v4.runtime.CommonTokenStream;
import org.antlr.v4.runtime.tree.ParseTree;
import org.antlr.v4.runtime.tree.ParseTreeWalker;
import org.antlr.v4.runtime.tree.pattern.ParseTreeMatch;
import org.antlr.v4.runtime.tree.pattern.ParseTreePattern;

public class Main {
    public static void main(String[] args) {
        String javaCode = "public class Main {\n" +
                "            public static void main() {\n" +
                "                    int i =0;\n" +
                "            }\n" +
                "}";


        JavaGLexer javaGLexer = new JavaGLexer(CharStreams.fromString(javaCode));
        CommonTokenStream tokens = new CommonTokenStream(javaGLexer);
        JavaGParser javaGParser = new JavaGParser(tokens);
        ParseTree tree = javaGParser.compilationUnit();
        ParseTreePattern p = javaGParser.compileParseTreePattern("<variableDeclaratorId> = <variableInitializer>", javaGParser.RULE_variableDeclarator);
        ParseTreeWalker walker = new ParseTreeWalker();
        IDListener idListener = new IDListener();
        walker.walk(idListener, tree);
        ParseTreeMatch match;
        for (JavaGParser.VariableDeclaratorContext ctx: idListener.getVarCTXs())
        {
            match  = p.match(ctx);
            if (match.succeeded()) {
                System.out.println("Match \n" + " - IDENTIFIER: " +
                        match.get("variableDeclaratorId").getText() +
                        "\n - INITIALIZER: " + match.get("variableInitializer").getText());
            }
        }
    }
}

IDListener extends JavaGBaseListener and overrides enterVariableDeclarator(), and puts variableDeclator nodes in a list, that is retrievable by getVarCTXs().

Output is:

Match
 - IDENTIFIER: i
 - INITIALIZER: 0