ANTLR 4 and AST visitors

Question

I'm trying to use ASTs with ANTLR4, with this files:

Builder.java

import org.antlr.v4.runtime.ANTLRInputStream;
import org.antlr.v4.runtime.CharStream;
import org.antlr.v4.runtime.CommonTokenStream;
import org.antlr.v4.runtime.TokenStream;

public class Builder
{

    public static void main(String[] args)
    {
        CharStream input = new ANTLRInputStream("ON M1==2 && M3 == 5 && (M2 > 1 || M5 <= 5.0) "
                                              + "DO P5:42 P4:10");
        ExprLexer lexer = new ExprLexer(input);
        TokenStream tokens = new CommonTokenStream(lexer);
        ExprParser parser = new ExprParser(tokens);
        parser.addParseListener(new ExprTestListener());
        ExprParser.ExpressionContext uu = parser.expression();
    }

}

ExprTestListener:

import org.antlr.v4.runtime.ParserRuleContext;
import org.antlr.v4.runtime.Token;
import org.antlr.v4.runtime.tree.TerminalNode;
import org.antlr.v4.runtime.tree.ErrorNode;

public class ExprTestListener extends ExprBaseListener {
    @Override public void enterExpression(ExprParser.ExpressionContext ctx)
    {
        System.out.println(ctx);
    }
    @Override public void exitExpression(ExprParser.ExpressionContext ctx)
    {
        System.out.println(ctx);
    }

    @Override public void enterActionexpr(ExprParser.ActionexprContext ctx)
    {
        System.out.println(ctx);
    }
    @Override public void exitActionexpr(ExprParser.ActionexprContext ctx)
    {
        System.out.println(ctx);
    }

    @Override public void enterCondexpr(ExprParser.CondexprContext ctx)
    {
        System.out.println(ctx);
    }
    @Override public void exitCondexpr(ExprParser.CondexprContext ctx)
    {
        System.out.println(ctx);
    }

    @Override public void enterCond(ExprParser.CondContext ctx)
    {
        System.out.println(ctx);
    }
    @Override public void exitCond(ExprParser.CondContext ctx)
    {
        System.out.println(ctx);
    }

    @Override public void enterEveryRule(ParserRuleContext ctx)
    {
        System.out.println(ctx);
    }
    @Override public void exitEveryRule(ParserRuleContext ctx)
    {
        System.out.println(ctx);
    }
    @Override public void visitTerminal(TerminalNode node)
    {
    }
    @Override public void visitErrorNode(ErrorNode node)
    {
    }
}

Expr.g:

grammar Expr;
options
{
  // antlr will generate java lexer and parser
  language = Java;

}
WS      : [ \t\r\n]+ -> skip ;
OP      : '&&' | '||';
COMP    : '==' | '<' | '>' | '<=' | '>=' | '!=';
fragment INT     : [0-9]+;
REAL    : INT '.' INT | INT;

ACTION  : 'P' INT ':' INT;
MEASURE : 'M' INT;

// ***************** parser rules:
cond       : MEASURE COMP REAL;
condexpr   : '(' condexpr ')' | cond OP condexpr | cond;
actionexpr : ACTION actionexpr | ACTION;
expression : 'ON' condexpr 'DO' actionexpr;

I have this output:

[]
[]
[29]
[29]
[16 29]
[16 29]
[16 29]
[16 29]
[18 29]
[18 29]
[16 18 29]
[16 18 29]
[16 18 29]
[16 18 29]
[18 18 29]
[18 18 29]
[13 18 18 29]
[13 18 18 29]
[16 13 18 18 29]
[16 13 18 18 29]
[16 13 18 18 29]
[16 13 18 18 29]
[18 13 18 18 29]
[18 13 18 18 29]
[20 18 13 18 18 29]
[20 18 13 18 18 29]
[20 18 13 18 18 29]
[20 18 13 18 18 29]
[18 13 18 18 29]
[18 13 18 18 29]
[13 18 18 29]
[13 18 18 29]
[18 18 29]
[18 18 29]
[18 29]
[18 29]
[29]
[29]
[31]
[31]
[24 31]
[24 31]
[24 31]
[24 31]
[31]
[31]
[]
[]

I find it hard to understand visitors with ANTLR4.

I have tree goals:

Get collect INTs of MEASUREs and ACTIONs (in two different sets)
Replace some OPs (for example != by <>)
Get the condexpr (the top item) string with OPs remplaced (se my previous point)

Sam Harwell Sam Harwell · Accepted Answer · 2013-02-03T00:35:23

First I'll explain what you have observed above:

First and foremost, please read the documentation for the methods you call. The Parser.addParseListener documentation includes the following note:

THIS IS ONLY FOR ADVANCED USERS. Please give your ParseTreeListener to a ParseTreeWalker instead of giving it to the parser!!!!

The implementation of toString() for the ParserRuleContext class simply prints the rule invocation stack at the time the context was created. You are printing this once when the listener enters a rule, and once when it exits. For actionexpr, cond, and condexpr you print it again, resulting in a total of 4 identical output lines for each of those contexts.

Now some notes on your goals:

Inside of enterCond and exitCond, the MEASURE text is available by calling ctx.MEASURE().getText().
Inside of enterActionexpr and exitActionexpr, the ACTION text is available by calling ctx.ACTION().getText().
You can change the COND token by creating a new TerminalNodeImpl and CommonToken for the updated token, and assigning it to the correct index in the field CondContext.children using either a visitor or a listener.

ANTLR 4 and AST visitors

2 Answers