antlr - preserve line number and position in tree grammar?

Question

Is there an easy way to add line number information to created nodes in the tree grammar?

e.g. parser grammar

rule: a '+' b -> ^(PLUS a b);

tree grammar:

rule: ^(PLUS a b) { print_message_with_line_number_of(a); };

I tried looking into a.start.token, etc., but the ones I looked at were nulls.

Bart Kiers Bart Kiers · Accepted Answer · 2012-04-01T10:18:09

If the parser rule a contains a real token as its root, then this works:

parse
 : ^(PLUS a b) {System.out.println("line=" + $a.start.getLine());}
 ;

However, if a has an imaginary token as its root:

grammar T;

tokens {
  IMAG;
}

a : SomeToken -> ^(IMAG SomeToken)
  ;

then the token IMAG has (obviously) no line number associated with it (it's not really in the input after all!). In such cases, you need to manually create a token, set a line number to that token, and insert it in the root of your AST. That would look like:

grammar T;

tokens {
  IMAG;
}

@parser::members {
  private CommonToken token(String text, int type, int line) {
    CommonToken t = new CommonToken(type, text);
    t.setLine(line);
    return t;
  }
}

a : SomeToken -> ^({token("imag", IMAG, $SomeToken.getLine())} SomeToken)
  ;

That way, the root IMAG will get the same line number as SomeToken.

antlr - preserve line number and position in tree grammar?

1 Answers