How can I parse nested source files with ANTLR4 - Trying one more time

Question

I found the code (reproduced below) in an article from Terrence Parr showing how INCLUDE files could be handled in ANTLR3 for Java. I tried to add this to a grammar I use with ANTLR4 (with a C++ target) but when I tried to generate a parser, I got the errors

error(50): : syntax error: '^' came as a complete surprise to me 
error(50): : syntax error: mismatched input '->' expecting SEMI while matching a rule 
error(50): : syntax error: '^' came as a complete surprise to me 
error(50): : syntax error: '(' came as a complete surprise to me while matching rule preamble

and I have no idea what these error means. Can anyone explain and perhaps show me the way forward?

(NB: I'm not wild about polluting the grammar file with code, I'm using the visitor pattern but I'll take it if I can!)

Thanks

include_filename :
  ('a'..'z' | 'A'..'Z' | '.' | '_')+
;

include_statement
@init { CommonTree includetree = null; }
 :
  'include' include_filename ';' {
    try {
      CharStream inputstream = null;
      inputstream = new ANTLRFileStream($include_filename.text);
      gramLexer innerlexer = new gramLexer(inputstream);
      gramParser innerparser = new gramParser(new CommonTokenStream(innerlexer));
      includetree = (CommonTree)(innerparser.program().getTree());
    } catch (Exception fnf) {
      ;
    }
  }
  -> ^('include' include_filename ^({includetree}))
;

Mike Lischke Mike Lischke · Accepted Answer · 2019-11-29T08:20:22

Starting with ANTLR4 it is no longer possible to manipulate the generated parse tree with grammar rules. In fact ANTLR3 generated an AST (abstract syntax tree), which is a subset of a parse tree (as generated by ANTLR4). That in turn means you cannot keep the tree rewrite syntax (the part starting with ->). Hence you should change the code to:

include_statement
@init { CommonTree includetree = null; }
 :
  'include' Include_filename ';' {
    try {
      CharStream inputstream = null;
      inputstream = new ANTLRFileStream($include_filename.text);
      gramLexer innerlexer = new gramLexer(inputstream);
      gramParser innerparser = new gramParser(new CommonTokenStream(innerlexer));
      includetree = (CommonTree)(innerparser.program().getTree());
    } catch (Exception fnf) {
      ;
    }
  }
;

How can I parse nested source files with ANTLR4 - Trying one more time

1 Answers