ANTLR4 Hangs Up When Dealing with EOF

Question

I've a simple ANTLR4 grammar :

grammar Test;
preprocessing_file: oneline+;

oneline: IDENTIFIER? new_line;

new_line: EOF|CRLF
;

WS: [ \t\f]+ -> channel(2);

CRLF: '\r'? '\n';


IDENTIFIER:  (NONDIGIT | DIGIT )+
; 
  fragment DIGIT: [0-9];
  fragment NONDIGIT: [_a-zA-Z]  ;

I was testing how to use a newline rule that allows the last line not being terminated by a CRLF. I tested the grammar with ANTLR v4.1 and v4.5.3.

An input file of a few lines of text caused ANTLR4 to freeze and OutOfMemoryException after a while. It looks like ANTLR4 entered some infinite loop. Is it a bug with ANTLR4? Did I do anything wrong? Also, if I remove the EOF in new_line rule, everything works fine.

GRosenberg GRosenberg · Accepted Answer · 2016-04-30T00:30:20

Just need to add an EOF to the main rule. Without it, the main rule will never terminate - the lexer will continue to produce EOF tokens that are successfully consumed by the new_line rule.

preprocessing_file: oneline+? EOF ;
oneline: IDENTIFIER? new_line ;
new_line: EOF|CRLF ;

ANTLR4 Hangs Up When Dealing with EOF

2 Answers