Antlr4 - Match empty set in lexer

Question

I just upgraded a project that was using the alpha version of ANTLR4 and matching an empty set worked fine. All that would happen was that ANTLR4 would throw a warning saying that it was matching and empty set.

Now using ANTLR language support 1.2.1 and parser generator 4.3.0 it now throws the error: "Error 1 '->' came as a complete surprise to me while matching a lexer rule" in both places I require it to match an empty set.

Some of the code it breaks on:

mode FILE_MODE;
     ASCII: FASCII           -> mode( PATH_MODE );
     UTF8: FUTF8             -> mode( PATH_MODE );
     UTF16: FUTF16           -> mode( PATH_MODE );
     EMPTY_FILE_TYPE:        -> skip, mode( PATH_MODE );  // Throws the error on this line

How does one now match an empty rule?

Terry

GRosenberg GRosenberg · Accepted Answer · 2016-06-05T19:59:44

One does not.

The warning before indicates that Antlr was likely not generating functional code for that line. If the grammar was working before, it was likely due to undefined behaviors in that particular Antlr version.

Now just remove the line. Whatever was detecting the need to do a mode switch to FILE_MODE should determine to do so only if there is some file mode content to match. If not, then move on to considering if there is path mode content.

Determining the best mode plan/guard conditions is tricky. Sometimes the only solution is a predicate using lookaheads. To help further, exemplary source and the full relevant rule set is needed.

Antlr4 - Match empty set in lexer

1 Answers