ANTLR4 Grammar only matching first part of parser rule

Question

I'm using ANTLR 4 to try and parse task definitions. The task definitions look a little like the following:

task = { priority = 10; };

My grammar file then looks like the following:

grammar TaskGrammar;

/* Parser rules */

task : 'task' ASSIGNMENT_OP block EOF;

logical_entity : (TRUE | FALSE) # LogicalConst
               | IDENTIFIER     # LogicalVariable
               ;

numeric_entity : DECIMAL              # NumericConst
               | IDENTIFIER           # NumericVariable
               ;

block : LBRACE (statement)* RBRACE SEMICOLON;

assignment : IDENTIFIER ASSIGNMENT_OP DECIMAL SEMICOLON
            | IDENTIFIER ASSIGNMENT_OP block SEMICOLON
            | IDENTIFIER ASSIGNMENT_OP QUOTED_STRING SEMICOLON
            | IDENTIFIER ASSIGNMENT_OP CONSTANT SEMICOLON;

functionCall : IDENTIFIER LPAREN (parameter)*? RPAREN SEMICOLON;

parameter : DECIMAL
            | QUOTED_STRING;

statement : assignment
            | functionCall;

/* Lexxer rules */

IF   : 'if' ;
THEN : 'then';

AND : 'and' ;
OR  : 'or' ;

TRUE  : 'true' ;
FALSE : 'false' ;

MULT  : '*' ;
DIV   : '/' ;
PLUS  : '+' ;
MINUS : '-' ;

GT : '>' ;
GE : '>=' ;
LT : '<' ;
LE : '<=' ;
EQ : '==' ;

ASSIGNMENT_OP : '=' ;

LPAREN : '(' ;
RPAREN : ')' ;

LBRACE : '{' ;
RBRACE : '}' ;

SEMICOLON : ';' ;

// DECIMAL, IDENTIFIER, COMMENTS, WS are set using regular expressions

DECIMAL : '-'?[0-9]+('.'[0-9]+)? ;

IDENTIFIER : [a-zA-Z_][a-zA-Z_0-9]* ;

Value: STR_EXT | QUOTED_STRING | SINGLE_QUOTED
;

STR_EXT
  :
  [a-zA-Z0-9_/\.,\-:=~+!?$&^*\[\]@|]+;

Comment
    :
    '#' ~[\r\n]*;

CONSTANT : StringCharacters;

QUOTED_STRING
  :
  '"' StringCharacters? '"'
  ;

fragment
StringCharacters
    :   (~["\\] | EscapeSequence)+
    ;
fragment
EscapeSequence
    :   '\\' [btnfr"'\\]?
    ;
SINGLE_QUOTED
:
'\'' ~['\\]* '\'';

// COMMENT and WS are stripped from the output token stream by sending
// to a different channel 'skip'

COMMENT : '//' .+? ('\n'|EOF) -> skip ;

WS : [ \r\t\u000C\n]+ -> skip ;

This grammar compiles fine in ANTLR, but when it comes to trying to use the parser, I get the following error:

line 1:0 mismatched input 'task = { priority = 10; return = AND; };' expecting 'task'
org.antlr.v4.runtime.InputMismatchException

It looks like the parser isn't recognising the block part of the definition, but I can't quite see why. The block parse rule definition should match as far as I can tell. I would expect to have a TaskContext, with a child BlockContext containing a single AssignmentContext. I get the TaskContext, but it has the above exception.

Am I missing something here? This is my first attempt at using Antler, so may be getting confused between Lexxer and Parser rules...

Yep, tried that and got a slightly different exception, saying "expected task" instead. — Euan T
The problem is probably in your lexer rules, maybe IDENTIFIER. It would be helpful to include everything. — Marc Q.
Hi @MarcQ. I have updated the question to have all of the grammar. Thanks. — Euan T

Bart Kiers Bart Kiers · Accepted Answer · 2015-02-18T19:42:44

Your STR_EXT consumes the entire input. That rule has to go: ANTLR's lexer will always try to match as much characters as possible.

I also see that CONSTANT might consume that entire input. It has to go to, or at least be changed to consume less chars.

ANTLR4 Grammar only matching first part of parser rule

1 Answers