Parsing input string with antlr grammar

Question

I have a following problem with the grammar:

The input string i'm going to parse is as follows:

ruledef COMPLEX1 
    ftp command args = abc
    ftp command args = ftp
    ftp command args = cde
exit

The grammar i use:

grammar main;

/*Lexical*/
NUM : [0-9]+;
STRING : [0-9a-zA-Z]+;
WS : [ \t\r\n]+ -> skip; // Whitespace definition: skip spaces, tabs and newlines

ruledefrule: 'ruledef' STRING (ruledef_ftpcommandargsrule )* 'exit';
ruledef_ftpcommandargsrule: 'ftp' 'command' 'args' '=' STRING ;

When i run it through antlr i receive an error:

line 3:23 missing STRING at 'ftp'

What is more any of the words used in the input for example 'command' or 'args' causes the same problem.

ftp command args = ftp
ftp command args = args
ftp command args = command

Does anybody know how to deal with that kind of problems?

Tomer Aberbach Tomer Aberbach · Accepted Answer · 2018-11-29T22:21:05

Your issue is that string literals in your grammar such as 'ruledef' and 'exit' implicitly have their own token types and are matched before everything else, including before STRING. Therefore, STRING does not contain 'ruledef', 'exit', 'ftp', 'command', and 'args' in its set of possible values. It is as if you have implicitly written the following grammar:

grammar main;

/* Lexical */
RULEDEF : 'ruledef' ;
EXIT : 'exit' ;
FTP : 'ftp' ;
COMMAND : 'command' ;
ARGS : 'args' ;
NUM : [0-9]+ ;
STRING : [0-9a-zA-Z]+ ;
WS : [ \t\r\n]+ -> skip ; // Whitespace definition: skip spaces, tabs and newlines

ruledefrule : RULEDEF STRING ruledef_ftpcommandargsrule* EXIT ;
ruledef_ftpcommandargsrule : FTP COMMAND ARGS '=' STRING ;

The grammar above doesn't support the inputs you mentioned because 'ruledef', 'exit', 'ftp', 'command', and 'args' are all captured by tokens other than STRING so they do not match it in ruledef_ftpcommandargsrule. The way to solve this is to make another rule, let's call it string, which can be STRING, 'ruledef', 'exit', 'ftp', 'command', or 'args'. And then use that rule in place of STRING where ever the behavior is desired:

grammar main;

/* Lexical */
NUM : [0-9]+ ;
STRING : [0-9a-zA-Z]+ ;
WS : [ \t\r\n]+ -> skip ; // Whitespace definition: skip spaces, tabs and newlines

ruledefrule : 'ruledef' string ruledef_ftpcommandargsrule* 'exit' ;
ruledef_ftpcommandargsrule : 'ftp' 'command' 'args' '=' string ;
string : STRING | 'ruledef' | 'exit' | 'ftp' | 'command' | 'args' ;

Let me know if you'd like me to clarify anything.

Parsing input string with antlr grammar

2 Answers