Why is ANTLR parsing skipping token?

Question

I'm trying to do some very basic C++ function declaration parsing. Here is my rule for parsing an input parameter:

arg : 'const'? 'unsigned'? t=STRING m=TYPEMOD? n=STRING 
 -> ^(ARG $n $t $m?) ;

STRING  : ('a'..'z'|'A'..'Z'|'0'..'9'|'_'|'::')+ ;

TYPEMOD 
 : ('*' | '&')+ ;

The problem is I'm trying to pass it something like:

int *param

It matches the "int" with t, but then skips the TYPEMOD, throws an exception saying the * is unwanted when trying to match the n, and then matches "param" with n. Why would it skip the TYPEMOD token and not match it?

Thanks in advance!

EDIT:

Here are my whitespace rules (I included my comment rules as well), everything seems to work fine when parsing other stuff like class declarations, properties, structs, it's just these TYPEMOD things...

COMMENT_LINE 
    :   '//' (~'\n')* '\n' { $channel = HIDDEN; } ;

COMMENT_BLOCK 
    :   '/*' .* '*/' { $channel = HIDDEN; } ;

NL      :   ('\r' | '\n')+ { $channel = HIDDEN; } ;

WS      :   (' '|'\t')+ { $channel = HIDDEN; } ;

Actually it's a problem on lexing, not parsing. Sorry I can't give you a concrete tip on what's wrong. — Volker Stolz
The formatting didn't work for the ws rule code, so I put my comments in the question. — Steve

WayneH WayneH · Accepted Answer · 2010-07-21T23:31:43

With only minor changes I think this is what you want the rules to look like to continue with your work:

 arg : 'const'? 'unsigned'? t=STRING m=typemod? n=STRING 
 -> ^(ARG $n $t $m?) ;

typemod : TYPEMOD+;

STRING  : ('a'..'z'|'A'..'Z'|'0'..'9'|'_'|'::')+ ;

TYPEMOD 
 : '*'|'&' ;

The STRING lexer rule I did not change, but I did modify your arg and TYPEMOD rules and created a new parser rule typemod.

Oh yeah, I used the java target, so hopefully this will work for you.

Hope this helps, good luck.

Why is ANTLR parsing skipping token?

1 Answers