Antlr - mismatched input error - token not recognised

Question

I have the following ANTLR grammar.

grammar DDGrammar;

ddstmt: dd2 EOF;

dd2: splddstart inlinerec;
splddstart: '//' NAME DDWORD '*' NL;
inlinerec: NON_JCL_CARD* END_OF_FILE ;

DDWORD:'DD';
//DUMMYWORD: 'DUMMY';

NAME: [A-Z@#$]+;

NON_JCL_CARD  : ~'/'  {getCharPositionInLine() == 1}? .*? ( NL | EOF ) ;
END_OF_FILE   : '/'  {getCharPositionInLine() == 1}? '*' ;

NL  : '\r' '\n' ;
WS  : [ \t]+ -> skip ;

For the input :

//SYSIN    DD  *     
SORT FIELDS=COPY
INCLUDE COND
any other program input @ $ ! & %
/*

I get the following error.

DDGrammar::ddstmt:1:2: mismatched input 'SYSIN DD * \r\n' expecting NAME Looks like SYSIN is not recognised as a NAME token. Actually a similar grammar did work sometime back. See mismatched input error. But now the same doesnt seem to work for me.

Answer 1

My guess is that you didn't regenerate the parser/lexer classes since the following code works just fine:

String source = "//SYSIN    DD  *     \r\n" +
        "SORT FIELDS=COPY\r\n" +
        "INCLUDE COND\r\n" +
        "any other program input @ $ ! & %\r\n" +
        "/*";

DDGrammarLexer lexer = new DDGrammarLexer(CharStreams.fromString(source));
DDGrammarParser parser = new DDGrammarParser(new CommonTokenStream(lexer));

parser.ddstmt();

Answer 2

JCL is painful to parse because of it's context sensitivity and the significance of whitspace.

Handling instream data is particularly thorny - there are a few optionals in there that can throw things out if you are not aware of them.

For example, there are some optional keywords that may appear after DD * (or DD DATA); these may or may not appear on the same physical line as the DD statement itself. Another is that the delimiter can be other than "/*" if the optional "DLM=" operator is used in the instream DD statement. I needed a pretty nasty Java function to handle the variability, which is never recommended.