Invalid casting Issues with antlr grammar / parser in C#

Question

I'm using Anlr 3.5 to generate a parser and lexer from the below included grammar. This grammar is to be used to read strings so as to convert them into an object graph for later evaluation. I however am having issues when I attempt to use a unaryExpression clause, doing so throws a class cast exception reporting:

Unable to cast object of type 'operandExpression_return' to type 'unaryExpression_return'.

One example input that causes this is

([p1 eq \"p1Value\"]) and [p2 eq \"p2Value\"] or [p3 > \"p3\"\"Val\"\"ue\"]

it appears to be the parenthesis (unaryExpression) that explicitly trigger the exception. Running the same statement minus the parenthesis appears to properly parse, as does the sub expression within the parenthesis.

I can clearly see via the exception the wrong type is being used. I just don't follow as to why it is being chosen by the parser nor what correction the grammar needs to avoid this error.

Of note each the two branches of the expression clause do have multiple matching alternatives with each of the binaryOperator branches, but from my understanding this should be ok only causing a less efficient greedy parser and not causing the errors I'm seeing.

Grammar used:

grammar QueryExpressionGrammar;

options {
    language=CSharp3;
    TokenLabelType=CommonToken;
    output=AST;
    ASTLabelType=CommonTree;
}

@lexer::namespace{namespace}
@parser::namespace{namespace}

@parser::members {
public static string CleanQuotedString(string input)
{
    return input == null
        ? null
        : input.Substring(1, input.Length - 2).Replace("\"\"", "\"");
}
}

/*------------------------------------------------------------------
 * PARSER RULES
 *------------------------------------------------------------------*/

public parse returns [IQueryExpression value]
    : exp=expression EOF {$value = $exp.value;}
    ;

expression returns [IQueryExpression value]
    : lhs=operandExpression { $value = $lhs.value; } ( op=binaryOperator rhs=expression {$value = new BinaryOperationQueryExpression($lhs.value, $op.value, $rhs.value);} )*
    | lhs=unaryExpression { $value = $lhs.value; } ( op=binaryOperator rhs=expression {$value = new BinaryOperationQueryExpression($lhs.value, $op.value, $rhs.value);} )*
    ;

binaryOperator returns [BinaryOperator value]
    : BinaryOperatorAnd {$value = BinaryOperator.And;}
    | BinaryOperatorOr {$value = BinaryOperator.Or;}
    ;

unaryExpression  returns [IQueryExpression value]
    : UnaryOperatorNot sub=expression {$value = new UnaryOperationQueryExpression(UnaryOperator.Not, $sub.value);}
    | OPEN_PAREN sub=expression CLOSE_PAREN {$value = new UnaryOperationQueryExpression(UnaryOperator.Paren, $sub.value);}
    ;

operandExpression returns [IQueryExpression value]
    : OPEN_BRACKET p=PROPERTY OperandQueryExpressionOperatorEq v=QUOTED_STRING CLOSE_BRACKET {$value = new OperandQueryExpression($p.text, OperandQueryExpressionOperator.Eq, CleanQuotedString($v.text));}
    | OPEN_BRACKET p=PROPERTY OperandQueryExpressionOperatorLt v=QUOTED_STRING CLOSE_BRACKET {$value = new OperandQueryExpression($p.text, OperandQueryExpressionOperator.Lt, CleanQuotedString($v.text));}
    | OPEN_BRACKET p=PROPERTY OperandQueryExpressionOperatorLe v=QUOTED_STRING CLOSE_BRACKET {$value = new OperandQueryExpression($p.text, OperandQueryExpressionOperator.Le, CleanQuotedString($v.text));}
    | OPEN_BRACKET p=PROPERTY OperandQueryExpressionOperatorGt v=QUOTED_STRING CLOSE_BRACKET {$value = new OperandQueryExpression($p.text, OperandQueryExpressionOperator.Gt, CleanQuotedString($v.text));}
    | OPEN_BRACKET p=PROPERTY OperandQueryExpressionOperatorGe v=QUOTED_STRING CLOSE_BRACKET {$value = new OperandQueryExpression($p.text, OperandQueryExpressionOperator.Ge, CleanQuotedString($v.text));}
    | OPEN_BRACKET p=PROPERTY OperandQueryExpressionOperatorLike v=QUOTED_STRING CLOSE_BRACKET {$value = new OperandQueryExpression($p.text, OperandQueryExpressionOperator.Like, CleanQuotedString($v.text));}
    | OPEN_BRACKET p=PROPERTY OperandQueryExpressionOperatorIlike v=QUOTED_STRING CLOSE_BRACKET {$value = new OperandQueryExpression($p.text, OperandQueryExpressionOperator.Ilike, CleanQuotedString($v.text));}
    | OPEN_BRACKET p=PROPERTY NonValueBasedOperandQueryExpressionOperatorIsNull CLOSE_BRACKET {$value = new OperandQueryExpression($p.text, OperandQueryExpressionOperator.IsNull, null);}
    | OPEN_BRACKET p=PROPERTY NonValueBasedOperandQueryExpressionOperatorNotNull CLOSE_BRACKET {$value = new OperandQueryExpression($p.text, OperandQueryExpressionOperator.NotNull, null);}
    ;

/*------------------------------------------------------------------
 * LEXER RULES
 *------------------------------------------------------------------*/

UnaryOperatorNot
    : 'not'
    | '!'
    ;

BinaryOperatorAnd
    : 'and'
    | '&&'
    | '&'
    ;

BinaryOperatorOr
    : 'or'
    | '||'
    | '|'
    ;

// OperandQueryExpressionOperator that uses a comparison value
OperandQueryExpressionOperatorEq
    : 'eq'
    | '=='
    | '='
    ;

OperandQueryExpressionOperatorLt
    : 'lt'
    | '<'
    ;

OperandQueryExpressionOperatorLe
    : 'le'
    | '<='
    ;

OperandQueryExpressionOperatorGt
    : 'gt'
    | '>'
    ;

OperandQueryExpressionOperatorGe
    : 'ge'
    | '>='
    ;

OperandQueryExpressionOperatorLike
    : 'like'
    ;

OperandQueryExpressionOperatorIlike
    : 'ilike'
    ;

// OperandQueryExpressionOperator that does not use a comparison value
NonValueBasedOperandQueryExpressionOperatorIsNull
    : 'null'
    | 'isnull'
    ;

NonValueBasedOperandQueryExpressionOperatorNotNull
    : 'notnull'
    ;

OPEN_BRACKET: '[';
CLOSE_BRACKET:  ']';
OPEN_PAREN: '(';
CLOSE_PAREN: ')';
PROPERTY: LETTER (ALPHA_NUMERIC | SPECIAL)*; // property definition is in line with the definition of a property definition for c#
QUOTED_STRING: QUOTE (~QUOTE | (QUOTE QUOTE))* QUOTE; // values are characters, or if they contain quotes they are escaped by double quoting and surrounding value in quotes

fragment DIGIT: ('0'..'9');
fragment LETTER: (('a'..'z')|('A'..'Z'));   
fragment ALPHA_NUMERIC: (LETTER|DIGIT);
fragment SPECIAL: ('_'|'-');    
fragment QUOTE: '\u0022';

WHITESPACE: (' ' | '\t' | '\n' | '\r'){ Skip(); }; // valid whitespace characters

Snippet of exception throwing code within the parser within the private QueryExpressionGrammarParser.expression_return expression() method

...
switch (alt3)
{
case 1:
    DebugEnterAlt(1);
    // QueryExpressionGrammar.g:37:4: lhs= operandExpression (op= binaryOperator rhs= expression )*
    {
    root_0 = (CommonTree)adaptor.Nil();

    DebugLocation(37, 7);
    PushFollow(Follow._operandExpression_in_expression111);
    lhs=operandExpression();
    PopFollow();

    adaptor.AddChild(root_0, lhs.Tree);
    DebugLocation(37, 26);


    /// v v v Exception throwing on line below v v v
     retval.value = (lhs!=null?((QueryExpressionGrammarParser.unaryExpression_return)lhs).value:default(IQueryExpression)); 


    DebugLocation(37, 51);
    // QueryExpressionGrammar.g:37:51: (op= binaryOperator rhs= expression )*
    try { DebugEnterSubRule(1);
...

Try simplifying the expression to see just what causes it to fail. For example, does it fail if the expression loses the "or ..." part? Have you tried using something besides 'lhs' to hold the unary expression since the other 'lhs' is holding an operand expression? (Grasping!) — Lee Meador
@LeeMeador Well I'll be damned. Yeah, it seems it was just a matter of renaming the variables. I was entirely under the impression that those Lhs vars were appearing in different scopes. Too bad I can't mark a comment as accepted. — rheone

Lee Meador Lee Meador · Accepted Answer · 2013-01-08T23:44:07

So here is the solution we figured out in the comments. Change this one:

expression returns [IQueryExpression value]
    : lhs=operandExpression { $value = $lhs.value; } ( op=binaryOperator rhs=expression {$value = new BinaryOperationQueryExpression($lhs.value, $op.value, $rhs.value);} )*
    | ulhs=unaryExpression { $value = $ulhs.value; } ( op=binaryOperator rhs=expression {$value = new BinaryOperationQueryExpression($ulhs.value, $op.value, $rhs.value);} )*
    ;

Invalid casting Issues with antlr grammar / parser in C#

1 Answers