ANTLR4 custom tokens

Question

I am using C# version for generating ANTLR4 files. I have used custom tokens by using this option TokenLabelType=Token. Everything is fine but c# compiler gives error in MAtch(..) and input(...) because this does not type cast to my custom tokens. Whereas ANTLR3 gives the proper casting for this functions. I am extending my own token from this class Antlr4.Runtime.CommonToken.

C# compiler throws this error

Cannot implicitly convert type 'Antlr4.Runtime.IToken' to 'Grammar.ActionParser.Token'. An explicit conversion exists (are you missing a cast?)".

Please tell how to resolve this issue.

Answer 1

Is your custom token class named 'Token'? If so, check the using statements to make sure that the parser is not confusing the Antlr4.Runtime.Token class for your token class. If not, change it to the actual type of your custom token class.

Answer 2

I'm having exactly the same issue in ANTLR4.7 with C++ target. All the labels on tokens in contexts properly use my custom token class, however the code that assigns those labels is not ready for that, trying to assign base Token * to MyToken *. I've used workaround for this issue by declaring

virtual MyToken *match(size_t ttype) override;
virtual MyToken *matchWildcard() override;
virtual MyToken *getCurrentToken() override;
virtual MyToken *consume() override;

in generated parser. All these overrides do is call base version of themselves and static_cast<MyToken *> on result of that call. I assume something similar should be possible in C#.

The proper way would probably require fixing appropriate .stg file (under /org/antlr/v4/tool/templates/codegen/<target language> inside antlr .jar file - there is also a way of overriding it without the need to edit original .jar) so the generated code for label assignments does these casts without the need to override return type (or to make it generate the above routines automatically).