ANTLR4 - How do I get the token TYPE as the token text in ANTLR?

Question

Say I have a grammar that has tokens like this:

AND        : 'AND' | 'and' | '&&' | '&';
OR         : 'OR' | 'or' | '||' | '|' ;
NOT        : 'NOT' | 'not' | '~' | '!';

When I visualize the ParseTree using TreeViewer or print the tree using tree.toStringTree(), each node's text is the same as what was matched.

So if I parse "A and B or C", the two binary operators will be "and" / "or". If I parse "A && B || C", they'll be "&&" / "||".

What I would LIKE is for them to always be "AND" / "OR / "NOT", regardless of what literal symbol was matched. Is this possible?

toStringTree is just a debugging feature, you can do what you want within your own code. I don't really understand what your problem is. XY problem maybe? — Lucas Trzesniewski
To put it another way, when traversing the ParseTree and printing the text associated with a node, it is the original literal symbol ("&&") rather than the token type ("AND"). — Ryan O.

Mike Lischke Mike Lischke · Accepted Answer · 2016-06-30T06:41:19

This is what the vocabulary is for. Use yourLexer.getVocabulary() or yourParser.getVocabulary() and then vocabulary.getSymbolicName(tokenType) for the text representation of the token type. If that returns an empty string try as second step vocabulary.getLiteralName(tokenType), which returns the text used to define the token.

ANTLR4 - How do I get the token TYPE as the token text in ANTLR?

1 Answers