antlr4 works fine on a .g4 grammar file, but gradle generateGrammarSources fails on the same file with TokenStreamException: unexpected char: '-'/"

Question

I am experimenting with the antlr plugin for gradle and trying to generate some source files for my grammar. Using the ANTLR Mega tutorial as a guide. When I run the antlr4 tool manually on my grammar through the terminal, it succeeds and generates *.java, *.tokens & *.interp files. I am trying to generate these files through the gradle plugin and getting an error "> ANTLR Panic: TokenStreamException: unexpected char: '-'".

Do I expect gradle generateGrammarSources to have the equivalent functionality of running the antlr command directly on my grammar?

The grammar file is based on one of Antlr's example: https://github.com/antlr/grammars-v4/blob/master/arithmetic/arithmetic.g4

user-MBP:antlr user$ antlr4 
ANTLR Parser Generator  Version 4.7.2

user-MBP:main-project user$ ls  src/main/antlr/
arithmetic.g4

user-MBP:antlr user$ antlr4 arithmetic.g4 
user-MBP:antlr user$ ls
arithmetic.g4               arithmetic.tokens           arithmeticLexer.interp      arithmeticLexer.tokens      arithmeticParser.java
arithmetic.interp           arithmeticBaseListener.java arithmeticLexer.java        arithmeticListener.java

Gradle code to do the same:

apply plugin: 'antlr'

...

dependencies {
...
    compile group: 'org.antlr', name: 'antlr4-runtime', version: '4.7.1'
...
}

// as the .g4 grammar file is already present in src/main/antlr, not  // explicitly specifying any package directory

Running the gradle command

user-MBP:main-project user$ gradle generateGrammarSource --stacktrace

Caused by: java.lang.RuntimeException: ANTLR Panic: TokenStreamException: unexpected char: '-'
        at antlr.Utils.error(Utils.java:34)
        at antlr.Tool.fatalError(Tool.java:445)
        at antlr.Tool.doEverything(Tool.java:280)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

I expect the above gradle command to successfully run and generate equivalent source files in the build/ directory, but it fails.

nirmal nirmal · Accepted Answer · 2019-06-05T09:58:42

I was missing the antlr tool that generates the grammar in my gradle dependencies. ANTLR has two components: - the tool used to generate the lexer and parser from the grammar (antlr dependency) - the runtime required to run the source files generated (antlr4-runtime)

Adding the following line in my gradle dependencies solved the issue.

    dependencies {
        ...
        antlr "org.antlr:antlr4:4.7.1"
        ...
    }

antlr4 works fine on a .g4 grammar file, but gradle generateGrammarSources fails on the same file with TokenStreamException: unexpected char: '-'/"

1 Answers