ANTLR4 grammar testing methods

1
votes

I am writing antlr4 grammar for a closed programming language. The application code i would be writing in python.

Now, i would like to unit/integration test all the grammar rules against the possible combinations of the language. For unit testing i am thinking of the below approaches. If anyone has worked on something similar can you give me directions on which method would be suitable ?

  1. Files with lots of possible combination and their respective tree output in another file.So in python i would be reading all the files generating parse tree and then comparing the tree result with the output file.
  2. For every grammar rule creating a test file in that writing many methods which would give the input in string and match the tree output with string. But i am thinking this would be becoming too complex for rules supporting many combinations.It would too many test function calls and in that creating parse tree.
  3. In this method it would be similar to method 2 but instead of comparing the tree strings to the output i would be checking whether each token/subrule in the grammar is identified as per the expectations or not. But over here it would be complicated for rules which are supporting multiple options as i would need to check which option passed and then again test accordingly.

Any help for this would be very much appreciated.

1
python-3.xantlr4

1 Answers

1
votes

I would keep the unit test simple (so that you can write a lot of them): parsing should expect 0 errors (or should expect n errors depending on your test).

Of course, you can print out something that will help you analyze quickly why a unit test is failing.

My opinion is that you should write your input files as the same time as your rules (don't write the all grammar and then all the unit tests)

As I was using Junit, I found Parameterized test used with Reflections very useful: I put all my test input files in one folder and need just one test file. Each time you will add or modify a rule or a test file, all your junit tests will be played without a need to update them.