Tutorial for walking ANTLR ASTs in C#?

Question

Is anyone aware of tutorials for walking ANTLR-generated ASTs in C#? The closest I was able to find is this, but it's not terribly helpful.

My goal is to walk through trees that I'm generating based on a domain-specific language that I'm working on, and to use the trees to output generated C# code.

A Java-based tutorial would be helpful, too -- anything that provides clear examples of how to traverse ANTLR ASTs.

kpozin kpozin · Accepted Answer · 2009-05-29T19:06:09

I managed to figure this out by adapting the example at the end of Manuel Abadia's article.

Here's my version, which I happen to be using to convert parsed code to C#. These are the steps:

Instantiate an ANTLRStringStream or subclass with your input (it can be a file or string).
Instantiate your generated lexer, passing in that string stream.
Instantiate a token stream with the lexer.
Instantiate your parser with that token stream.
Get the top-level value from your parser, and turn it into a CommonTree.
Traverse the tree:

To get the literal text of a node, use node.Text. To get the token name of a node, use node.Token.Text.

Note that node.Token.Text will only give you the actual name of your token if it's an imaginary token with no corresponding string. If it's a real token, then node.Token.Text will return its string.

For example, if you had the following in your grammar:

tokens { PROGRAM, FUNCDEC }

EQUALS : '==';
ASSIGN : '=';

Then you'll get "PROGRAM", "FUNCDEC", "==", and "=" from the corresponding accesses of node.Token.Text.

You can see part of my example below, or you can browse the full version.

public static string Convert(string input)
{
    ANTLRStringStream sStream = new ANTLRStringStream(input);
    MyGrammarLexer lexer = new MyGrammarLexer(sStream);

    CommonTokenStream tStream = new CommonTokenStream(lexer);

    MyGrammarParser parser = new MyGrammarParser (tStream);
    MyGrammarParser.program_return parserResult = parser.program();

    CommonTree ast = (CommonTree)parserResult.Tree;

    Print(ast);
    string output = header + body + footer;

    return output;
}

public static void PrintChildren(CT ast)
{
    PrintChildren(ast, " ", true);
}

public static void PrintChildren(CT ast, string delim, bool final)
{
    if (ast.Children == null)
    {
        return;
    }

    int num = ast.Children.Count;

    for (int i = 0; i < num; ++i)
    {
        CT d = (CT)(ast.Children[i]);
        Print(d);
        if (final || i < num - 1)
        {
            body += delim;
        }
    }
}

public static void Print(CommonTree ast)
{
    switch (ast.Token.Text)
    {
        case "PROGRAM":
            //body += header;
            PrintChildren(ast);
            //body += footer;
            break;
        case "GLOBALS":
            body += "\r\n\r\n// GLOBALS\r\n";
            PrintChildren(ast);
            break;
        case "GLOBAL":
            body += "public static ";
            PrintChildren(ast);
            body += ";\r\n";
            break;

      ....
    }
}

Tutorial for walking ANTLR ASTs in C#?

4 Answers