ANTLR : AST eval problems

Question

Allo,
I would like to eval an AST that i generated.
I wrote a grammar generating an AST, and now I'm triying to write the grammar to evaluate this tree. Here's my grammar :

tree grammar XHTML2CSVTree;

options {
    tokenVocab=XHTML2CSV;
    ASTLabelType=CommonTree;
}

@members {
                // variables and methods to be included in the java file generated
}

/*------------------------------------------------------------------
 * TREE RULES
 *------------------------------------------------------------------*/

// example
tableau returns [String csv]
    : ^(TABLEAU {String retour="";}(l=ligne{retour += $l.csv;})* {System.out.println(retour);})
    ;

ligne returns [String csv] 
    : ^(LIGNE {String ret="";}(c=cellule{ret += $c.csv;})+)
    ;

cellule returns [String csv]    
    : ^(CELLULE s=CHAINE){ $csv = $s.text;}
    ;

And here's the grammar building the AST :

grammar XHTML2CSV;

options {
    output=AST;
    ASTLabelType=CommonTree;
}

tokens {
    CELLULE;
    LIGNE;
    TABLEAU;
    CELLULEG    =  '<td>';                                                  // simple lexemes
    CELLULED    =  '</td>';
    DEBUTCOL    =  '<tr>';
    FINCOL      =  '</tr>';
    DTAB        =  '<table';
    FTAB        =  '>';
    FINTAB      =  '</table>';  

                                // anonymous tokens (usefull to give meaningfull name to AST labels)
                    // simple lexemes
}

@members {
                                                // variables and methods to be included in the java file generated
}

/*------------------------------------------------------------------
 * PARSER RULES
 *------------------------------------------------------------------*/
tableau
    : DTAB STAB* FTAB ligne* FINTAB -> ^(TABLEAU ligne*)
    ;

ligne 
    : DEBUTCOL cellule+ FINCOL -> ^(LIGNE cellule+)
    ;

cellule
    : CELLULEG CHAINE CELLULED -> ^(CELLULE CHAINE)
    ;           

/*------------------------------------------------------------------
 * LEXER RULES
 *------------------------------------------------------------------*/
STAB    
    :    ' '.*'=\"'.*'\"'
    ;   

WS
     : ( '\t' | ' ' | '\r' | '\n'| '\u000C' )+ {$channel = HIDDEN;}
     ;                                                              // skip white spaces

CHAINE  :  (~('\"' | ',' | '\n' | '<' | '>'))+
    ;   

                    // complex lexemes

XHTML2CSV.g works, i can see the AST generated in ANTLRworks, but i cannot parse this AST to generated CSV code. I get errors :

XHTML2CSVTree.java:144: ';' expected
            match(input, Token.DOWN, null); 
                 ^
XHTML2CSVTree.java:144: not a statement
            match(input, Token.DOWN, null); 
                  ^
XHTML2CSVTree.java:144: ';' expected
            match(input, Token.DOWN, null); 
                       ^
XHTML2CSVTree.java:144: not a statement
            match(input, Token.DOWN, null); 
                              ^
XHTML2CSVTree.java:144: ';' expected
            match(input, Token.DOWN, null); 
                               ^

5 errors

If someone could help me,
Thanks.

eo

Edit :
My main class looks like :

import org.antlr.runtime.ANTLRFileStream;
import org.antlr.runtime.CommonTokenStream;
import org.antlr.runtime.tree.CommonTreeNodeStream;
import org.antlr.runtime.tree.Tree;

public class Main {
   public static void main(String args[]) throws Exception {


      try {
          XHTML2CSVLexer lex = new XHTML2CSVLexer(new ANTLRFileStream(args[0]));    // create lexer to read the file specified from command line (i.e., first argument, e.g., java Main test1.xhtml)
          CommonTokenStream tokens = new CommonTokenStream(lex);                    // transform it into a token stream
          XHTML2CSVParser parser = new XHTML2CSVParser(tokens);                     // create the parser that reads from the token stream

          Tree t = (Tree) parser.cellule().tree;                                // (try to) parse a given rule specified in the parser file, e.g., my_main_rule


           CommonTreeNodeStream nodes = new CommonTreeNodeStream(t);                // transform it into a common data structure readable by the tree pattern
           nodes.setTokenStream(tokens);                                            // declare which token to use (i.e., labels of the nodes defined in the parser, mainly anonymous tokens)
           XHTML2CSVTree tparser = new XHTML2CSVTree(nodes);                        // instantiate the tree pattern
           System.out.println(tparser.cellule());                                                   // apply patterns
       } catch (Exception e) {
           e.printStackTrace();
       }
   }
}

Bart Kiers Bart Kiers · Accepted Answer · 2011-10-09T18:00:52

The ligne rule in you tree grammar:

ligne returns [String csv] 
  : ^(LIGNE {Sting ret="";r}(c=cellule{ret += $c.csv;})+)
  ; //         ^          ^ 
    //         |          |
    // problem 1, problem 2

has 2 problems:

it contains Sting where it should be String;
there's a trailing r that is messing up your custom Java code.

It should be:

ligne returns [String csv] 
  : ^(LIGNE {String ret="";}(c=cellule{ret += $c.csv;})+)
  ;

EDIT

If I generate a lexer and parser (1), generate a tree walker (2), compile all .java source files (3) and run the Main class (4):

java -cp antlr-3.3.jar org.antlr.Tool XHTML2CSV.g
java -cp antlr-3.3.jar org.antlr.Tool XHTML2CSVTree.g
javac -cp antlr-3.3.jar *.java
java -cp .:antlr-3.3.jar Main test.txt

the following gets printed to the console:

table data

where the file test.txt contains:

<td>table data</td>

So I don't see any problem. Perhaps you're trying to parse a <table>? This would go wrong since both your parser and tree-walker are invoking the cellule rule, not the tableau rule.

ANTLR : AST eval problems

eo

1 Answers

EDIT