ANTLR: using stringTemplate

Question

(I'm a Noob with Antlr)... I'm having difficulties getting my grammar with StringTemplates. Basically I'm trying to write a little DSL. I can get my grammar the way I want (it parses correctly), but I can't get the generation of the target code to work with templates. So here's a snippet of my grammar:

grammar Pfig;

options { 
    output=template;  
 language=CSharp2;
 }

conf 
    : globalName
    ;


globalName 
    : 'GlobalName:'  ID
     -> localConf(name ={$ID.text})
    ;

I simplified it quite a bit just to get the essence across. Basically, when the lex/parse comes across `GlobalName: Foo' I want it to spit out text based on the StringTemplate called 'localConf'. Super straightforward.

So now, let's fire up the parser in a test app and have it process an input file.

// C# processing a file with the lex/parser.
// the 'app.pfig' file just has one line that reads 'GlobalName: Bla'
using (FileStream fs = File.OpenRead("c:\\app.pfig"))
        {
            PfigParser parser = new PfigParser(new CommonTokenStream(
                new PfigLexer(new ANTLRInputStream(fs))));

            using (TextReader tr = File.OpenText("./Pfig.stg"))
            {
                parser.TemplateLib = new StringTemplateGroup(tr);
            }

            var parseResult = parser.conf();
            string code = parseResult.Template.ToString(); // Fail: template is null
        }

I can step through the parser code and see that it correctly identifies my text and applies the stringTemplate correctly. The problem is that since this 'globalName' rule is a subrule of 'conf' it doesn't get executed directly--the method just finds it and returns. But the calling 'Conf' method does not keep the return value from the subrule--it goes to thin air. This means that my resultant template on the last line is null.

If I get rid of the 'conf' rule in my grammar and call 'globalName' directly, it will work (since it's the only rule on the stack). But I obviously want more than one rule. I've generated the parser in Java and it does the same thing:

// antlr generated parser code
public PfigParser.conf_return conf() // throws RecognitionException [1]
{   
    PfigParser.conf_return retval = new PfigParser.conf_return();

    try 
 {
        {
         PushFollow(FOLLOW_globalName_in_conf30);
         globalName(); // <- it calls globalName() but doesn't keep the return.
         state.followingStackPointer--;

        }

        retval.Stop = input.LT(-1);

    }

// snip

It's simple to see I don't get some basic concept with how the Template approach is supposed to work with Antlr. I'm quite sure this is my problem but I'm a loggerheads to know what I'm doing wrong... the examples I've seen don't really show real-world template emission of code.

Answer 1

Basically, you need to explicitly forward the template output to the subrule:

conf 
    : a=globalName -> {$a.st}
    ;

It is not beautiful, but works.

Answer 2

It seems I've finally caught the templates idea :) Templates should be 'nested' one into another hierarchically. The following example works just fine:

TemplatesTest.g:

grammar TemplatesTest;

options
{
    output=template;
    language=CSharp2;
}

ID  :   ('a'..'z'|'A'..'Z'|'_') ('a'..'z'|'A'..'Z'|'0'..'9'|'_')*
    ;

INT :   '0'..'9'+
    ;

COMMENT
    :   '//' ~('\n'|'\r')* '\r'? '\n' {$channel=HIDDEN;}
    |   '/*' ( options {greedy=false;} : . )* '*/' {$channel=HIDDEN;}
    ;

WS  :   ( ' '
        | '\t'
        | '\r'
        | '\n'
        ) {$channel=HIDDEN;}
    ;

STRING
    :  '\'' ( ESC_SEQ | ~('\\'|'\'') )* '\''
    ;

fragment
HEX_DIGIT : ('0'..'9'|'a'..'f'|'A'..'F') ;

fragment
ESC_SEQ
    :   '\\' ('b'|'t'|'n'|'f'|'r'|'\"'|'\''|'\\')
    |   UNICODE_ESC
    |   OCTAL_ESC
    ;

fragment
OCTAL_ESC
    :   '\\' ('0'..'3') ('0'..'7') ('0'..'7')
    |   '\\' ('0'..'7') ('0'..'7')
    |   '\\' ('0'..'7')
    ;

fragment
UNICODE_ESC
    :   '\\' 'u' HEX_DIGIT HEX_DIGIT HEX_DIGIT HEX_DIGIT
    ;

doc
    : (e+=expr)+ -> doc(expressions={$e})
    ;

expr
    : ID '=' INT -> expression(id={$ID.text}, value={$INT.text})
    ;

TemplatesTest.stg:

group TemplatesTest;

doc(expressions) ::=
<<
srart expressions
<expressions; separator="\n">
end
>>

expression(id, value) ::= 
<<
<id> := <value>;
>>

Test C# code:

var lexer = new TemplatesTestLexer(new ANTLRFileStream("sample.txt"));
var parser = new TemplatesTestParser(new CommonTokenStream(lexer));

using (var reader = new StreamReader("TemplatesTest.stg"))
{
  parser.TemplateLib = new StringTemplateGroup(reader);
}

var doc = parser.doc();
Console.WriteLine(doc.Template);

Sample input:

a = 5
b = 6
c = 7

Output is:

srart expressions
a := 5;
b := 6;
c := 7;
end