writing a typesafe visitor with labeled rules

Question

I am migrating my prototype from a listener to a visitor pattern. In the prototype, I have a grammar fragment like this:

thingList: thing+ ;

thing
  : A aSpec    # aRule
  | B bSpec    # bRule
  ;

Moving to a visitor pattern, I am not sure how I write visitThingList. Every visitor returns a specializes subclass of "Node", and I would love somehow when to be able to write something like this, say a "thingList" cares about the first thing in the list some how ...

visitThingList(cx: ThingListContext): ast.ThingList {
  ...
  const firstThing = super.visit(cx.thing(0));

The problem with this is in typing. Each visit returns a specialized type which is a subclass of ast.Node. Because I am using super.visit, the return value will be the base class of my node tree. However, I know because I am looking at the grammar and because I wrote both vistARule and visitBRule that the result of the visit will be of type ast.Thing.

So we make visitThingList express it's expectation with cast ...

visitThingList(cx: ThingListContext): ast.ThingList {
  const firstThing = super.visit(cx.thing(0));
  if (!firstThing instanceof ast.Thing) {
    throw "no matching visitor for thing";
  }
  // firstThing is now known to be of type ast.Thing
  ...

In much of my translator, type problems with ast Nodes are a compile time issue, I fix them in my editor. In this case, I am producing a more fragile walk, which will only reveal the fragility at runtime and then only with certain inputs.

I think I could change my grammar, to make it possible to encode the type expectations of vistThingList() by creating a vistThing() entry point

thingList: thing+ ;
thing: aRule | bRule;
aRule: A aSpec;
bRule: B bSpec;

With vistThing() typed to match the expectation:

visitThing(cx: ThingContext): ast.Thing { }
visitThingList(cx: ThingListContext) {
  const firstThing: ast.Thing = this.visitThing(cx.thing(0));

Now visitThingList can call this.visitThing() and the type enforcement of making sure all rules that a thing matches return ast.Thing belongs to visitThing(). If I do create a new rule for thing, the compiler will force me to change the return type of visitThing() and if I make it return something which is NOT a thing, visitThingList() will show type errors.

This also seems wrong though, because I don't feel like I should have to change my grammar in order to visit it.

I am new to ANTLR and wondering if there is a better pattern or approach to this.

Bart Kiers Bart Kiers · Accepted Answer · 2020-12-09T08:18:10

When I was using the listener pattern, I wrote something like:
enterThing(cx: ThingContext) { }
enterARule(cx : ARuleContext) { }
enterBRule(cx : BRuleContext) { }

Not quite: for a labeled rule like thing, the listener will not contain enterThing(...) and exitThing(...) methods. Only the enter... and exit... methods for the labels aSpec and bSpec will be created.

How would I write the visitor walk without changing the grammar?

I don't understand why you need to change the grammar. When you keep the grammar like you mentioned:

thingList: thing+ ;

thing
  : A aSpec    # aRule
  | B bSpec    # bRule
  ;

then the following visitor could be used (again, there is no visitThing(...) method!):

public class TestVisitor extends TBaseVisitor<Object> {

    @Override
    public Object visitThingList(TParser.ThingListContext ctx) {
        ...
    }

    @Override
    public Object visitARule(TParser.ARuleContext ctx) {
        ...
    }

    @Override
    public Object visitBRule(TParser.BRuleContext ctx) {
        ...
    }

    @Override
    public Object visitASpec(TParser.ASpecContext ctx) {
        ...
    }

    @Override
    public Object visitBSpec(TParser.BSpecContext ctx) {
        ...
    }
}

EDIT

I do not know how, as i iterate over that, to call the correct visitor for each element

You don't need to know. You can simply call the visitor's (super) visit(...) method and the correct method will be invoked:

class TestVisitor extends TBaseVisitor<Object> {

    @Override
    public Object visitThingList(TParser.ThingListContext ctx) {
        for (TParser.ThingContext child : ctx.thing()) {
            super.visit(child);
        }
        return null;
    }

    ...
}

And you don't even need to implement all methods. The ones you don't implement, will have a default visitChildren(ctx) in them, causing (as the name suggests) all child nodes under them being traversed.

In your case, the following visitor will already cause the visitASpec and visitBSpec being invoked:

class TestVisitor extends TBaseVisitor<Object> {

    @Override
    public Object visitASpec(TParser.ASpecContext ctx) {
        System.out.println("visitASpec");
        return null;
    }

    @Override
    public Object visitBSpec(TParser.BSpecContext ctx) {
        System.out.println("visitBSpec");
        return null;
    }
}

You can test this (in Java) like this:

String source = "... your input here ...";
TLexer lexer = new TLexer(CharStreams.fromString(source));
TParser parser = new TParser(new CommonTokenStream(lexer));
TestVisitor visitor = new TestVisitor();
visitor.visit(parser.thingList());

writing a typesafe visitor with labeled rules

1 Answers

EDIT