Scala Macro Annotations: c.TypeCheck of annotated type causes StackOverflowError

Question

I have a macro annotation which is intended to be applied to class definitions. Its purpose is an almost-but-not-quite serialization tool. It inspects the class's constructor parameters and then creates a factory method on the companion object which in turn supplies values for the parameters. It needs to know the types of the parameters in order to do this, so I've been calling Context.typeCheck on them.

The problem occurs when the annotated class's constructor takes a parameter of the same type as itself, or in other similar situations (for instance, if type A and type B both are annotated, and A has a parameter of B, and B has a parameter of A. Type parameters applied to the formal parameters count also). Any of these situations will result in the annotation being recursively invoked until StackOverflowError occurs.

I tried using "withMacrosDisabled=true" as an argument to c.typeCheck, and while this solves the problem, it introduces a different one. If the type being checked has not been previously seen, then the compiler remembers its definition, and its macros are never invoked at all. This isn't a problem for the self-reference case, but it does happen in the mutual reference case.

So I'm stuck. Is there a workaround? Can I solve this with c.openMacros?

Another option, if available, is that I don't strictly need the full definition of the type, I could get by with just its fully qualified name (scala.xml.NodeSeq instead of just NodeSeq). I get the TypeName in the AST, but these are rarely fully qualified, and I don't know how to get the fully qualified name without doing a full typeCheck.

As a side question, what is "withMacrosDisabled" good for? If using it prevents all macro expansion forever on the types found in the passed tree, not just for the current c.typeCheck, that seems like too big of a hammer. And you can't really use it even if that's actually what you want, because the macro evaluation would then depend on the order the types are encountered in their own source.

Edit: Thinking about it, I think the compiler should ensure that each macro is expanded exactly once. In the case of a cycle, as in my example, at least one of the macros involved would still see an incompletely-processed class, which seems unavoidable in a case like this as it is, in effect, a circular dependency. I guess, a flag on the resulting Type to indicate that macro processing is not final would be the best way to deal with it, but that probably can't be done in Paradise.

Answer 1

This seems to be quite related to the discussion started in Can't access Parent's Members while dealing with Macro Annotations (also see a link to a more lengthy elaboration in my answer over there).

If possible I would like to avoid situations when macros see half-expanded or half-populated types in order to reduce the potential for confusion. Last months I've been thinking about ways to avoid that, but there's been a number of higher-priority distractions, so I haven't gotten far yet.

Two potential ideas that I'm pondering are: 1) come up with a notation to specify effects of macro annotations, so that we don't have to expand macros in order to know what classes and members comprise our program (if that works out, then the macro engine can first precompute the list of members and only then launch macro expansions), 2) figure out a mechanism of specifying how macros depend on program elements, so that expansions are ordered correctly. Just yesterday I've also learned about Backstage Java and David Herman's work on typed hygienic macros - this should also be relevant. What do you think about these directions of thought?

In the meanwhile, while I'm trying to figure out a principled solution to the problem of dependencies, I'm also interested in unblocking your use case by providing a workaround or a patch to paradise that would be immediately useful. Could you elaborate on your project, so that we could come up with a fix?

Answer 2

The workaround I ended up using is this:

val open = c.openMacros
val checkRecursion = open.count({check=>(c.macroApplication.toString == check.macroApplication.toString) && (c.enclosingPosition.toString == check.enclosingPosition.toString)})
if (checkRecursion > 2) // see note
  {do something to terminate macro expansion}

When you terminate the macro expansion, you can't just throw an exception (unless you catch it later), you have to return a valid tree (I just return the original input).

The effect of this is that whichever macro-annottee got evaluated first will end up short-circuiting its evaluation when it gets encountered the second time, after the compiler has initiated macro expansions of the whole graph cycle. At this point every annottee in the cycle will have a macro in-flight, all waiting on typechecks of each other. The version of the annottee returned by the short-circuit macro will then be used by these typechecks. (In mine, I just return the original input, but you could in principle do anything that doesn't need to do typechecks). However, the eventual output that the rest of the world sees, after macro expansion is done, is the output of the top-level macro. Caveat: I return entirely un-typechecked trees as the output of my macro - not sure what would happen if you returned a tree that had this inconsistent typechecking done on it. Probably nothing good.

In a simple graph with one cycle in it, every macro will see a fully-processed version of every class except the class that originally triggered the cycle. But more complex dependencies could result in macros potentially appearing in various states of expansion or non-expansion when seen by each other.

In my code this is good enough because I only need to check the class's name and type conformance and my macros don't change these things. IOW my dependency isn't really circular, the compiler just thinks it is.

note: checkRecursion gets compared against 2 because for some reason the current macro expansion always appears twice in the result of c.openMacros.