How to do scoping for qualified names in a C++-like language

Question

I've implemented a grammar for a language that's best described as a scriptable version of C++, without the preprocessor. I'm trying to get the scoping working. (The grammar is about 500 lines, so that might give you a sense of how much of C++ syntax is included and how much has been left out. It includes enums, classes, global and class member functions, as well as some funky stuff that is not really relevant here.)

I feel a little stupid asking such a broad question that probably has a simple answer, but I feel like at this point I've burned enough time bouncing between the XText documentation, the book, web searches, the relevant blogs, and looking at the XText code that it's better to ask.

I'm trying to figure out scoping. Some of the key pieces I have working already:

1. It provides the appropriate qualified names, so that object member variable x of a class C is C.x. In theory, this should make it possible, once I have a type system (like the one in the XText book) working, for me to determine an object's type and then use that to import that class's scope.
2. In places in the grammar where it's possible for a lexer rule to have multiple corresponding data types, I've figured out how to use ref=[ecore::EObject|GenericDataTypeRule] where I have GenericDataTypeRule: name=ID;

So the question is as follows:

How do I (efficiently!) get my MyCppLikeDSLScopeProvider to allow the following kinds of references:

class MyClass {
    void memberFunction();
    Integer j;
    ...
}

Integer MyClass::memberFunction() {
     return j;
}

Some things I've tried:

• I've tried using Scopes.scopeFor(context.getAllContentsOfType(ClassDecl), and that works fine if the class is defined in the same file, but somehow I can't figure out how to get this to work scalably if the class is in another Resource. For example, if everything is within the same file, and I'm looking for something that could be either a class, variable, function, or enum, I can do it:

  def scope_SymbolicValue_ref(EObject context, EReference eRef) {
      var Iterable<EObject> crossRefClassDeclTargets = context.getAllContentsOfType(ClassDecl).map[it as EObject]
      var Iterable<EObject> crossRefDataDefTargets = context.getAllContentsOfType(DataDef).map[it as EObject]
      var Iterable<EObject> crossRefFxnTargets = context.getAllContentsOfType(FunctionSpec).map[it as EObject]
      var Iterable<EObject> crossRefEnumTargets = context.getAllContentsOfType(SimpleEnum).map[it as EObject]
  
      var List<EObject> allCrossRefTargets = new ArrayList()
      allCrossRefTargets.addAll(crossRefClassDeclTargets)
      allCrossRefTargets.addAll(crossRefDataDefTargets)
      allCrossRefTargets.addAll(crossRefEnumTargets)
      allCrossRefTargets.addAll(crossRefFxnTargets)
      return Scopes.scopeFor(allCrossRefTargets)
  }
  

• class inheritance is not included in this example but I have managed to get that to work following the example from the book and customizing for multiple inheritance, although it doesn't really do a proper DFS or BFS, it's more like a "stupid but terminating search": at each step, look at edges from every vertex, and at the end of the step see whether the number of vertices is equal to the number of vertices you started with; so maybe I should speed that up by using a non-stupid graph search.
• I did try using context.resourceSet and iterating over all the resources and filtering for all classes, but my codebase is about 300k lines in about 300 files, and this was way too slow.
• I've tried to write a custom class that implements IScope. Scopes.ScopeFor translates a list of EObject's into a list of EObjectDescription's, but what if I have two lists of EObjectDescription's (for example, I can get that from two different classes that implement IScope) that I want to combine? Surely there must be a trivial way to take the union of two scopes and I'm just brain-dead to not be finding it.
• I've started contemplating just writing my own cache of objects, which would move me to a completely 100% customized class that implements IScope from scratch, but clearly XText has quite a number of classes that inherit from AbstractScopeProvider and I just need to figure out which one I need to use.

Thanks! I'm happy to write more detail but I imagine this is trivial.

Answer 1

If you need to be able to reference any globally visible class/method/etc. from the current or other models, this is done by the global scope provider, which accesses the Xtext index.

In this case you need a rule like

MethodImpl:
  containingClass=[ClassDeclaration] '::' method=[Method] ;

In the scope provider you have to define a scope method for the "method" property which creates a scope for all methods of containingClass, e.g.:

def scope_MethodImpl_method(MethodImpl methodImpl, EReference ref) {
  Scopes::scopeFor(methodImpl.containingClass.members.filter(Method))
}

The trickier part is the scoping for containingClass: C++ doesn't have anything like an import mechanism. You can only provide a method implementation for a class you defined before in the same file or in a directly or indirectly #included header file. So if your subset of C++ allows #include, you somehow have to emulate this behaviour (This might involve creating a custom IDefaultResourceDescriptionStrategy or similar stuff, but that was not part of the question).

In any case, it will probably make most sense to use a cache, as you wrote, for which you should use IResourceScopeCache to cache all classes from directly or indirectly included files. Ignoring cyclic includes, it might look something like this:

@Inject
IResourceScopeCache cache

def Set<ClassDeclaration> getImportedClasses(Module module) {
    cache.get("INCLUDED_CLASSES", module.eResource) [
        (module.classDeclarations + module.includes.map[module.importedClasses].flatten).toSet
    ]
}