Is it possible to dynamically create a metaclass for a class with several bases, in Python 3?

4
votes

In Python 2, with a trick it is possible to create a class with several bases, although the bases have metaclasses that are not subclass of each other.

The trick is that these metaclasses have themselves a metaclass (name it a "metametaclass"), and this metametaclass provides the metaclasses with a call method that dynamically creates a common sub-metaclass of the base metaclasses, if necessary. Eventually, a class results whose metaclass is the new sub-metaclass. Here is the code:

>>> class MetaMeta(type):
...     def __call__(mcls, name, bases, methods):
...         metabases = set(type(X) for X in bases)
...         metabases.add(mcls)
...         if len(metabases) > 1:
...             mcls = type(''.join([X.__name__ for X in metabases]), tuple(metabases), {})
...         return mcls.__new__(mcls, name, bases, methods)
... 
>>> class Meta1(type):
...     __metaclass__ = MetaMeta
... 
>>> class Meta2(type):
...     __metaclass__ = MetaMeta
... 
>>> class C1:
...     __metaclass__ = Meta1
... 
>>> class C2:
...     __metaclass__ = Meta2
... 
>>> type(C1)
<class '__main__.Meta1'>
>>> type(C2)
<class '__main__.Meta2'>
>>> class C3(C1,C2): pass
... 
>>> type(C3)
<class '__main__.Meta1Meta2'>

This example (of course changing the syntax to class C1(metaclass=Meta1) etc) doesn't work in Python 3.

Question 1: Do I understand correctly that in Python 2, first C3 is constructed, using the metaclass of the first base, and an error would only result if type(C3) were not a common subclass of type(C1) and type(C2), whereas in Python 3 the error is raised earlier?

Question 2: (How) Is it possible to make the above example work in Python 3? I did try to use a subclass of abc.ABCMeta as metametaclass, but even though using a custom __subclasscheck__ makes issubclass(Meta1, Meta2) return True, the creation of C3 would still result in an error.

Note: Of course I could make Python 3 happy by statically defining Meta1Meta2 and explicitly using it as a metaclass for C3. However, that's not what I want. I want that the common sub-metaclass is created dynamically.

3
pythonmetaclass
This feels a little bit like complexity for the sake of making things complex :-). Why do you want to do this instead of the more easy to read and understand variant where you create a new metaclass for C3 by mixing together C1 and C2's metaclasses explicitly? - mgilson
I am a developer of SageMath, whose user language currently is Python 2.7. In that CAS, different metaclasses implementing separate features are used. Say, five features, which means that there are 32 possible feature combinations. It is a bit awkward to statically define each possible combinations, and in fact SageMath currently provides only some combinations. In addition, creating new classes should be easy for the user: It should not be needed for the user to look up the metaclasses of all bases and choose the right combined metaclass explicitly. - Simon King

3 Answers

2
votes

In Python 3 at the time the metaclass is used it have to be ready, and it can't know about the bases of the final (non-meta) class in order to dynamically create a metaclass at that point.

But instead of complicating things (I confess I could not wrap my head around your need for a meta-meta-class) - you can simply use normal class hierarchy with collaborative use of super for your metaclasses. You can even build the final metaclass dynamically with a simple call to type:

class A(type):
    def __new__(metacls, name, bases,attrs):
        attrs['A'] = "Metaclass A processed"
        return super().__new__(metacls, name, bases,attrs)


class B(type):
    def __new__(metacls, name, bases,attrs):
        attrs['B'] = "Metaclass A processed"
        return super().__new__(metacls, name, bases,attrs)


C = type("C", (A, B), {})

class Example(metaclass=C): pass

And:

In[47] :Example.A
Out[47]: 'Metaclass A processed'

In[48]: Example.B
Out[48]: 'Metaclass A processed'

If your metaclasses are not designed to be collaborative in the first place, it will be very tricky to create any automatic method to combine them - and it would possibly involve monkey-patching the call to type.__new__ in some of the metaclasses constructors.

As for not needing to explictly build C, you can use a normal function as the metaclass parameter, that will inspect the bases and build a dynamic derived metaclass:

def Auto(name, bases, attrs):
    basemetaclasses = []
    for base in bases:
        metacls = type(base)
        if isinstance(metacls, type) and metacls is not type and not metacls in basemetaclasses:
            basemetaclasses.append(metacls)
    dynamic = type(''.join(b.__name__ for b in basemetaclasses), tuple(basemetaclasses), {})
    return dynamic(name, bases, attrs)

(This code is very similar to yours - but I used a three-line explicit for instead of a set in order to preserve the metaclass order - which might matter)

You have them to pass Auto as a metaclass for derived classes, but otherwise it works as in your example:

In [61]: class AA(metaclass=A):pass

In [62]: class BB(metaclass=B):pass

In [63]: class CC(AA,BB): pass
---------------------------------------------------------------------------
...
TypeError:   metaclass conflict
...

In [66]: class CC(AA,BB, metaclass=Auto): pass

In [67]: type(CC)
Out[67]: __main__.AB

In [68]: CC.A
Out[68]: 'Metaclass A processed'
0
votes

Here's an example that shows some options that you have in python3.x. Specifically, C3 has a metaclass that is created dynamically, but in a lot of ways, explicitly. C4 has a metaclass that is created dynamically within it's metaclass function. C5 is just to demonstrate that it too has the same metaclass properies that C4 has. (We didn't lose anything through inheritance which can happen if you use a function as a metaclass instead of a type ...)

class Meta1(type):
    def foo(cls):
        print(cls)


class Meta2(type):
    def bar(cls):
        print(cls)


class C1(object, metaclass=Meta1):
    """C1"""


class C2(object, metaclass=Meta2):
    """C2"""


class C3(C1, C2, metaclass=type('Meta3', (Meta1, Meta2), {})):
    """C3"""

def meta_mixer(name, bases, dct):
    cls = type('MixedMeta', tuple(type(b) for b in bases), dct)
    return cls(name, bases, dct)


class C4(C1, C2, metaclass=meta_mixer):
    """C4"""


C1.foo()
C2.bar()

C3.foo()
C3.bar()

C4.foo()
C4.bar()

class C5(C4):
    """C5"""

C5.foo()
C5.bar()

It should be noted that we're playing with fire here (the same way that you're playing with fire in your original example). There is no guarantee that the metaclasses will play nicely in cooperative multiple inheritance. If they weren't designed for it, chances are that you'll run into bugs using this at some point. If they were designed for it, there'd be no reason to be doing this hacky runtime mixing :-).

0
votes

In 95% of cases, it should be possible to use the machinery introduced in Python 3.6 due to PEP 447 to do most of what metaclasses can do using special new hooks. In that case, you will not need to combine metaclasses since your hooks can call super and their behavior is combined due to inheritance.

As for your general case, I believe that mgilson is right and that you are probably making things too complicated. I have yet to see a case for combining metaclasses that is not covered by PEP 447.