Return inherited Python class based on C++ return types

Question

Let's say I have wrapped my C++ classes Foo and Bar and can access them from Python just fine through the SWIG generated module wrap_py:

// C++

class Bar
{
    int i;
    Bar(int i) { this.i = i; }
}

class Foo 
{
public:
    Foo(Bar* bar) { this.bar = bar; }
    Bar* GetBar() { return this.bar; }
private:
    Bar* bar;
}

In Python, I created my user facing class that are a shallow proxy mainly adding docstrings and allowing the IDE to do tab completion on the argument names:

// Python
class Bar(wrap_py.Bar):
    '''Some description ...
    Args:
       i (int): ...
    '''
    def __init__(self, i): 
        super(Bar, self).__init__(i)

class Foo(wrap_py.Foo):
    '''Some description ...
    Args:
       bar (instance of Bar): ...
    '''
    def __init__(self, bar): 
        super(Foo, self).__init__(bar)

The problem is here that Foo.GetBar(), which is automatically generated from the C++ class, returns the swig instance of type wrap_py.Bar, which doesn't have the docstrings and also doesn't show the parameter names (swig exposes all parameters as *args). Instead, I would want it to provide my own shallow proxy Bar.

So, how can I tell SWIG, to automatically return Bar instead of the bare wrap_py.Bar?

EDIT: Ideally, this would be possible for the return type Bar and not only for a concrete function signature:

%feature("shadow") Foo::GetBar() %{ def bar(*args): result = $action result.__class__ = Bar return result %}

EDIT 2: I've come up with the following decorator, which I need to put in front of every function/method that returns a SWIG type:

def typemap(f):
from functools import wraps @wraps(f) def wrapper(*args, **kwds): typemap = { wrap_py.Bar: Bar, # more types to come... } result = f(*args, **kwds) if isinstance(result, (tuple, list, set)): for r in result: r.__class__ = typemap.get(r.__class__, r.__class__) elif isinstance(result, dict): for k,v in result.items(): k.__class__ = typemap.get(k.__class__, k.__class__) v.__class__ = typemap.get(v.__class__, v.__class__) else: result.__class__ = typemap.get(result.__class__, result.__class__) return result return wrapper Definitely, this is not nice and calls for omissions.

Answer 1

There's a problem with both of the solutions that you've proposed. Consider the following test case:

b=Bar(1)
b.woof=2
print(b.woof)
g=(Foo(b).GetBar())
print(type(g))
print(g.woof)

In that example we expect the final print statement to have the same value for the 'woof' attribute as the original Bar object we created did. That is to say we'd expect not only the type to match, but for it to be the same instance. With both the shadow and the decorator approach to wrapping things you're still creating new Python objects each time when returning the same underlying C++ Bar instance.

To work around that what you probably want to do is set up a dictionary mapping original C++ objects 1:1 onto Python proxy objects and use that everywhere there's a Bar object returned.

As a starting point to illustrate this I've set up the following example. Your C++ had multiple issues fixed in it and became test.hh:

class Bar
{
    int i;
public:
    Bar(int i) { this->i = i; }
};

class Foo 
{
public:
    Foo(Bar* bar) { this->bar = bar; }
    Bar* GetBar() { return this->bar; }
private:
    Bar* bar;
};

I wrote a test.i SWIG wrapper that extended Bar to provide __hash__ based on the address of C++ objects:

%module test
%{
#include "test.hh"
%}

%include <stdint.i>
%include "test.hh"

%extend Bar {
  intptr_t __hash__() {
    return reinterpret_cast<intptr_t>($self);
  }
}

And then finally wrap.py was extended from your Python to implement the object mapping and instance lookup, including overriding GetBar to use this mechanics:

import test as wrap_py

class Bar(wrap_py.Bar):
    '''Some description ...
    Args:
       i (int): ...
    '''
    def __init__(self, i): 
        super(Bar, self).__init__(i)
        Bar._storenative(self)

    _objs={}

    @classmethod
    def _storenative(cls, o):
        print('Storing: %d' % hash(o))
        cls._objs[hash(o)]=o

    @classmethod
    def _lookup(cls, o):
        print('Lookup: %d' % hash(o))
        return cls._objs.get(hash(o), o)

class Foo(wrap_py.Foo):
    '''Some description ...
    Args:
       bar (instance of Bar): ...
    '''
    def __init__(self, bar): 
        super(Foo, self).__init__(bar)

    def GetBar(self):
        return Bar._lookup(super(Foo, self).GetBar())

if __name__=='__main__':
  b=Bar(1)
  b.woof=2
  print(b.woof)
  g=(Foo(b).GetBar())
  print(type(g))
  print(g.woof)

There are a few issues with this first cut though. Firstly as you noted we still have to manually override each and every function that could return an instance of Bar to add the extra lookup call. Secondly the lookup dictionary can cause the Python proxy objects to outlive their C++ counterparts (and in the worst case incorrectly map a Python Bar proxy onto a C++ object that is not really proxied by any Python derived object. To solve the latter problem we could look at weak references, but that too has flaws (the Python objects can get destroyed prematurely instead).

To get this to work transparently for all methods which return Bar instances you could go down one of two roads:

1. Implement __getattribute__ in your proxy class, have it return a function that wrapped and did lookups appropriately based on the return type.
2. Write an out typemap in SWIG that does something similar to the above, but based on C++ code not Python code for the mechanics.

To implement #2 you'll just need to write a single %typemap(out) Bar* which looks to see if this is the first time we've seen an instance of Bar at a given address and returns a reference to the same object if was seen before, or creates a new one otherwise. Note that you'll need to use swig -builtin if you aren't already to prevent the intermediate proxy object making this harder than it needed to be. So our interface can simply become:

%module test
%{
#include "test.hh"
#include <map>

namespace {
  typedef std::map<Bar*,PyObject*> proxy_map_t;
  proxy_map_t proxy_map;
}
%}

%typemap(out) Bar* {
  assert($1);
  const auto it = proxy_map.find($1);
  if (it != proxy_map.end()) {
    $result = it->second;
  }
  else {
    $result = SWIG_NewPointerObj(SWIG_as_voidptr($1), $1_descriptor, $owner);
    proxy_map.insert(std::make_pair($1, $result));
  }
  Py_INCREF($result);
}

%include "test.hh"

Which then compiles and runs with the Python unmodified from above.

swig3.0 -python -c++ -Wall -builtin test.i
g++ -shared -o _test.so test_wrap.cxx -Wall -Wextra -fPIC -I/usr/include/python2.7/ -std=c++11
python wrap.py

There's one outstanding issue with this still: we don't get to see when Bar* instances get deleted, so we can end up in a situation where we accidentally recycle our Python proxy objects across the life of multiple C++ ones. Depending on what you're aiming to do you could use weak reference inside the map to work around this, or you could (ab)use operator new() to hook the creation of Bar* instances.