Cython extension class: How do I expose methods in the auto-generated C struct?

Question

I have existing C++ code that defines some classes I need to use, but I need to be able to send those classes to Python code. Specifically, I need to create class instances in C++, create Python objects to serve as wrappers for these C++ objects, then pass these Python objects to Python code for processing. This is just one piece of a larger C++ program, so it needs to be done ultimately in C++ using the C/Python API.

To make my life easier, I have used Cython to define extension classes (cdef classes) that serve as the Python wrappers for my C++ objects. I am using the typical format where the cdef class contains a pointer to the C++ class, which is then initialized when the cdef class instance is created. Since I also want to be able to replace the pointer if I have an existing C++ object to wrap, I have added methods to my cdef classes to accept() the C++ object and take its pointer. My other cdef classes successfully use the accept() method in Cython, for example when one object owns another.

Here is a sample of my Cython code:

MyCPlus.pxd

cdef extern from "MyCPlus.h" namespace "mynamespace":
    cdef cppclass MyCPlus_Class:
        MyCPlus_Class() except +

PyModule.pyx

cimport MyCPlus
from libcpp cimport bool

cdef class Py_Class [object Py_Class, type PyType_Class]:
    cdef MyCPlus.MyCPlus_Class* thisptr
    cdef bool owned

    cdef void accept(self, MyCPlus.MyCPlus_Class &indata):
        if self.owned:
            del self.thisptr
        self.thisptr = &indata
        self.owned = False

    def __cinit__(self):
        self.thisptr = new MyCPlus.MyCPlus_Class()
        self.owned = True

    def __dealloc__(self):
        if self.owned:
            del self.thisptr

The problem comes when I try to access the accept() method from C++. I tried using the public and api keywords on my cdef class and on the accept() method, but I cannot figure out how to expose this method in the C struct in Cython's auto-generated .h file. No matter what I try, the C struct looks like this:

PyModule.h (auto-generated by Cython)

struct Py_Class {
  PyObject_HEAD
  struct __pyx_vtabstruct_11PyModule_Py_Class *__pyx_vtab;
  mynamespace::MyCPlus_Class *thisptr;
  bool owned;
};

I also tried typing the self input as a Py_Class, and I even tried forward-declaring Py_Class with the public and api keywords. I also experimented with making accept() a static method. Nothing I've tried works to expose the accept() method so that I can use it from C++. I did try accessing it through __pyx_vtab, but I got a compiler error, "invalid use of incomplete type". I have searched quite a bit, but haven't seen a solution to this. Can anyone help me? Please and thank you!

static void __pyx_f_8PyModule_8Py_Class_accept(struct __pyx_obj_8PyModule_Py_Class *__pyx_v_self, mynamespace::MyCPlus_Class &__pyx_v_indata); /* proto*/ looks plausible as something you could use. — DavidW
@DavidW, That might work for a time, but I'm really looking for a solution of how to tell Cython to put this method into my API. The line you quoted above is buried deep in the auto-generated .cpp file, and it could change unexpectedly. I need a more maintainable solution. Thank you for the attempt, though. — Carrie D.
The other thing you could try is declaring accept() in both the pxd and pyx files – if you have it as a fully declared class method (not just a extern method prototype to be exposed) and Cython generates an correspondingly named accessible method, it’d be yours to either call or override — fish2000
@fish2000 The accept() method is not part of my original C++ class (and I can't add it) so there would be no reason to put in in the pxd. What I need is for it to be included in the C-struct of my extension class so I can create an extension class wrapper for my C++ object that I can then send to Python. It's funny because Cython does put in that __pyx_vtab, but that seems to be for Cython use only, not for external C/C++ code. I'm thinking that Cython just isn't designed to do this yet. — Carrie D.

Edward Elrick Edward Elrick · Accepted Answer · 2017-02-10T05:54:06

As you pointed in your comment, it does seem that the __pyx_vtab member is for Cython use only, since it doesn't even define the struct type for it in the exported header(s).

Adding to your response, one approach could also be:

cdef api class Py_Class [object Py_Class, type Py_ClassType]:
    ...
    cdef void accept(self, MyCPlus.MyCPlus_Class &indata):
        ...  # do stuff here
    ...


cdef api void (*Py_Class_accept)(Py_Class self, MyCPlus.MyCPlus_Class &indata)
Py_Class_accept = &Py_Class.accept

Basically, we define a function pointer and set it to the extension's method we want to expose. This is not that much different to your response's cdef'd function; the main difference would be that we can define our methods as usual in the class definition without having to duplicate functionality or method/function calls to another function to expose it. One caveat is that we would've to define our function pointer's signature almost verbatim to the method's one in addition to the self's extension type (in this case) and etc; then again this also applies for regular functions.

Do note that I tried this up on a C-level Cython .pyx file, I haven't and do not intent to test it on a CPP implementation file. But hopefully this might work just as fine, I guess.

Cython extension class: How do I expose methods in the auto-generated C struct?

3 Answers