Python/Cython/C and callbacks, calling a Python function from C using Cython

Question

I have the following question. We have to pass callback functions to the C code. If the function is a Cython function in the same module, the situation is quite simple

In Cython:

def callme(int x):
    c_callme(x, <int (*)(int)>&callbackme) 

cdef int callbackme(int x):
    print <int> x
    return <int>x

In C:

int c_callme(int x, int (*f)(int))
{
    printf("---%d\n",x);
    printf("--%d\n",f(x));
    return x;
}

The question is as follows: we want to generalize this code in the most Pythonic way so that it can accept also python functions as callback arguments (of course, some additional layer is needed), and also C/Cython functions from another module. I suppose, for C/Cython functions from a separate module one has to get the address of these functions (convert to long int?) and for Python function a certain wrapper is needed

Answer 1

In this example, extracted from a Python wrapper to the Cubature integration C library, a Python function is passed to a C function that has the prototype given by cfunction. You can create a function with the same prototype, called cfunction_cb (callback), and returning the same type, int in this example):

cdef object f
ctypedef int (*cfunction) (double a, double b, double c, void *args)

cdef int cfunction_cb(double a, double b, double c, void *args):
    global f
    result_from_function = (<object>f)(a, b, c, *<tuple>args)
    for k in range(fdim):
        fval[k] = fval_buffer[k]
    return 0

When calling the C function, you can cast your callback wrapper using the C prototype:

def main(pythonf, double a, double b, double c, args): 
    global f
    f = pythonf
    c_function( <cfunction> cfunction_cb,
                double a,
                double b,
                double c,
                <void *> args )

In this example it is also shown how to pass arguments to your Python function, through C.

Answer 2

I think the easiest way would be to wrap C callbacks in

cdef class Callback(object):
    cdef int (*f)(int)        # I hope I got the syntax right...

    def __call__(int x):
        return self.f(x)

so you can pass objects of this type, as well as any other callable, to the function that must call the callback.

However, if you must call the callback from C, then you should may pass it an extra argument, namely the optional address of a Python object, perhaps cast to `void*.

(Please do not cast function pointers to long, or you may get undefined behavior. I'm not sure if you can safely cast a function pointer to anything, even void*.)