16
votes

I want to get some string from a C/C++ library with ctypes into python. My code looks like this:

Code in lib:

const char* get(struct something *x) 
{
    [...]
    // buf is a stringstream
   return strdup(buf.str().c_str());
}

void freeme(char *ptr)
{
    free(ptr);
}

Python code:

fillprototype(lib.get, c_char_p, POINTER(some_model)])
fillprototype(lib.freeme, None, [c_char_p])

// what i want to do here: get a string into python so that i can work
// with it and release the memory in the lib.
c_str = lib.get(some_model)
y = ''.join(c_str)
lib.freeme(c_str) 

If i print() c_str, everything is there. Problem is (or seems to be) in the last Python line. I cannot free the memory - the library is getting a wrong pointer. What I am doing wrong here? (And please don't suggest boost::python or so).

*** glibc detected *** python2: munmap_chunk(): invalid pointer: 0x00000000026443fc ***
2
Why does get return a const char * but freeme expect char *? Are you expecting your callers to cast away const? - David Schwartz
good point.. unfortunately that didn't solve the problem. - snøreven
Is this 32-bit or 64-bit code? And do you know what line of code the error occurred in? It isn't necessarily the free(ptr). - David Schwartz
64bit. I don't know which line it is: gdb bt shows me only a lot of python stuff. But I just printed ptr in get() and in freeme() and got this: "allocated memory ptr: 0x2592a20", "free memory ptr: 0x7f2bf5fad3fc", "python2: munmap_chunk(): invalid pointer: 0x00007f2bf5fad3fc" - snøreven
I bet Python created its own string, just like C++ would do if you did string c_str = lib.get(some_model);. - David Schwartz

2 Answers

29
votes

As David Schwartz pointed out, if you set restype to c_char_p, ctypes returns a regular Python string object. A simple way to get around this is to use a void * and cast the result:

string.c:

#include <stdlib.h>
#include <string.h>
#include <stdio.h>

char *get(void)
{
    char *buf = "Hello World";
    char *new_buf = strdup(buf);
    printf("allocated address: %p\n", new_buf);
    return new_buf;
}

void freeme(char *ptr)
{
    printf("freeing address: %p\n", ptr);
    free(ptr);
}

Python usage:

from ctypes import *

lib = cdll.LoadLibrary('./string.so')
lib.freeme.argtypes = c_void_p,
lib.freeme.restype = None
lib.get.argtypes = []
lib.get.restype = c_void_p

>>> ptr = lib.get()
allocated address: 0x9facad8
>>> hex(ptr)
'0x9facad8'
>>> cast(ptr, c_char_p).value
'Hello World'
>>> lib.freeme(ptr)
freeing address: 0x9facad8

You can also use a subclass of c_char_p. It turns out that ctypes doesn't call the getfunc for a subclass of a simple type.

class c_char_p_sub(c_char_p):
    pass

lib.get.restype = c_char_p_sub

The value attribute returns the string. You can leave the parameter for freeme as the more generic c_void_p. That accepts any pointer type or integer address.

0
votes

a cheap hack around freeing the memory is to allocate it statically in C and never free it. only works if you know the maximum amount you need, of course, and if you know that you're done with it before a second call to your routine.

static char _externalStringBuffer[MAX_BUFFER_SIZE];
char *get() {
    // put characters in _externalStringBuffer
    return _externalStringBuffer;
}