Boost.Python: Fill in a passed in buffer in Python

Question

I was wondering whether it's possible to fill in a buffer (with the following conditions) in Python and if so how?

I have a buffer in C++ that I need to fill in Python. The Address of the buffer is obtained through the GetAddress method which returns a void pointer to the buffer's address.

#include <boost/smart_ptr/shared_ptr.hpp>
class Foo
{
public:
    Foo(const unsigned int length)
    {
        m_buffer = boost::shared_ptr< unsigned char >( new unsigned char[ length ] );
    }

    ~Foo(){}

    void* GetAddress( ) const
    {
        // cast for the sake of this question
        return reinterpret_cast< void* >( m_buffer.get() );
    }

private:
    boost::shared_ptr< unsigned char > m_buffer;
    Foo();
    Foo(const Foo&);
};

Using Py++ I can generate the Boost.Python wrapper to export the class to Python as follows:

#include "boost/python.hpp"
#include "foo.hpp"

namespace bp = boost::python;

BOOST_PYTHON_MODULE(MyWrapper){
    { //::Foo
        typedef bp::class_< Foo, boost::noncopyable > Foo_exposer_t;
        Foo_exposer_t Foo_exposer = Foo_exposer_t( "Foo", bp::init< unsigned int >(( bp::arg("length") )) );
        bp::scope Foo_scope( Foo_exposer );
        bp::implicitly_convertible< unsigned int const, Foo >();
        { //::Foo::GetAddress

            typedef void * ( ::Foo::*GetAddress_function_type )(  ) const;

            Foo_exposer.def( 
                "GetAddress"
                , GetAddress_function_type( &::Foo::GetAddress )
                , bp::return_value_policy< bp::return_opaque_pointer >() );

        }
    }
}

In Python, the output of the GetAddress is a void * to the memory address:

>>> import MyWrapper
>>> foo = MyWrapper.Foo(100)
>>> address = foo.GetAddress()
>>> print address
<void * object at 0x01E200B0>
>>>

Problem is Python doesn't let me do anything with the void * address object. If I try to access the second element in buffer, none of the following work:

>>> address + 1
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: unsupported operand type(s) for +: 'void *' and 'int'
>>> address[1]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: 'void *' object is unsubscriptable
>>>

Environment: Visual Studio 2008, Boost 1.44, gcc-xml 0.9.0, py++ 1.0.0, pygccxml 1.1.0, Python 2.6.6

So... what is your question? What doesn't work as expected? What can't you figure out how to do, exactly? — Karl Knechtel
Problem is I can get the address of the buffer as a void pointer but I can't do anything with it. For example, it doesn't support iterating over and value assignment. All in all good point! I'll edit my question.. — Ehsan
So what sort of interface for filling the buffer are you looking for? Indexing in? Treating it as a stream? Have you looked at existing classes like buffer, StringIO etc.? What are you going to do with the buffer once it's filled? — Karl Knechtel
I have looked at buffer and StringIO but don't know how I can pass the void pointer object to them. — Ehsan

Matthew Scouten Matthew Scouten · Accepted Answer · 2012-05-01T21:59:42

Python does not really deal with pointers. You can export one as a opaque cookie, but you can never do anything with it (except pass it back to c++).

What I would do in your case, is to reverse it. Instead of returning to pointer to python, have a function in c++ that takes a "buffer" from python.

namespace bp = boost::python
void FillBuffer(Foo& this, bp::list buff)
{
    unsigned char* addr = reinterpret_cast< unsigned char* >( this.GetAddress() );
    for(int i = 0; i < bp::len(buff); i++)
        addr[i] = bp::extract< unsigned char >( buff[i] );
}

Foo_exposer.def("FillBuffer", &FillBuffer);

Now you can pass a list in to fill the buffer. You can create an similar function to stuff a buffer into a list and return it python. You will, of course, want to be much more careful about buffer overruns and such, but this should give you the right idea.

Boost.Python: Fill in a passed in buffer in Python

1 Answers