Interfacing C++ with Python

Question

I have a c++ (C++ 17) library and a function dependent on this library. I want to call this function from python(python 3). How can it be done?

I have read some tutorials using swig and ctypes but I don't know how can I make it work for an external shared library.

Details: I have this libclickhouse-cpp-lib.so .so file and the header files for this library in /path/to/header/files directory.

/*File cpp_reader.h*/
#pragma once
int reader();

/* File cpp_reader.cpp*/
#include <clickhouse/client.h>
#include <iostream>
#include "cpp_reader.h"
using namespace clickhouse;
int reader()
{
    Client client(ClientOptions().SetHost("localhost"));
    int numrows=0;
    client.Select("SELECT count(*) from test.test_table",
        [&numrows] (const Block& block)
        {
            for (size_t i=0;i<block.GetRowCount();++i)
            {
                numrows+=block[0]->As<ColumnUInt64>()->At(i);
            }
        }
    );
    return(numrows);
}

I want to call this read function from python. I have gone through some posts using swig and ctypes but haven't been able to figure out. If it can be done easily using anything else, please suggest that too.

Additional Information: This is how I run the code in c++

/*File: main.cpp*/
#include <clickhouse/client.h>
#include <iostream>
#include <Python.h>
using namespace clickhouse;
int main()
{
    /// Initialize client connection.
    Client client(ClientOptions().SetHost("localhost"));
    int numrows=0;
    client.Select("SELECT count(*) from test.test_table",
        [&numrows] (const Block& block)
        {
            for (size_t i=0;i<block.GetRowCount();++i)
            {
                numrows+=block[0]->As<ColumnUInt64>()->At(i);
            }
        }
    );
    std::cout<<"Number of Rows: "<<numrows<<"\n";
}

Compilation:
g++ -std=c++1z main.cpp -I/path/to/header/files -I/usr/include/python3.6m/ -L. /path/to/libclickhouse-cpp-lib.so -o outfile

LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/path/to/so_file/directory
export LD_LIBRARY_PATH

/path/to/so_file/directory contains libclickhouse-cpp-lib.so

./outfile

pybind11 is the easiest way in my experience. boost.python is also decent if you are already using Boost. I wouldn't recommend ctypes unless you are just hacking around with hobby projects, and I wouldn't recommend swig unless you need to expose your C++ to a bunch of other languages in addition to Python. — 0x5453
I see no reason for cpp_reader.h to #include <clickhouse/client.h> or even iostream. I suggest you remove them to lessen the confusion on any binding generator — Omer Tuchfeld
Did you try google.com/search?q=how+to+write+a+python+library+in+c%2B%2B ? — user253751

Omer Tuchfeld Omer Tuchfeld · Accepted Answer · 2020-10-01T17:35:39

In your case it may be best to keep it simple and not use binding generators, because all you have here is a simple function that doesn't receive any parameters and simply returns an int.

Your goal can be achieved as follows:

In cpp_reader.cpp, prepend the following line before the reader definition:

extern "C" __attribute__ ((visibility ("default")))
int reader()
{
    ....

Now from Python you can simply do:

from ctypes import cdll
import os
lib = cdll.LoadLibrary(os.path.abspath("libclickhouse-cpp-lib.so"))
num_rows = lib.reader()

Also don't forget to add -shared to the g++ commandline when compiling the shared object

Interfacing C++ with Python

1 Answers