I'm getting different results for the same shared library when a function (from the shared library) is executed by a standalone executable and when using Python's ctypes for calling the exact same function.
The difference was narrowed down to be the version of libm used by the library.
When using the library from a standalone executable we link against openlibm, but when the library is linked by ctypes, it's linked against Ubuntu's libm.
How can I make ctypes link my shared library against openlibm?
The shared library header (fft.h):
extern "C" {
void fft(const char* srcPath);
};
The shared library implementation (fft.cpp):
#include <iostream>
#include <fstream>
#include <nlohmann/json.hpp>
#include <Eigen/Eigen>
#include <unsupported/Eigen/FFT>
void fft(const char* srcPath) {
std::cout.precision(17);
std::ifstream fs(srcPath, std::ofstream::in);
// Get length of file
fs.seekg(0, std::ifstream::end);
auto length = fs.tellg();
fs.seekg(0, std::ifstream::beg);
// Read the file
auto srcRaw = new char[length];
fs.read(srcRaw, length);
// Convert data to vector of doubles
nlohmann::json j = nlohmann::json::parse(srcRaw);
std::vector<double> srcDoubles = j;
int n = 65536;
Eigen::VectorXd srcEigen(n);
Eigen::VectorXcd dstEigen(n);
std::copy(srcDoubles.data(), srcDoubles.data() + srcDoubles.size(), srcEigen.data());
Eigen::FFT<double> fft;
fft.fwd(dstEigen, srcEigen);
std::vector<std::complex<double>> dstTmp(dstEigen.data(), dstEigen.data() + dstEigen.size());
for (size_t i = 0; i < dstTmp.size(); i++) {
std::cout << "Result[" << i << "] = " << dstTmp[i].real() << "+i*" << dstTmp[i].imag() << std::endl;
}
delete[] srcRaw;
}
Using the library from a standalone executable:
#include <fft.h>
int main(int argc, char** argv) {
fft("input_data");
}
Using Python's ctypes:
from ctypes import c_char_p, CDLL
class LibCFft(CDLL):
def __init__(self):
super(LibCFft, self).__init__("libexample.so")
self._addfunc('fft', [c_char_p], None)
def _addfunc(self, name, argtypes, restype):
attr = getattr(self, name)
attr.argtypes = argtypes
attr.restype = restype
return lambda x: x
class CFft(object):
def fft(self):
LibCFft().fft("input_data")
if __name__ == '__main__':
CFft().fft()
Lib build:
clang++ /opt/eigen-eigen-43d9075b23ef -isystem /opt/spdlog/include -isystem /opt/nlohmann/include -O3 -DNDEBUG -fPIC -std=gnu++14 -o CMakeFiles/example.dir/fft.cpp.o -c /home/user/fft.cpp
clang++ -fPIC -O3 -DNDEBUG -shared -Wl,-soname,libexample.so -o libbugatone.so CMakeFiles/example.dir/generated/fft.cpp.o -lopenlibm
Executable build:
clang++ -O3 -DNDEBUG -latomic -nodefaultlibs -lopenlibm -lc -lc++ -lgcc -lgcc_s -lstdc++ -latomic -lm CMakeFiles/eigen_tester.dir/eigen_tester.cpp.o -o eigen_tester libexample.so -lopenlibm
Thanks in advance for the help.
expfunction is looked for by its name. Both libs have their own version ofexpso any call toexpcan be redirected to either one of them. When building my lib without-lopenlibmI get the same results (e.g.expfunction results) as when I use my lib with ctypes. When building my lib with-lopenlibmI get the expected results when using my own executable but I get the wrong results when using ctypes to load this lib. - Karpadldd example.soAnyway, ctypes doesn't have anything to do with linking, it's dynamically loading. But, again without a mcve ([SO]: How to create a Minimal, Complete, and Verifiable example (mcve)) it's hard to tell - CristiFatilddoutput is what I expect (openlibm appears there before libm). ctypes is not linking, but I guess Python binary is loaded with libm so math function calls are done using libm and not openlibm (i.e. the priority between libraries changes). - Karpaddlopen/dlsym) in your C++ code. But I don't know how scalable is that. - CristiFati