Cross-Compiling ROS Melodic on RaspberryPi 3B+ Problem

Question

After doing some research, I found that there isn't a single "tried-and-true" way of cross-compiling ROS for RaspberryPi. The best I could come up with was downloading /usr, /lib, and /opt from the target device onto my development machine (installed at $HOME/Projects/TargetResources), making a CMake Toolchain file, and calling catkin_make_isolated -DCMAKE_TOOLCHAIN_FILE=.... Though the build process starts, I'm met with an immediate error:

$ catkin_make_isolated -DCMAKE_TOOLCHAIN_FILE=${HOME}/Projects/${MY_PROJ}/RPI3+_Melodic_Toolchain.cmake
CMake Error at ${HOME}/Projects/TargetResources/opt/ros/melodic/share/catkin/cmake/assert.cmake:17 (message):
Assertion failed: check for file existence, but filename
  (RT_LIBRARY-NOTFOUND) unset.  Message: RT Library

Call Stack (most recent call first):

  ${HOME}/Projects/TargetResources/opt/ros/melodic/share/catkin/cmake/tools/rt.cmake:42 (assert_file_exists)
  ${HOME}/Projects/TargetResources/opt/ros/melodic/share/catkin/cmake/all.cmake:159 (include)
  ${HOME}/Projects/TargetResources/opt/ros/melodic/share/catkin/cmake/catkinConfig.cmake:20 (include)
  CMakeLists.txt:10 (find_package)

After doing some digging, the target device (i.e., RaspberryPi 3+) does not have librt.so. Interestingly enough, if I just run catkin_make for the project on the device, everything builds successfully. So I don't think attempting to install it is really necessary (or the right solution).

Additionally, I should note that the target has ROS Melodic Base installed whereas my development machine has ROS Melodic Desktop installed. I'm not sure if this would cause the issue I'm experiencing, but I don't want to prematurely rule it out.

So, my question is how should I proceed? Did I overlook something whilst setting up the toolchain or am I assuming something about ROS/Catkin that is false?

Thank you in advance for all of your help and consideration.

Edit / Addendum

As requested, here is the toolchain file:

set(CMAKE_SYSTEM_NAME Linux)
set(CMAKE_SYSTEM_VERSION 1)

SET(CMAKE_C_COMPILER   $ENV{HOME}/Projects/RpiDevTools/arm-bcm2708/arm-rpi-4.9.3-linux-gnueabihf/bin/arm-linux-gnueabihf-gcc)
SET(CMAKE_CXX_COMPILER $ENV{HOME}/Projects/RpiDevTools/arm-bcm2708/arm-rpi-4.9.3-linux-gnueabihf/bin/arm-linux-gnueabihf-g++)

SET(CMAKE_FIND_ROOT_PATH $ENV{HOME}/Projects/RpiDevTools/arm-bcm2708/arm-rpi-4.9.3-linux-gnueabihf/arm-linux-gnueabihf/sysroot/)

set(CMAKE_FIND_ROOT_PATH_MODE_PROGRAM NEVER)
set(CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ONLY)
set(CMAKE_FIND_ROOT_PATH_MODE_INCLUDE ONLY)

In sysroot, I copied the RaspberryPi's /opt directory as that's where ROS Melodic libraries are installed. I've found that those libraries may have dependencies (for example log4cxx/level.h) which are not found in the RaspberryPi/Tools sysroot/usr/include directory. Likewise, resolving the shared object dependencies are equally cumbersome.

Don't you have /usr/lib/arm-linux-gnueabihf/librt.so ? It comes from the package libc6-dev on my RPi 3 (dpkg -S /usr/lib/arm-linux-gnueabihf/librt.so) — Victor Lamoine
Thank you for the full path! find ${HOME}/Projects/TargetResources/lib -f librt.so returned nothing, but when I navigated down into that directory, I found that all I have is a soft link of librt.so.1 -> librt-2.24.so. I might just need to add the additional soft-link. — jhill515
It should works as this is what I have: pastebin.com/raw/bxaeZjjQ — Victor Lamoine
I did some digging, and failed to notice the extra /usr/lib/arm-linux-gnueabihf/librt.so argument to your listing. Yes, I have what you have except with v2.24 (see here). The soft-link work around isn't working for me. I'm also trying to look into the cross-compile tools. If it's supposed to be part of libc6, there's a chance that the RaspberryPi Development Tools library I grabbed from git-hub isn't referencing its own libraries? — jhill515
Without seeing the RPI3+_Melodic_Toolchain.cmake file it's difficult to say. Add it in your question — Victor Lamoine

jhill515 jhill515 · Accepted Answer · 2019-03-18T18:51:03

It appears that the issue stems from the cross-compilation tools itself: using arm-linux-gnueabihf-gcc & arm-linux-gnueabihf-g++ from https://github.com/raspberrypi/tools means that the compilers are configured with libc libraries in a pre-known configuration. For example, math.h includes bits/math-vector.h; in the RaspberryPi Tools from GitHub, this is located at .../sysroot/usr/include/bits whereas copied straight from the RaspberryPi 3B+ itself is located at /usr/include/arm-linux-gnueabf/bits. The respective compilers (native and cross-compile) were built with those their respective configurations, but not the others and does not search (well) the generalized paths.

Cross-Compiling ROS Melodic on RaspberryPi 3B+ Problem

1 Answers