Failed to compile bindsnoop eBPF c code outside of kernel tree

0
votes

I'm trying to build a BPF program written in C into the bpf bytecode. The program C code was taken from https://github.com/iovisor/bcc/blob/master/tools/bindsnoop.py (I used C code from bpf_text with some customizations). I didn't want to use bcc python tools, my purpose is to write both kernel and user space code in C, and I would like to build a kernel program outside of the kernel tree.

I use Ubuntu 20.10 and here is uname -a output: Linux bb-VirtualBox 5.8.0-63-generic #71-Ubuntu SMP Tue Jul 13 15:59:12 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

Linux headers are installed, but for some reason I have two directories containing linux headers for 5.8.0-63: /usr/src/linux-headers-5.8.0-63 and /usr/src/linux-headers-5.8.0-63-generic

I used this post to create a makefile for my program: https://blogs.oracle.com/linux/post/bpf-in-depth-building-bpf-programs so the makefile looks like below:

OBJS = kprobe_bindsnoop_kern.o

LLC ?= llc
CLANG ?= clang
INC_FLAGS = -nostdinc -isystem `$(CLANG) -print-file-name=include`
EXTRA_CFLAGS ?= -O2 -emit-llvm

linuxhdrsgen ?= /usr/src/linux-headers-5.8.0-63-generic

LINUXINCLUDE =  -I$(linuxhdrsgen)/arch/x86/include \
                -I$(linuxhdrsgen)/arch/x86/include/generated/uapi \
                -I$(linuxhdrsgen)/arch/x86/include/generated \
                -I$(linuxhdrsgen)/arch/x86/include/uapi \
                -I$(linuxhdrsgen)/arch/x86/include/generated/uapi \
                -I$(linuxhdrsgen)/include/uapi \
                -I$(linuxhdrsgen)/include/generated/uapi \
                -I$(linuxhdrsgen)/include \
                -include ${linuxhdrsgen}/include/linux/kconfig.h

all: $(OBJS)

.PHONY: clean

clean:
    rm -f $(OBJS)

INC_FLAGS = -nostdinc -isystem `$(CLANG) -print-file-name=include`

$(OBJS):  %.o:%.c
    $(CLANG) $(INC_FLAGS) \
        -D__KERNEL__ -D__ASM_SYSREG_H \
        -Wno-unused-value -Wno-pointer-sign \
        -Wno-compare-distinct-pointer-types \
        -Wno-gnu-variable-sized-type-not-at-end \
        -Wno-address-of-packed-member -Wno-tautological-compare \
        -Wno-unknown-warning-option \
        -I../include $(LINUXINCLUDE) \
        $(EXTRA_CFLAGS) -c $< -o -| $(LLC) -march=bpf -filetype=obj -o $@

And kprobe_bindsnoop_kern.c starts from the following set of include files:

#include <uapi/linux/ptrace.h>
#pragma clang diagnostic push
#pragma clang diagnostic ignored "-Wtautological-compare"
#include <net/sock.h>
#pragma clang diagnostic pop
#include <net/inet_sock.h>
#include <net/net_namespace.h>
#include <bcc/proto.h>

Below you can see the compilation output:

clang -nostdinc -isystem `clang -print-file-name=include` \
    -D__KERNEL__ -D__ASM_SYSREG_H \
    -Wno-unused-value -Wno-pointer-sign \
    -Wno-compare-distinct-pointer-types \
    -Wno-gnu-variable-sized-type-not-at-end \
    -Wno-address-of-packed-member -Wno-tautological-compare \
    -Wno-unknown-warning-option \
    -I../include -I/usr/src/linux-headers-5.8.0-63-generic/arch/x86/include/ -I/usr/src/linux-headers-5.8.0-63-generic/arch/x86/include/generated/uapi -I/usr/src/linux-headers-5.8.0-63-generic/arch/x86/include/generated/ -I/usr/src/linux-headers-5.8.0-63-generic/arch/x86/include/uapi -I/usr/src/linux-headers-5.8.0-63-generic/arch/x86/include/generated/uapi -I/usr/src/linux-headers-5.8.0-63-generic/include/uapi -I/usr/src/linux-headers-5.8.0-63-generic/include/generated/uapi -I/usr/src/linux-headers-5.8.0-63-generic/include -include /usr/src/linux-headers-5.8.0-63-generic/include/linux/kconfig.h \
    -O2 -emit-llvm -c kprobe_bindsnoop_kern.c -o -| llc -march=bpf -filetype=obj -o kprobe_bindsnoop_kern.o
In file included from kprobe_bindsnoop_kern.c:22:
In file included from /usr/src/linux-headers-5.8.0-63-generic/include/uapi/linux/ptrace.h:143:
In file included from /usr/src/linux-headers-5.8.0-63-generic/arch/x86/include/asm/ptrace.h:5:
In file included from /usr/src/linux-headers-5.8.0-63-generic/arch/x86/include/asm/segment.h:6:
/usr/src/linux-headers-5.8.0-63-generic/arch/x86/include/asm/alternative.h:59:2: error: unknown type name 's32'
        s32 instr_offset;       /* original instruction */
        ^
/usr/src/linux-headers-5.8.0-63-generic/arch/x86/include/asm/alternative.h:60:2: error: unknown type name 's32'
        s32 repl_offset;        /* offset to replacement instruction */
        ^
/usr/src/linux-headers-5.8.0-63-generic/arch/x86/include/asm/alternative.h:61:2: error: unknown type name 'u16'
        u16 cpuid;              /* cpuid bit set for replacement */
        ^
/usr/src/linux-headers-5.8.0-63-generic/arch/x86/include/asm/alternative.h:62:2: error: unknown type name 'u8'
        u8  instrlen;           /* length of original instruction */
        ^
/usr/src/linux-headers-5.8.0-63-generic/arch/x86/include/asm/alternative.h:63:2: error: unknown type name 'u8'
        u8  replacementlen;     /* length of new instruction */
        ^
/usr/src/linux-headers-5.8.0-63-generic/arch/x86/include/asm/alternative.h:64:2: error: unknown type name 'u8'
        u8  padlen;             /* length of build-time padding */
        ^
/usr/src/linux-headers-5.8.0-63-generic/arch/x86/include/asm/alternative.h:85:8: error: unknown type name 'bool'
extern bool skip_smp_alternatives;
       ^
In file included from kprobe_bindsnoop_kern.c:22:
In file included from /usr/src/linux-headers-5.8.0-63-generic/include/uapi/linux/ptrace.h:143:
In file included from /usr/src/linux-headers-5.8.0-63-generic/arch/x86/include/asm/ptrace.h:5:
In file included from /usr/src/linux-headers-5.8.0-63-generic/arch/x86/include/asm/segment.h:169:
In file included from /usr/src/linux-headers-5.8.0-63-generic/arch/x86/include/asm/cache.h:5:
In file included from /usr/src/linux-headers-5.8.0-63-generic/include/linux/linkage.h:7:
In file included from /usr/src/linux-headers-5.8.0-63-generic/include/linux/export.h:43:
In file included from /usr/src/linux-headers-5.8.0-63-generic/include/linux/compiler.h:266:
/usr/src/linux-headers-5.8.0-63-generic/include/linux/kasan-checks.h:16:15: error: unknown type name 'bool'
static inline bool __kasan_check_read(const volatile void *p, unsigned int size)
              ^
/usr/src/linux-headers-5.8.0-63-generic/include/linux/kasan-checks.h:18:9: error: use of undeclared identifier 'true'
        return true;
               ^
/usr/src/linux-headers-5.8.0-63-generic/include/linux/kasan-checks.h:20:15: error: unknown type name 'bool'
static inline bool __kasan_check_write(const volatile void *p, unsigned int size)
              ^
/usr/src/linux-headers-5.8.0-63-generic/include/linux/kasan-checks.h:22:9: error: use of undeclared identifier 'true'
        return true;
               ^
/usr/src/linux-headers-5.8.0-63-generic/include/linux/kasan-checks.h:34:15: error: unknown type name 'bool'
static inline bool kasan_check_read(const volatile void *p, unsigned int size)
              ^
/usr/src/linux-headers-5.8.0-63-generic/include/linux/kasan-checks.h:36:9: error: use of undeclared identifier 'true'
        return true;
               ^
/usr/src/linux-headers-5.8.0-63-generic/include/linux/kasan-checks.h:38:15: error: unknown type name 'bool'
static inline bool kasan_check_write(const volatile void *p, unsigned int size)
              ^
/usr/src/linux-headers-5.8.0-63-generic/include/linux/kasan-checks.h:40:9: error: use of undeclared identifier 'true'
        return true;
               ^
In file included from kprobe_bindsnoop_kern.c:22:
In file included from /usr/src/linux-headers-5.8.0-63-generic/include/uapi/linux/ptrace.h:143:
In file included from /usr/src/linux-headers-5.8.0-63-generic/arch/x86/include/asm/ptrace.h:5:
In file included from /usr/src/linux-headers-5.8.0-63-generic/arch/x86/include/asm/segment.h:169:
In file included from /usr/src/linux-headers-5.8.0-63-generic/arch/x86/include/asm/cache.h:5:
In file included from /usr/src/linux-headers-5.8.0-63-generic/include/linux/linkage.h:7:
In file included from /usr/src/linux-headers-5.8.0-63-generic/include/linux/export.h:43:
In file included from /usr/src/linux-headers-5.8.0-63-generic/include/linux/compiler.h:267:
/usr/src/linux-headers-5.8.0-63-generic/include/linux/kcsan-checks.h:148:67: error: unknown type name 'size_t'
static inline void __kcsan_check_access(const volatile void *ptr, size_t size,
                                                                  ^
/usr/src/linux-headers-5.8.0-63-generic/include/linux/kcsan-checks.h:164:53: error: unknown type name 'size_t'
kcsan_begin_scoped_access(const volatile void *ptr, size_t size, int type,
                                                    ^
/usr/src/linux-headers-5.8.0-63-generic/include/linux/kcsan-checks.h:184:65: error: unknown type name 'size_t'
static inline void kcsan_check_access(const volatile void *ptr, size_t size,
                                                                ^
In file included from kprobe_bindsnoop_kern.c:22:
In file included from /usr/src/linux-headers-5.8.0-63-generic/include/uapi/linux/ptrace.h:143:
In file included from /usr/src/linux-headers-5.8.0-63-generic/arch/x86/include/asm/ptrace.h:6:
In file included from /usr/src/linux-headers-5.8.0-63-generic/arch/x86/include/asm/page_types.h:7:
In file included from /usr/src/linux-headers-5.8.0-63-generic/include/linux/mem_encrypt.h:17:
In file included from /usr/src/linux-headers-5.8.0-63-generic/arch/x86/include/asm/mem_encrypt.h:15:
/usr/src/linux-headers-5.8.0-63-generic/include/linux/init.h:155:8: error: unknown type name 'bool'
extern bool rodata_enabled;
       ^
fatal error: too many errors emitted, stopping now [-ferror-limit=]
20 errors generated.

I tried to add #include <linux/types.h> or #include <asm/types.h> but it doesn't help. I checked many posts, like this Error compiling eBPF C code out of kernel tree and this how to build BPF program out of the kernel tree but still failed to get answer. What am I missing? Any help will be greatly appreciated ))

1
clinuxebpf
What am I missing? A handfull of typedefs. – wildplasser
These s32 and u8 and so on are likely defined somewhere in some bcc header. You may be able to replace them with __s32, __u8 and the like, that you probably pull from the headers you already include. – Qeole
@wildplasser thanks! Do you mean I need to add manually all typedefs for bool, size_t, u8, u16, etc? I don't think it is a good idea – Sheifa
@Qeole I added the typdefs but it does not help - looks like I need to rewrite all typedefs from linux/types.h and more. It now complains about atomic_t, phys_addr_t, bool, true/false and so on. Seems like a neverending story ... – Sheifa
You are probably missing an include file. Maybe conditionally. If you really cannot find&fix it, the only way is to write one yourself. (which isn't too hard) – wildplasser

1 Answers

1
votes

Thanks to everyone who tried to help me! Finally the issue was sooooo simple - just needed to change an order of the include directories specified in the makefile.

Just moved "-I$(linuxhdrsgen)/include" to be the first in the list and it solved the issue. Meaning, now the list of include dirs looks as follows:

LINUXINCLUDE = -I$(linuxhdrsgen)/include
-I$(linuxhdrsgen)/arch/x86/include
-I$(linuxhdrsgen)/arch/x86/include/generated/uapi
-I$(linuxhdrsgen)/arch/x86/include/generated
-I$(linuxhdrsgen)/arch/x86/include/uapi
-I$(linuxhdrsgen)/arch/x86/include/generated/uapi
-I$(linuxhdrsgen)/include/uapi
-I$(linuxhdrsgen)/include/generated/uapi
-include ${linuxhdrsgen}/include/linux/kconfig.h