Unable to add a custom hello system call on x64 ubuntu linux

Question

I am trying to add hello system call and then compile the kernel but at the end (after spending ~2.5 hrs) it gives the following error:

d: arch/x86/entry/syscall_64.o:(.rodata+0x1120): undefined reference to __x64___x64_sys_hello' make: *** [Makefile:1166: vmlinux] Error 1

I tried to change the 548 64 printmsg sys_hello to 548 64 printmsg __x64_sys_printmsg in syscall_64.tbl file as well but still no success. [Current kernel 5.8.0-48-generic -> new 5.10.26].

here is the code for hello.c:

#include <linux/kernel.h>

asmlinkage long sys_hello(void){ // I already tried renaming it __x64_sys_hello as well

    printk("Hello World\n");
    return 0;

}

Is there a way I can fix this issue?

Try changing asmlinkage long sys_hello(void) to SYSCALL_DEFINE0(hello). You also need #include <linux/syscalls.h>. — Ian Abbott

Abhiram Padmanabhan Abhiram Padmanabhan · Accepted Answer · 2021-07-25T15:12:10

I have been working with Kernel 5.4.0-77 as listed below, the issue you are having is because there is a reference error, and the compiler is unable to find the code for the particular definition. Do the steps as mentioned below and try to see if they solve your problem.

jack@blr:~$ uname -a
Linux blr 5.4.0-77-generic #86-Ubuntu SMP Thu Jun 17 02:35:03 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

Add a folder printmsg in your main source code folder, example :

jack@blr:/working/focal/printmsg$ ls
Makefile  printmsg.c

Create the two files listed, as shown below :

Printmsg.c

#include <linux/kernel.h>
#include <linux/syscalls.h>
SYSCALL_DEFINE0(printmsg)
{
        printk("printmsg SYSTEM CALL IS ALIVE\n");
        return 0;
}

Makefile

obj-y := printmsg.o

Now we have to add the system call references to our syscalls.h and syscall_64.tbl files as shown below respectively.

root@blr:/working/focal# vi include/linux/syscalls.h
asmlinkage long sys_printmsg(void); - add this before the last #endif 
root@blr:/working/focal# vi arch/x86/entry/syscalls/syscall_64.tbl
548     common  printmsg                 __x64_sys_printmsg

– Make sure you use tabs here instead of space and follow whatever format is followed in the file as is and align the new text with the rest of the code.

Return to the main source folder, you should have a Makefile there – edit the following line in that file, adding the folder where our code exists, so that it can be compiled during the kernel build.

jack@blr:/working/focal$ cat Makefile |grep core-y
core-y          += kernel/ certs/ mm/ fs/ ipc/ security/ crypto/ block/ printmsg/

Once the code is compiled you can check if the code is working by writing a simple userspcae program as shown below:

jack@blr:~/kerndev/asg3$ cat printmsg.c
#include <stdio.h>
#include <linux/kernel.h>
#include <sys/syscall.h>
#include <linux/unistd.h>
#include <unistd.h>
#include <errno.h>
int main()
{
         long  amma = syscall(548);
         int errnum =errno;
         if(amma==-1)
         { perror("Error is :" );
         printf("System call sys_jackall returned %ld and error is %d\n",amma,errnum);
         }
         else
         printf("System call PRINTMSG was called sussefully\n");
         return 0;
}

Compiling and running the program:

jack@blr:~/kerndev/asg3$ gcc printmsg.c
jack@blr:~/kerndev/asg3$ ./a.out
System call PRINTMSG was called sussefully

Unable to add a custom hello system call on x64 ubuntu linux

1 Answers