how to implement my own system call in Linux kernel 4.x?

Question

i am newbie to kernel and i want to implement my own system call. I have searched so many links and almost reached but still not able to get the exact output.

I have followed own system call on kernel-3.8.8 (youtube video tutorial).

my setup is :
os: ubuntu 14.04 LTS
arch: x86_64
The procedure i have followed is

1. edit the sytemcall table linux-4.7/arch/x86/entry/syscalls_64.tbl
2. add the prototype to the syscalls.h in /usr/src/linux-4.7/include/linux
3. Create a systcall defnition and added the syscall dir into kernel Makefile
4. compile the kernel and reboot
5. write a user application to verify the syscall is working or not.

I could able to get the syscall invokation by syscall() api using syscall num. but what i want is the traditional way like open calls sys_open. i want like mycall need to invoke sys_mycall.

Answer 1

You may edit your glibc to add wrapper around your syscall. Something like it is in the syscalls.list file in glibc/sysdeps/unix (search for your platform) https://github.com/lattera/glibc/blob/master/sysdeps/unix/syscalls.list https://github.com/lattera/glibc/blob/master/sysdeps/unix/sysv/linux/x86_64/syscalls.list

# File name Caller  Syscall name    Args    Strong name Weak names

accept      -   accept      Ci:iBN  __libc_accept   accept
access      -   access      i:si    __access    access
close       -   close       Ci:i    __libc_close    __close close
open        -   open        Ci:siv  __libc_open __open open
read        -   read        Ci:ibn  __libc_read __read read
uname       -   uname       i:p __uname     uname
write       -   write       Ci:ibn  __libc_write    __write write

To decode this format, use "comments in the script which processes this file: sysdeps/unix/make-syscalls.sh.", as it was recommended in https://blog.packagecloud.io/eng/2016/04/05/the-definitive-guide-to-linux-system-calls/

# This script is used to process the syscall data encoded in the various
# syscalls.list files to produce thin assembly syscall wrappers around the
# appropriate OS syscall. See syscall-template.s for more details on the
# actual wrapper.
#
# Syscall Signature Prefixes:
#
# E: errno and return value are not set by the call
# V: errno is not set, but errno or zero (success) is returned from the call
#
# Syscall Signature Key Letters:
#
# a: unchecked address (e.g., 1st arg to mmap)
# b: non-NULL buffer (e.g., 2nd arg to read; return value from mmap)
# B: optionally-NULL buffer (e.g., 4th arg to getsockopt)
# f: buffer of 2 ints (e.g., 4th arg to socketpair)
# F: 3rd arg to fcntl
# i: scalar (any signedness & size: int, long, long long, enum, whatever)
# I: 3rd arg to ioctl
# n: scalar buffer length (e.g., 3rd arg to read)
# N: pointer to value/return scalar buffer length (e.g., 6th arg to recvfrom)
# p: non-NULL pointer to typed object (e.g., any non-void* arg)
# P: optionally-NULL pointer to typed object (e.g., 2nd argument to gettimeofday)
# s: non-NULL string (e.g., 1st arg to open)
# S: optionally-NULL string (e.g., 1st arg to acct)
# v: vararg scalar (e.g., optional 3rd arg to open)
# V: byte-per-page vector (3rd arg to mincore)
# W: wait status, optionally-NULL pointer to int (e.g., 2nd arg of wait4)

More information about glibc's syscall wrapper at official site: https://sourceware.org/glibc/wiki/SyscallWrappers

There are three types of OS kernel system call wrappers that are used by glibc: assembly, macro, and bespoke.

Assembly syscalls Simple kernel system calls in glibc are translated from a list of names into an assembly wrapper that is then compiled. ... The list of syscalls that use wrappers is kept in the syscalls.list files: ... ./sysdeps/unix/sysv/linux/x86_64/syscalls.list

Don't forget to define __NR number in linux headers for your syscall

There are instructions from kernel.org, the only linux kernel developer portal, or in Documentation/adding-syscalls.* files inside linux kernel sources: https://www.kernel.org/doc/html/v4.10/process/adding-syscalls.html https://github.com/torvalds/linux/blob/master/Documentation/process/adding-syscalls.rst

The method will be different for other OS like FreeBSD: https://wiki.freebsd.org/AddingSyscalls