What is the purpose of using trap table

Question

I have a question when reading OSTEP. The question is how does the trap know which code to run inside the OS? I understand that is a terrible idea, cause it means user can do anything in the kernel. But I'm not clear the example given in the book "Imagine jumping into code to access a file, but just after permission check" Can anyone explain this? And also what will happen if a program can directly issue system call without using trap table? Thanks!

mevets mevets · Accepted Answer · 2020-09-16T03:14:03

A trap table is what is conventionally used by the system call handler to invoke the requested operating service routine. It is a table that is protected by the kernel, so does not enable execution at an arbitrary address. Rather it maps a requested service (typically a small number, search /usr/include for syscall.h) to a function that provides that service.

The application performs some operation like:

_open:
mov   $_SYS_OPEN, %r0
syscall
jmp-on-carry seterrno
ret

The bit of code in the kernel that gets invoked on the syscall operator does something like:

int n;
if ((n = Proc->reg[0]) < 0 || n >= _N_SYSCALLS) {
    return seterror(Proc, ENOSYS);
} else {
    return (*sys_call_table[n])(Proc);
}

The details vary by which os you are working with, but the overall idea is the same, and has remained relatively constant since the 1970s.

What is the purpose of using trap table

1 Answers