First off, let me apologize if I'm writing this in the wrong place. I can't seem to find a valgrind users forum in which to post things of this nature, and since this place seems fairly all encompassing, I figured I'd try it out.
I have a somewhat unorthodox linux kernel I need to utilize when running my code. I don't know too much about the kernel, specifics since I didn't write it. My kernel, for whatever reason, looks like it extends the default linux kernel's syscall table. What appears to happen is that upon insmod of a .ko file, the current, standard syscall table is extended, starting at what appears to be 333, and going to 341, with the original syscall table being saved, and restored when the .ko is rmmoded. These extra syscalls seem to perform some sort of unique IPC communication with the proprietary software I am running.
Of course, this seems to play hell with valgrind.
I'd like to use valgrind to memcheck my programs of course, but valgrind always crashes the instant my application performs a syscall. Here is the output I get from valgrind, minus a few things I'd rather
==26045== Thread 2:
==26045== Syscall param preadv(vector) points to unaddressable byte(s)
==26045== at 0x4000982: ??? (in /lib/ld-2.9.so)
==26045== by 0x426C756: syscall (in /lib/libc-2.9.so)
==26045== by 0x4037430: com_lock (coms.c:114)
--23932-- VALGRIND INTERNAL ERROR: Valgrind received a signal 11 (SIGSEGV) - exiting
--23932-- si_code=1; Faulting address: 0x165; sp: 0x63a6dde4
valgrind: the 'impossible' happened:
Killed by fatal signal
==23932== at 0x3809D5E0: vgSysWrap_linux_sys_preadv_before (syswrap-linux.c:3413)
==23932== by 0x380785CB: vgPlain_client_syscall (syswrap-main.c:1382)
==23932== by 0x38076330: vgPlain_scheduler (scheduler.c:929)
==23932== by 0x380A13E8: run_a_thread_NORETURN (syswrap-linux.c:98)
==23932== by 0x380A1732: vgModuleLocal_start_thread_NORETURN (syswrap-linux.c:268)
==23932== by 0x380A8AC8: ??? (in /usr/local/lib/valgrind/memcheck-x86-linux)
Unfortunately it is impossible for me to not use these extra syscalls. The challenge I'm facing is how to deal with them.
What I've learned so far is that I can specify in valgrind new and or customized syscall wrappers. Unfortunately I don't quite know how to pull that off when syscall addresses are getting all switched around during my runtime.
If anyone has any sort of experience writing valgrind syscall wrappers for syscalls they've added to their kernel that are non standard (and know how to tell valgrind to switch wrappers) please reply back with some details, references, anything. I'd really appreciate it.
I am using valgrind 3.7, with g++ 4.1.1 and a custom kernel built off 2.6.29 gentoo (for all intents and purposes, it's 2.6.29 gentoo until my proprietary kernel module is loaded)
Edit:
I have discovered a few more things relating to this since I first posted. It appears that valgrind has a file called include/vki/vki-scnums-x86-linux.h. According to the comments in it (valgrind 3.7.0 here remember), it's more or less a cut and paste of asm-i386/unistd.h from a 2.6.9 kernel. At instruction enumeration 333, __NR_preadv is defined. For me, valgrind dies when it performs a syscall for something called my_ipc, which is a syscall unique to my kernel, enumerated at none other than 333. It appears that there is a syscall mapping error between what valgrind was compiled against, and what is actually happening in the kernel. As such, the syscall wrapper valgrind tries to call when it sees syscall 333 can't really handle syscall 333 since the function call signature probably doesn't match up.
I think preadv looks like...
preadv(int fd, const struct iovec *iov, int iovcnt, off_t offset);
and from what I can tell, my ipc call looks like
asmlinkage long my_ipc (uint, long, long, long, long, long);
