Backtrace points to fopen crash

Question

Following is the backtrace I got from a recent crash. It points to the fopen call. Not sure whats going on here. There is enough space on the device. & Even if the file is not their , it should return NULL.

Crashing does not make sense. here is the bt

(gdb) bt 
#0  0xb788f50e in __open_nocancel () from /tmp/user_20130523143934/x86/lib/tls/libc.so.6
#1  0xb78453e7 in *__GI__IO_file_open (fp=0x814b0e4, filename=0xb78fc1f5 "/init_log.cfg", posix_mode=-2, prot=438, read_write=8, is32not64=0) at fileops.c:233
#2  0xb784555c in _IO_new_file_fopen (fp=0x814b0e4, filename=0xb78fc1f5 "/init_log.cfg", mode=0x1b6 <Address 0x1b6 out of bounds>, is32not64=1) at fileops.c:332
#3  0xb783d519 in __fopen_internal (filename=0xb78fc1f5 "/init_log.cfg", mode=0xb78fc1f3 "r", is32=1) at iofopen.c:93
#4  0xb783d55f in _IO_new_fopen (filename=0xb78fc1f5 "/init_log.cfg", mode=0xb78fc1f3 "r") at iofopen.c:107
#5  0xb78fbe00 in init_setup (app_uuid=1009, mod_uuid=0, hdl=0xfffffffe) at ../init_setup_api.c:1013
#6  0x0805f431 in main (argc=1, argv=0xbffffa64) at .. vpg_agent.c:307


(gdb) frame 5
#5  0xb78fbe00 in init_setup (app_uuid=1009, mod_uuid=0, hdl=0xfffffffe) at ../init_setup_api.c:1013
1013        if ((fp = fopen(MY_FILE, "r")) == NULL) {
(gdb) l

Appreciate your help.

PS: One more thing, its a remotely mapped filesystem. Just got to know about it. That might be contributing.

Most probable a memory corruption (maybe just before the fopen or even simultaneously if it's multi-threaded application). Or maybe your program is out of virtual memory. — nullptr
another process crashed at the same time, that also pointed to fopen — vindyz
its a static macro , points to a file location. Thet shouldnt be problem rt, if its not thr it should return NULL, not crash — vindyz
gdb tells that you are using libc that is not in system's usual path (/tmp/user_20130523...). Maybe the problem is there, you're using non-standard libc (maybe it's tracking files or something like this) and it crashes? — nullptr
yes this is non-standard libc, but this has been inplace for years. We just have wrappers around actual function calls. these wrappers are not crashing — vindyz

Employed Russian Employed Russian · Accepted Answer · 2013-05-24T02:23:26

The __open_nocancel is usually a tiny wrapper around the system call.

Here is what it looks like on my system:

(gdb) disas __open_nocancel
Dump of assembler code for function __open_nocancel:
   0x000ddf5a <+0>:     push   %ebx
   0x000ddf5b <+1>:     mov    0x10(%esp),%edx
   0x000ddf5f <+5>:     mov    0xc(%esp),%ecx
   0x000ddf63 <+9>:     mov    0x8(%esp),%ebx
   0x000ddf67 <+13>:    mov    $0x5,%eax
   0x000ddf6c <+18>:    call   *%gs:0x10
   0x000ddf73 <+25>:    pop    %ebx
   0x000ddf74 <+26>:    cmp    $0xfffff001,%eax
   0x000ddf79 <+31>:    jae    0xddfad <open+93>
   0x000ddf7b <+33>:    ret    
End of assembler dump.

It is very likely that your PC 0xb788f50e corresponds to my instruction at 0x000ddf73, i.e. instruction immediately following the system call itself.

If that's the case, there are several likely explanations:

You are looking at the wrong thread, not the one that actually crashed.
What does (gdb) info thread show?
Your kernel is unhappy with something happening on the system, and is terminating processes left and right. Check /var/log/messages for clues.

On the other hand, if your instruction at 0xb788f50e corresponds to my instruction at 0x000ddf6c (i.e. the call itself), then perhaps you've set your ulimit -s too low?

Finally, if you have SELinux enabled, it may be terminating you because you are violating its policies. Check appopriate selinux.log.

Backtrace points to fopen crash

1 Answers