how to use GDB to debug loader/linker issue

Question

I'm debugging a particularly weird issue that occurs before my program starts, ie. this happens at load time before code starts executing at the "_start" symbol. And yes I am modifying the ELF manually. There are no indications of a bad ELF format tho, and I'm using libelf to make the modifications and have been very successful until now.

With GDB, I can see the .plt.got, .data (rw data) and .bss sections in memory and put a break point at the "_start" address (or entry point address returned by readelf. The .plt.got and .data all look good before I run the program. Then I run the program and my .data section along with the last entry in the .plt.got are getting wiped out by zeros.

Initially I thought it was the .bss initialization going to the wrong address, but the .bss data (a couple global vars) is loaded correctly. Then I've also observed that if I change the size of .data, the block of addresses that it getting inited grows as well - its always 16bytes larger than the size of my .data section, and it does not grow if I change the .bss section size.

How can I debug this? GDB won't let me intercept or add break points to libraries loaded at runtime (or I don't know how to) and I'm sure its probably some data gone wrong that the loader/linker is using as a basis for initializing some memory.

I'm also looking for some pointers to default std gnuc stuff that gets run at load/link time and performs some initialization and what processes in that block perform initialization of data in the program's pages, especially anything that would key off the size of the .data section.

Here is some relevant data:

$ ld --version
GNU ld version 2.26.20160125

gcc --version
gcc (GCC) 6.3.1 20161221 (Red Hat 6.3.1-1)


Relocation section '.rela.plt' at offset 0x870 contains 24 entries:
  Offset          Info           Type           Sym. Value    Sym. Name + Addend
000000605f98  000100000007 R_X86_64_JUMP_SLO 0000000000000000 getenv@GLIBC_2.2.5 + 0
000000605fa0  000200000007 R_X86_64_JUMP_SLO 0000000000000000 free@GLIBC_2.2.5 + 0
000000605fa8  000300000007 R_X86_64_JUMP_SLO 0000000000000000 putchar@GLIBC_2.2.5 + 0
000000605fb0  000400000007 R_X86_64_JUMP_SLO 0000000000000000 __errno_location@GLIBC_2.2.5 + 0
000000605fb8  000500000007 R_X86_64_JUMP_SLO 0000000000000000 strncmp@GLIBC_2.2.5 + 0
000000605fc0  000600000007 R_X86_64_JUMP_SLO 0000000000000000 puts@GLIBC_2.2.5 + 0
000000605fc8  000700000007 R_X86_64_JUMP_SLO 0000000000000000 readlink@GLIBC_2.2.5 + 0
000000605fd0  000800000007 R_X86_64_JUMP_SLO 0000000000000000 __mempcpy@GLIBC_2.2.5 + 0
000000605fd8  000900000007 R_X86_64_JUMP_SLO 0000000000000000 textdomain@GLIBC_2.2.5 + 0
000000605fe0  000a00000007 R_X86_64_JUMP_SLO 0000000000000000 pathconf@GLIBC_2.2.5 + 0
000000605fe8  000b00000007 R_X86_64_JUMP_SLO 0000000000000000 dcgettext@GLIBC_2.2.5 + 0
000000605ff0  000c00000007 R_X86_64_JUMP_SLO 0000000000000000 printf@GLIBC_2.2.5 + 0
000000605ff8  000d00000007 R_X86_64_JUMP_SLO 0000000000000000 __libc_start_main@GLIBC_2.2.5 + 0
000000606000  000e00000007 R_X86_64_JUMP_SLO 0000000000000000 strcmp@GLIBC_2.2.5 + 0
000000606008  000f00000007 R_X86_64_JUMP_SLO 0000000000000000 __dcgettext@GLIBC_2.2.5 + 0
000000606010  001000000007 R_X86_64_JUMP_SLO 0000000000000000 fprintf@GLIBC_2.2.5 + 0
000000606018  001200000007 R_X86_64_JUMP_SLO 0000000000000000 memcpy@GLIBC_2.14 + 0
000000606020  001300000007 R_X86_64_JUMP_SLO 0000000000000000 malloc@GLIBC_2.2.5 + 0
000000606028  001400000007 R_X86_64_JUMP_SLO 0000000000000000 confstr@GLIBC_2.2.5 + 0
000000606030  001500000007 R_X86_64_JUMP_SLO 0000000000000000 setlocale@GLIBC_2.2.5 + 0
000000606038  001600000007 R_X86_64_JUMP_SLO 0000000000000000 error@GLIBC_2.2.5 + 0
000000606040  001700000007 R_X86_64_JUMP_SLO 0000000000000000 sysconf@GLIBC_2.2.5 + 0
000000606048  001800000007 R_X86_64_JUMP_SLO 0000000000000000 exit@GLIBC_2.2.5 + 0
000000606050  001900000007 R_X86_64_JUMP_SLO 0000000000000000 execv@GLIBC_2.2.5 + 0

The initialization starts at 0x606050 (wiping out the last entry in the .got.plt which is the execv@GLIBC_2.2.5)

And the relevant sections:

  [25] .got.plt          PROGBITS         0000000000605f80  00005f80
       00000000000000d8  0000000000000008  WA       0     0     8
  [26] .data             PROGBITS         0000000000606058  00006058
       0000000000000040  0000000000000000  WA       0     0     1
  [27] .bss              NOBITS           00000000006060e0  00006098
       0000000000000030  0000000000000000  WA       0     0     32

Note that however large I make the .data section, I'm seeing a block starting at 0x606050 16bytes larger than .data section getting set to zeros - which BTW overwrites all my rw data.

Answer 1

You can invoke ld.so directly: see man 8 ld.so. That will allow you to debug the dynamic linker's behavior through gdb. You can possibly get debug symbols and source for your distribution's ld.so through its package manager. Otherwise, you may be interested in building it from source for this purpose. (It is very unusual to have to debug the dynamic linker.)

You could be interested in setting a watchpoint to catch the place that zeroes your PLT entries.

Answer 2

Found this link useful on how to debug the loader: https://sourceware.org/glibc/wiki/Debugging/Loader_Debugging

Turns out my issue was that I did not adjust my rw LOAD segment for the new section sizes. So what confused me?

1. The debugger, ddd, shows the program data as if its loaded in memory before the program is run. At this point nothing has been loaded into memory. The rw .data section looked like it was in memory and correct when in-fact it was not.

2. Once the program was run and before it even started executing at the entry point (_start), the rw .data section appeared to be zero'ed out, or initialized to zeros because the rw .data section was never actually loaded into memory.

ddd is miss-leading since it highlights this memory to indicate it was changed, when in-fact it was always zero.

The debugger, ddd, also must be ignoring LOAD segment sizes when it displays the program data before it is actually loaded into memory.