How to call function (shm_open) in librt inside the glibc source code (malloc/morecore.c)?

Question

I am trying to modify the __default_morecore function in malloc/morecore.c. The original __default_morecore is a simple wrapper for sbrk, but I want to use shm_open function inside __default_morecore to create shared memory object. Here is the malloc/morecore.c code I modified:

...
/* Include header files for shm_open */

void *
__default_morecore (ptrdiff_t increment)
{
  int shm_fd;
  /* Create the shared memory object */
  shm_fd = shm_open(shm_name, O_CREAT | O_RDWR, 0644);
  if (shm_fd < 0){
    return NULL;
  }
...
}

I am pretty sure that my codes are right since I tested the code snippet in a small program and it works. After making this change, it will show below error message when using make for glibc 2.32 sources:

.../build/libc_pic.os: In function '__GI___default_morecore':
.../malloc/morecore.c:69: undefined reference to `shm_open'
collect2: error: ld returned 1 exit status
../Makerules:698: recipe for targe '.../build/libc.so' failed

I searched online for this error, and it's typically becasue doesn't link with -lrt. I tried add -lrt in 'LDFLAGS' and 'LIBS' during ../configure, but not working. Then I added -lrt at the end of build-shlib of Makerules (which is used to build libc.so), and the error was changed to be following:

//lib/x86_64-linux-gnu/librt.so.1: undefined reference to `__clock_getcpuclockid@GLIBC_PRIVATE'

//lib/x86_64-linux-gnu/librt.so.1: undefined reference to `__clock_nanosleep@GLIBC_PRIVATE'

//lib/x86_64-linux-gnu/librt.so.1: undefined reference to `__clock_settime@GLIBC_PRIVATE'

//lib/x86_64-linux-gnu/librt.so.1: undefined reference to `__clock_getres@GLIBC_PRIVATE'

//lib/x86_64-linux-gnu/libpthread.so.0: undefined reference to `__libc_vfork@GLIBC_PRIVATE'
collect2: error: ld returned 1 exit status ../Rules:215: recipe for target '/home/yifei/FSL_Repos/test_shm_mmap_malloc/glibc-malloc-modified/build/iconv/iconvconfig' failed

The new error shows that multiple symbols from librt.so and libpthread.so are undefined. I tried add -lpthread -lrt to many places in glibc Makefile/Makeconfig/Makerules, but it doesn't work. Also, from the above errors, I found the linking libraries are from my system in-built libraries, not the path of current modified glibc which I am building. Is that an expected behavior? I think it should use the librt.so and libpthread.so in my modified glibc I am building, but I cannot find librt.so, librt.a and libpthread.so in my build folder after the make terminates. Does it mean these are not generated at the stage of the error I occur? If yes, it might be a dependency issue considering I called a glibc function inside glibc. How to resolve that issue?

How can I call shm_open in malloc/morecore.c and link that librt without any error? Thanks!

Out of curiosity, why would you do that? Why do you need a shared memory object from a global shared implementation of standard library from morecore? Wouldn't it be easier to just overwrite __morecore hook for programs that need it rather then recompiling whole glibc? undefined reference to because symbols are resolved in order, would specifying libraries twice -lpthread -lrt -lpthread -lrt help? — KamilCuk
Thanks for your comment. Could I specify the modified implementation by __morecore hook outside the glibc without needs to recompile? If so, that would be more convenient and effective. I tried to use -lpthread -lrt -lpthread -lrt, but it doesn't seem to work, and it will still have the same error messages. Can I explicitly specify the shared library path that I would like to link? LD_LIBRARY_PATH or the -L option of gcc? — Jeffrey
@KamilCuk That is not going to work. We want the __morecore to be overridden at the very beginning, before the first malloc() is called. But this can't be achieved if a C++ program has one or more static STL containers (Their constructors may be called first). — Su Excelle

Florian Weimer Florian Weimer · Accepted Answer · 2020-09-13T18:33:16

shm_open is just a thin wrapper around the open function, see sysdeps/posix/shm_open.c. What is problematic is the construction of the file name in the SHM_GET_NAME macro (in sysdeps/posix/shm-directory.h). It calls __shm_directory (from sysdeps/unix/sysv/linux/shm-directory.c), and the current implementation potentially calls malloc under the covers.

I suggest hard-wiring the shared memory segment location to a file in /dev/shm, and call open directly (or rather __open64, otherwise the linknamespace tests will fail during make check).

The __morecore hook is not a complete solution because it only covers part of the main allocation arena used by glibc malloc. If you want to experiment with malloc, it may be easier to start out of with a simple out-of-tree malloc implementation and use the symbol interposition mechanism. But you still need to be careful which glibc functions you call in your interposing malloc.

How to call function (shm_open) in librt inside the glibc source code (malloc/morecore.c)?

1 Answers