Priority of LIBRARY_PATH with custom gcc installations

Question

Is there a way to get compilers to prefer libraries from LIBRARY_PATH instead of system paths. I am particularly looking for Clang. I partially solved the problem for GCC while writing this question, but it's also not quite clear.

Background

LIBRARY_PATH is a convenient environment variable to allow transparently linking libraries in non-standard directories, e.g. user installations, and in my case environment modules that provide different versions of a library. The idea is to do a module load libfoo/version and the compiler will transparently use the right libfoo.so.

For a shared library, one also needs to set LD_LIBRARY_PATH for ld.so to find the right library. If there are multiple libfoo.so in both LD_LIBRARY_PATH and /usr/lib, ld.so specifies that LD_LIBRARY_PATH is searched before default paths.

Problem

I have encountered issues when the library defines a soname - which is different between the two libfoo.so versions (which are symlinks to libfoo.so.1 and libfoo.so.2 respectively) in /usr/lib and LIBRARY_PATH. Then ld will link against the soname in /usr/lib and LD_LIBRARY_PATH can no longer prioritize the intended library.

I first encountered this with boost, but here's a small example:

echo "void foo() {}" > foo.c

# create an old libfoo version in /usr/lib
sudo gcc foo.c -fpic -shared -o /usr/lib/libfoo.so.1 -Wl,-soname,libfoo.so.1
sudo ln -s libfoo.so.1 /usr/lib/libfoo.so

# create the new libfoo that we want to transparently override
mkdir -p /tmp/XXX/lib
gcc foo.c -fpic -shared -o /tmp/XXX/lib/libfoo.so.2 -Wl,-soname,libfoo.so.2
ln -s libfoo.so.2 /tmp/XXX/lib/libfoo.so

export LIBRARY_PATH=/tmp/XXX/lib
export LD_LIBRARY_PATH=/tmp/XXX/lib

echo "void foo(); int main() { foo(); }" > main.c
gcc main.c -lfoo
ldd a.out| grep foo
    # under some conditions this shows libfoo.so.1 instead of .2
    libfoo.so.1 => /usr/lib/libfoo.so.1

GCC

I initially encountered this issue with a custom installation of GCC while it was working as expected for the system installation. gcc --print-search-dirs reveals a pattern:

/tmp/XXX/lib/x86_64-pc-linux-gnu/7.2.0/
/tmp/XXX/lib/x86_64-linux-gnu/
/tmp/XXX/lib/../lib64/
/opt/gcc/7.2.0/lib/gcc/x86_64-pc-linux-gnu/7.2.0/
/opt/gcc/7.2.0/lib/gcc/x86_64-pc-linux-gnu/7.2.0/../../../../x86_64-pc-linux-gnu/lib/x86_64-pc-linux-gnu/7.2.0/
/opt/gcc/7.2.0/lib/gcc/x86_64-pc-linux-gnu/7.2.0/../../../../x86_64-pc-linux-gnu/lib/x86_64-linux-gnu/
/opt/gcc/7.2.0/lib/gcc/x86_64-pc-linux-gnu/7.2.0/../../../../x86_64-pc-linux-gnu/lib/../lib64/
/opt/gcc/7.2.0/lib/gcc/x86_64-pc-linux-gnu/7.2.0/../../../x86_64-pc-linux-gnu/7.2.0/
/opt/gcc/7.2.0/lib/gcc/x86_64-pc-linux-gnu/7.2.0/../../../x86_64-linux-gnu/
/opt/gcc/7.2.0/lib/gcc/x86_64-pc-linux-gnu/7.2.0/../../../../lib64/
/lib/x86_64-pc-linux-gnu/7.2.0/
/lib/x86_64-linux-gnu/
/lib/../lib64/
/usr/lib/x86_64-pc-linux-gnu/7.2.0/
/usr/lib/x86_64-linux-gnu/
/usr/lib/../lib64/
/tmp/XXX/lib/
/opt/gcc/7.2.0/lib/gcc/x86_64-pc-linux-gnu/7.2.0/../../../../x86_64-pc-linux-gnu/lib/
/opt/gcc/7.2.0/lib/gcc/x86_64-pc-linux-gnu/7.2.0/../../../
/lib/
/usr/lib/

In addition to the normal search priority - where LIBRARY_PATH comes before system paths, GCC prioritizes a few "prefixes", including ../lib64. This can be worked around by creating another symlink:

ln -s lib /tmp/XXX/lib64

I thought this was related to the --libdir parameter during configure, which I omitted and is /usr/lib in the system installation, but even if i specify --libdir=$PREFIX/lib --libexecdir=$PREFIX/lib, it prefers ../lib64.

How to compile or control gcc at runtime so that it at least uses the ../lib instead of ../lib64 postfix?

Clang

Clang is even more uncooperative. It does not include LIBRARY_PATH in the output for --print-search-dirs and does not even include a -L/tmp/XXX/lib to it's call to ld if the libfoo.so can be found already in /usr/lib.

How can I get Clang to prioritize my library path transparently?

Notes

• Examples are from Archlinux, but I also tested under Ubuntu 16.04 which behaves similarly.
• Related questions for GCC: Why does g++ look in LIBRARY_PATH/../lib64 and where is this documented? and g++ searches /lib/../lib/, then /lib/.
• Overriding the search order with -L works, but is not transparent.
• gcc --print-search-dirs lists more directories than gcc -v. The latter filters out non-exiting paths.

Answer 1

I found out the reason why a custom installation of GCC is different. Debian distribution patches the GCC makefiles, that's how it gets the right priority of LIBRARY_PATH. Before building GCC, find gcc/config/i386/t-linux64 and there change all MULTILIB_OSDIRNAMES to these lines:

MULTILIB_OSDIRNAMES = m64=../lib$(call if_multiarch,:x86_64-linux-gnu)
MULTILIB_OSDIRNAMES+= m32=../lib32$(call if_multiarch,:i386-linux-gnu)
MULTILIB_OSDIRNAMES+= mx32=../libx32$(call if_multiarch,:x86_64-linux-gnux32)

Also add --libexecdir=/your/custom/path/lib --libdir=/your/custom/path/lib to configure.