How to automatically discover implicit dependencies when linking

Question

I'm tweaking an old build where many of the shared libraries don't link against libraries they actually depend on. There's some circular dependencies that have wormed their way into the build over the years I'd like to tackle, but it'll be far simpler if I can get all the non-circular dependencies worked out. Currently, unresolved symbols get resolved when linking executables -- which is how they managed to [unknowingly] introduce circular dependencies.

Here's a short bash script I put together to test some ideas out:

 #!/bin/bash

 mkdir -p objs

 for ((ii=0; ii<3; ii++)); do
   echo 'void FUNC(){}' | g++ -xc++ -DFUNC=f${ii} -fpic - -c -o objs/t${ii}.o
   g++ -fpic -shared objs/t${ii}.o -o libtest${ii}.so
 done

 # In the real-world case, libjoined.so isn't a trivial library.
 # Currently, it causes things to implicitly link against other
 # libraries libblah.so needs (like boost libraries)
 echo | g++ -xc++ -shared -fpic -L. -Wl,-R. -ltest{0,1,2} - -o libjoined.so

 g++ -xc++ -fpic - -c -o objs/b.o <<'EOF'
 extern void f0();
 extern void f1();
 extern void f2();

 void blah() { f0(); f1(); f2(); }
 EOF

 g++ -fpic -shared objs/b.o -L. -Wl,-R. -ljoined -o libblah.so -Wl,-zdefs

The last line will end up printing out some undefined symbol errors. If this were Solaris, the linker would examine other libraries that would otherwise satisfy the dependency implicitly and provide a hint:

 Undefined         first referenced
  symbol               in file
 void f1()             objs/b.o  (symbol belongs to implicit dependency ./libtest1.so)

... but I don't see similar behavior with GNU ld.

In the real-world case, there's a couple hundred libraries in our build plus all the external libraries they and the executables depend on. After adding -zdefs to the build, I'm getting around 10,000+ undefined references; identifying what dependencies are missing by hand would be rather annoying.

I need a fast way to identify the missing dependencies.

On the solaris side, I will probably try making a library like libjoined.so that just links against all of our libraries (where everything is built without -zdefs initially), then write a script that extracts the link command from logs and re-runs the command with -zdefs and links against this other library and dumps the hints to a text file or something.

On the linux side, I'm not sure what I can do to expedite this process.

Any suggestions are welcome.

Brian Vandenberg Brian Vandenberg · Accepted Answer · 2014-10-10T18:20:34

I found a way to do it with ld -r -p.

I can't copy the code over, but I'll give a mockup. I can't guarantee the following will run, but it should give you a good notion:

#!/usr/bin/perl

use warnings;
use strict;

my %symbols;
my %libs_missing_stuff;

foreach my $arg ( @ARGV) {
  # loop over libraries and make a map of the form:
  #   $symbols{$symbol} = $arg
  my @tmp = (`nm -Aa --defined-only ${arg}` =~ m{(\S+)$}g);
  foreach my $symbol ( @tmp ) {
    $symbols{$symbol} = $arg;
  }
}

foreach my $arg ( @ARGV ) {
# This is ultimately what I needed
  my @tmp = (`ldd -r -p $arg 2>&1` =~ m{symbol not found: (\S+)}g);
#             ^~~~~~~~^
  foreach my $missing_symbol ( @tmp ) {
    if( defined( $symbols{$missing_symbol} ) ) {
      $libs_missing_stuff{$arg}{$symbols{$missing_symbol}} = 1;
    }
  }
}

# loop over %libs_missing_stuff and print it out

The output looks like the following:

somelib : lib1 lib2 lib3
someotherlib : lib3 lib6 lib8
...

It's not perfect, and there's still the potential more than one library satisfies a dependency, or linked against some static library and appears to satisfy the dependency when in fact it should link against the other library. However, I have the means to figure that part out.

How to automatically discover implicit dependencies when linking

1 Answers