attribute((weak)) and static libraries

Question

I wanted to introduce a weak symbol into my code, however, I am unable to comprehend its behavior when *.a files are used.

This is my minimal example:

file a.h:

void foo() __attribute__((weak));

file a.c:

#include "a.h"
#include <stdio.h>

void foo() { printf("%s\n", __FILE__); }

file b.c:

#include <stdio.h>

void foo() { printf("%s\n", __FILE__); }

file main.cpp:

#include "a.h"
#include <stdio.h>

int main() { if (foo) foo(); else printf("no foo\n"); }

Now, depending if I use *.o files (make -c a.c and make -c b.c) or *.a files (ar cr a.o and ar cr b.o) the output is different:

1) g++ main.cpp a.o b.o prints b.c
2) g++ main.cpp b.o a.o prints b.c
3) g++ main.cpp a.a b.a prints no foo
4) g++ main.cpp b.a a.a prints no foo

1), 2) work just fine but the output for 3), 4) seems to be a little unexpected.

I was desperately trying to make this example work with archives so I made few changes:

file a.h:

void foo();

file a.c:

#include "a.h"
#include <stdio.h>

void __attribute__((weak)) foo() { printf("%s\n", __FILE__); }

After this modification:

1) g++ main.cpp a.a b.a prints a.c
2) g++ main.cpp b.a a.a prints b.c

So it works a bit better. After running nm a.a shows W _Z3foov so there is no violation of ODR. However, I don't know if this is a correct usage of weak attribute. According to gcc documentation:

The weak attribute causes the declaration to be emitted as a weak symbol rather than a global. This is primarily useful in defining library functions which can be overridden in user code, though it can also be used with non-function declarations. Weak symbols are supported for ELF targets, and also for a.out targets when using the GNU assembler and linker.

Yet I use weak attribute on the function definition not the declaration.

So the question is why weak doesn't work with *.a files? Is usage of weak attribute on a definition instead of a declaration correct?

UPDATE

It has dawned on me that weak attribute used with foo() method definition had no impact on the symbol resolution. Without the attribute final binary generates the same:

1) g++ main.cpp a.a b.a prints a.c
2) g++ main.cpp b.a a.a prints b.c

So simply the first definition of the symbol is used and this is consisten with default gcc behaviour. Even though nm a.a shows that a weak symbol was emitted, it doesn't seem to affect static linking.

Is it possible to use weak attribute with static linking at all?

DESCRIPTION OF THE PROBLEM I WANT TO SOLVE

I have a library that is used by >20 clients, let's call it library A. I also provide a library B which contains testing utils for A. Somehow I need to know that library A is used in testing mode, so the simplest solution seems to be replacing a symbol during linking with B (because clients are already linking with B).

I know there are cleaner solutions to this problem, however I absolutely can't impact clients' code or their build scripts (adding parameter that would indicate testing for A or some DEFINE for compilation is out of option).

I want to override original function implementation when linking with a testing framework. Previously I have dealt with similar issues using global variables, however, this is a fragile solution due to an undefined order of global initialization. — AndyB
Use objcopy for that. Replace or remove original function implementation from the framework or set the original function as weak using objcopy. You don't need to recompile library, thus impact compilation process of the original framework. ex. objcopy -N foo a.o — KamilCuk
How did you get it to output no foo? I can't reproduce it. In case of missing foo symbol the linker should emit an error, the foo symbol can't be equal to false, unless you somewhere define void (*foo)() = 0;. Did you use a C++ compiler with a C symbols? If you used make -c that usually runs a C compiler cc which creates a foo symbol, not a C++ _Z3foov symbol. — KamilCuk
To generate no foo output I have used following commands: g++ -c a.c ar cr a.a a.o g++ -c b.c ar cr b.a b.o g++ main.cpp a.a b.a About the void (*foo)() = 0; as Mike Kinghan has already mentioned, a weak symbol may be left undefined then its value is assumed to be 0. I have used g++ all the time. I have mixed c and cpp extensions by mistake when writing this post. This isn't an issue in my original project. — AndyB

Mike Kinghan Mike Kinghan · Accepted Answer · 2018-08-22T15:38:20

To explain what's going on here, let's talk first about your original source files, with

a.h (1):

void foo() __attribute__((weak));

and:

a.c (1):

#include "a.h"
#include <stdio.h>

void foo() { printf("%s\n", __FILE__); }

The mixture of .c and .cpp files in your sample code is irrelevant to the issues, and all the code is C, so we'll say that main.cpp is main.c and do all compiling and linking with gcc:

$ gcc -Wall -c main.c a.c b.c
ar rcs a.a a.o
ar rcs b.a b.o

First let's review the differences between a weakly declared symbol, like your:

void foo() __attribute__((weak));

and a strongly declared symbol, like

void foo();

which is the default:

When a weak reference to foo (i.e. a reference to weakly declared foo) is linked in a program, the linker need not find a definition of foo anywhere in the linkage: it may remain undefined. If a strong reference to foo is linked in a program, the linker needs to find a definition of foo.
A linkage may contain at most one strong definition of foo (i.e. a definition of foo that declares it strongly). Otherwise a multiple-definition error results. But it may contain multiple weak definitions of foo without error.
If a linkage contains one or more weak definitions of foo and also a strong definition, then the linker chooses the strong definition and ignores the weak ones.
If a linkage contains just one weak definition of foo and no strong definition, inevitably the linker uses the one weak definition.
If a linkage contains multiple weak definitions of foo and no strong definition, then the linker chooses one of the weak definitions arbitrarily.

Next, let's review the differences between inputting an object file in a linkage and inputting a static library.

A static library is merely an ar archive of object files that we may offer to the linker from which to select the ones it needs to carry on the linkage.

When an object file is input to a linkage, the linker unconditionally links it into the output file.

When static library is input to a linkage, the linker examines the archive to find any object files within it that provide definitions it needs for unresolved symbol references that have accrued from input files already linked. If it finds any such object files in the archive, it extracts them and links them into the output file, exactly as if they were individually named input files and the static library was not mentioned at all.

With these observations in mind, consider the compile-and-link command:

gcc main.c a.o b.o

Behind the scenes gcc breaks it down, as it must, into a compile-step and link step, just as if you had run:

gcc -c main.c     # compile
gcc main.o a.o b.o  # link

All three object files are linked unconditionally into the (default) program ./a.out. a.o contains a weak definition of foo, as we can see:

$ nm --defined a.o
0000000000000000 W foo

Whereas b.o contains a strong definition:

$ nm --defined b.o
0000000000000000 T foo

The linker will find both definitions and choose the strong one from b.o, as we can also see:

$ gcc main.o a.o b.o -Wl,-trace-symbol=foo
main.o: reference to foo
a.o: definition of foo
b.o: definition of foo
$ ./a.out
b.c

Reversing the linkage order of a.o and b.o will make no difference: there's still exactly one strong definition of foo, the one in b.o.

By contrast consider the compile-and-link command:

gcc main.cpp a.a b.a

which breaks down into:

gcc -c main.cpp     # compile
gcc main.o a.a b.a  # link

Here, only main.o is linked unconditionally. That puts an undefined weak reference to foo into the linkage:

$ nm --undefined main.o
                 w foo
                 U _GLOBAL_OFFSET_TABLE_
                 U puts

That weak reference to foo does not need a definition. So the linker will not attempt to find a definition that resolves it in any of the object files in either a.a or b.a and will leave it undefined in the program, as we can see:

$ gcc main.o a.a b.a -Wl,-trace-symbol=foo
main.o: reference to foo
$ nm --undefined a.out
                 w __cxa_finalize@@GLIBC_2.2.5
                 w foo
                 w __gmon_start__
                 w _ITM_deregisterTMCloneTable
                 w _ITM_registerTMCloneTable
                 U __libc_start_main@@GLIBC_2.2.5
                 U puts@@GLIBC_2.2.5

Hence:

$ ./a.out
no foo

Again, it doesn't matter if you reverse the linkage order of a.a and b.a, but this time it is because neither of them contributes anything to the linkage.

Let's turn now to the different behavior you discovered by changing a.h and a.c to:

a.h (2):

void foo();

a.c (2):

#include "a.h"
#include <stdio.h>

void __attribute__((weak)) foo() { printf("%s\n", __FILE__); }

Once again:

$ gcc -Wall -c main.c a.c b.c
main.c: In function ‘main’:
main.c:4:18: warning: the address of ‘foo’ will always evaluate as ‘true’ [-Waddress]
 int main() { if (foo) foo(); else printf("no foo\n"); }

See that warning? main.o now contains a strongly declared reference to foo:

$ nm --undefined main.o
                 U foo
                 U _GLOBAL_OFFSET_TABLE_

so the code (when linked) must have a non-null address for foo. Proceeding:

$ ar rcs a.a a.o
$ ar rcs b.a b.o

Then try the linkage:

$ gcc main.o a.o b.o
$ ./a.out
b.c

And with the object files reversed:

$ gcc main.o b.o a.o
$ ./a.out
b.c

As before, the order makes no difference. All the object files are linked. b.o provides a strong definition of foo, a.o provides a weak one, so b.o wins.

Next try the linkage:

$ gcc main.o a.a b.a
$ ./a.out
a.c

And with the order of the libraries reversed:

$ gcc main.o b.a a.a
$ ./a.out
b.c

That does make a difference. Why? Let's redo the linkages with diagnostics:

$ gcc main.o a.a b.a -Wl,-trace,-trace-symbol=foo
/usr/bin/x86_64-linux-gnu-ld: mode elf_x86_64
/usr/lib/gcc/x86_64-linux-gnu/7/../../../x86_64-linux-gnu/Scrt1.o
/usr/lib/gcc/x86_64-linux-gnu/7/../../../x86_64-linux-gnu/crti.o
/usr/lib/gcc/x86_64-linux-gnu/7/crtbeginS.o
main.o
(a.a)a.o
libgcc_s.so.1 (/usr/lib/gcc/x86_64-linux-gnu/7/libgcc_s.so.1)
/lib/x86_64-linux-gnu/libc.so.6
(/usr/lib/x86_64-linux-gnu/libc_nonshared.a)elf-init.oS
/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2
/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2
libgcc_s.so.1 (/usr/lib/gcc/x86_64-linux-gnu/7/libgcc_s.so.1)
/usr/lib/gcc/x86_64-linux-gnu/7/crtendS.o
/usr/lib/gcc/x86_64-linux-gnu/7/../../../x86_64-linux-gnu/crtn.o
main.o: reference to foo
a.a(a.o): definition of foo

Ignoring the default libraries, the only object files of ours that get linked were:

main.o
(a.a)a.o

And the definition of foo was taken from the archive member a.o of a.a:

a.a(a.o): definition of foo

Reversing the library order:

$ gcc main.o b.a a.a -Wl,-trace,-trace-symbol=foo
/usr/bin/x86_64-linux-gnu-ld: mode elf_x86_64
/usr/lib/gcc/x86_64-linux-gnu/7/../../../x86_64-linux-gnu/Scrt1.o
/usr/lib/gcc/x86_64-linux-gnu/7/../../../x86_64-linux-gnu/crti.o
/usr/lib/gcc/x86_64-linux-gnu/7/crtbeginS.o
main.o
(b.a)b.o
libgcc_s.so.1 (/usr/lib/gcc/x86_64-linux-gnu/7/libgcc_s.so.1)
/lib/x86_64-linux-gnu/libc.so.6
(/usr/lib/x86_64-linux-gnu/libc_nonshared.a)elf-init.oS
/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2
/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2
libgcc_s.so.1 (/usr/lib/gcc/x86_64-linux-gnu/7/libgcc_s.so.1)
/usr/lib/gcc/x86_64-linux-gnu/7/crtendS.o
/usr/lib/gcc/x86_64-linux-gnu/7/../../../x86_64-linux-gnu/crtn.o
main.o: reference to foo
b.a(b.o): definition of foo

This time the object files linked were:

main.o
(b.a)b.o

And the definition of foo was taken from b.o in b.a:

b.a(b.o): definition of foo

In the first linkage, the linker had an unresolved strong reference to foo for which it needed a definition when it reached a.a. So it looked in the archive for an object file that provides a definition, and found a.o. That definition was a weak one, but that didn't matter. No strong definition had been seen. a.o was extracted from a.a and linked, and the reference to foo was thus resolved. Next b.a was reached, where a strong definition of foo would have been found in b.o, if the linker still needed one and looked for it. But it didn't need one any more and didn't look. The linkage:

gcc main.o a.a b.a

is exactly the same as:

gcc main.o a.o

And likewise the linkage:

$ gcc main.o b.a a.a

is exactly the same as:

$ gcc main.o b.o

Your real problem...

... emerges in one of your comments to the post:

I want to override [the] original function implementation when linking with a testing framework.

You want to link a program inputting some static library lib1.a which has some member file1.o that defines a symbol foo, and you want to knock out that definition of foo and link a different one that is defined in some other object file file2.o.

__attribute__((weak)) isn't applicable to that problem. The solution is more elementary. You just make sure to input file2.o to the linkage before you input lib1.a (and before any other input that provides a definition of foo). Then the linker will resolve references to foo with the definition provided in file2.o and will not try to find any other definition when it reaches lib1.a. The linker will not consume lib1.a(file1.o) at all. It might as well not exist.

And what if you have put file2.o in another static library lib2.a? Then inputting lib2.a before lib1.a will do the job of linking lib2.a(file2.o) before lib1.a is reached and resolving foo to the definition in file2.o.

Likewise, of course, every definition provided by members of lib2.a will be linked in preference to a definition of the same symbol provided in lib1.a. If that's not what you want, then don't like lib2.a: link file2.o itself.

Finally

Is it possible to use [the] weak attribute with static linking at all?

Certainly. Here is a first-principles use-case:

foo.h (1)

#ifndef FOO_H
#define FOO_H

int __attribute__((weak)) foo(int i)
{
    return i != 0;
}

#endif

aa.c

#include "foo.h"

int a(void)
{
    return foo(0);
}

bb.c

#include "foo.h"

int b(void)
{
    return foo(42);
}

prog.c

#include <stdio.h>

extern int a(void);
extern int b(void);

int main(void)
{
    puts(a() ? "true" : "false");
    puts(b() ? "true" : "false");
    return 0;
}

Compile all the source files, requesting a seperate ELF section for each function:

$ gcc -Wall -ffunction-sections -c prog.c aa.c bb.c

Note that the weak definition of foo is compiled via foo.h into both aa.o and bb.o, as we can see:

$ nm --defined aa.o
0000000000000000 T a
0000000000000000 W foo
$ nm --defined bb.o
0000000000000000 T b
0000000000000000 W foo

Now link a program from all the object files, requesting the linker to discard unused sections (and give us the map-file, and some diagnostics):

$ gcc prog.o aa.o bb.o -Wl,--gc-sections,-Map=mapfile,-trace,-trace-symbol=foo
/usr/bin/x86_64-linux-gnu-ld: mode elf_x86_64
/usr/lib/gcc/x86_64-linux-gnu/7/../../../x86_64-linux-gnu/Scrt1.o
/usr/lib/gcc/x86_64-linux-gnu/7/../../../x86_64-linux-gnu/crti.o
/usr/lib/gcc/x86_64-linux-gnu/7/crtbeginS.o
prog.o
aa.o
bb.o
libgcc_s.so.1 (/usr/lib/gcc/x86_64-linux-gnu/7/libgcc_s.so.1)
/lib/x86_64-linux-gnu/libc.so.6
(/usr/lib/x86_64-linux-gnu/libc_nonshared.a)elf-init.oS
/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2
/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2
libgcc_s.so.1 (/usr/lib/gcc/x86_64-linux-gnu/7/libgcc_s.so.1)
/usr/lib/gcc/x86_64-linux-gnu/7/crtendS.o
/usr/lib/gcc/x86_64-linux-gnu/7/../../../x86_64-linux-gnu/crtn.o
aa.o: definition of foo

This linkage is no different from:

$ ar rcs libaabb.a aa.o bb.o
$ gcc prog.o libaabb.a

Despite the fact that both aa.o and bb.o were loaded, and each contains a definition of foo, no multiple-definition error results, because each definition is weak. aa.o was loaded before bb.o and the definition of foo was linked from aa.o.

So what happened to the definition of foo in bb.o? The mapfile shows us:

mapfile (1)

...
...
Discarded input sections
...
...
 .text.foo      0x0000000000000000       0x13 bb.o
...
...

The linker discarded the function section that contained the definition in bb.o

Let's reverse the linkage order of aa.o and bb.o:

$ gcc prog.o bb.o aa.o -Wl,--gc-sections,-Map=mapfile,-trace,-trace-symbol=foo
...
prog.o
bb.o
aa.o
...
bb.o: definition of foo

And now the opposite thing happens. bb.o is loaded before aa.o. The definition of foo is linked from bb.o and:

mapfile (2)

...
...
Discarded input sections
...
...
 .text.foo      0x0000000000000000       0x13 aa.o
...
...

the definition from aa.o is chucked away.

There you see how the linker arbitrarily chooses one of multiple weak definitions of a symbol, in the absence of a strong definition. It simply picks the first one you give it and ignores the rest.

What we've just done here is effectively what the GCC C++ compiler does for us when we define a global inline function. Rewrite:

foo.h (2)

#ifndef FOO_H
#define FOO_H

inline int foo(int i)
{
    return i != 0;
}

#endif

Rename our source files *.c -> *.cpp; compile and link:

$ g++ -Wall -c prog.cpp aa.cpp bb.cpp

Now there is a weak definition of foo (C++ mangled) in each of aa.o and bb.o:

$ nm --defined aa.o bb.o

aa.o:
0000000000000000 T _Z1av
0000000000000000 W _Z3fooi

bb.o:
0000000000000000 T _Z1bv
0000000000000000 W _Z3fooi

The linkage uses the first definition it finds:

$ g++ prog.o aa.o bb.o -Wl,-Map=mapfile,-trace,-trace-symbol=_Z3fooi
...
prog.o
aa.o
bb.o
...
aa.o: definition of _Z3fooi
bb.o: reference to _Z3fooi

and throws away the other one:

mapfile (3)

...
...
Discarded input sections
...
...
 .text._Z3fooi  0x0000000000000000       0x13 bb.o
...
...

And as you may know, every instantiation of the C++ function template in global scope (or instantiation of a class template member function) is an inline global function. Rewrite again:

#ifndef FOO_H
#define FOO_H

template<typename T>
T foo(T i)
{
    return i != 0;
}

#endif

Recompile:

$ g++ -Wall -c prog.cpp aa.cpp bb.cpp

Again:

$ nm --defined aa.o bb.o

aa.o:
0000000000000000 T _Z1av
0000000000000000 W _Z3fooIiET_S0_

bb.o:
0000000000000000 T _Z1bv
0000000000000000 W _Z3fooIiET_S0_

each of aa.o and bb.o has a weak definition of:

$ c++filt _Z3fooIiET_S0_
int foo<int>(int)

and the linkage behaviour is now familiar. One way:

$ g++ prog.o aa.o bb.o -Wl,-Map=mapfile,-trace,-trace-symbol=_Z3fooIiET_S0_
...
prog.o
aa.o
bb.o
...
aa.o: definition of _Z3fooIiET_S0_
bb.o: reference to _Z3fooIiET_S0_

and the other way:

$ g++ prog.o bb.o aa.o -Wl,-Map=mapfile,-trace,-trace-symbol=_Z3fooIiET_S0_
...
prog.o
bb.o
aa.o
...
bb.o: definition of _Z3fooIiET_S0_
aa.o: reference to _Z3fooIiET_S0_

Our program's behavior is unchanged by the rewrites:

$ ./a.out
false
true

So the application of the weak attribute to symbols in the linkage of ELF objects - whether static or dynamic - enables the GCC implementation of C++ templates for the GNU linker. You could fairly say it enables the GCC implementation of modern C++.

__attribute__((weak)) and static libraries

2 Answers

attribute((weak)) and static libraries