Bug in GNU make: target-specific variables are not expanded in implicit rules?

Question

I have been working on designing a multiple configuration Makefile (one that supports separate 'debug' and 'release' targets), and have come across a strange problem which appears to be a bug in GNU make.

It seems that GNU make is not expanding target-specific variables properly when those variables are referenced in an implicit rule. Here is a simplified Makefile which shows this issue:

all:
    @echo specify configuration 'debug' or 'release'

OBJS := foo.o bar.o

BUILDDIR = .build/$(CONFIG)

TARGET = $(addprefix $(BUILDDIR)/,$(OBJS))

debug: CONFIG := debug
release: CONFIG := release

#CONFIG := debug

debug: $(TARGET)
release: $(TARGET)

clean:
    rm -rf .build

$(BUILDDIR)/%.o: %.c
    @echo [$(BUILDDIR)/$*.o] should be [$@]
    @mkdir -p $(dir $@)
    $(CC) -c $< -o $@

When specifying the goal 'debug' to make, CONFIG is set to 'debug', and BUILDDIR and TARGET are likewise expanded properly. However, in the implicit rule to build the source file from the object, $@ is expanded as if CONFIG does not exist.

Here is the output from using this Makefile:

$ make debug
[.build/debug/foo.o] should be [.build//foo.o]
cc -c foo.c -o .build//foo.o
[.build/debug/bar.o] should be [.build//bar.o]
cc -c bar.c -o .build//bar.o

This shows that BUILDDIR is being expanded fine, but the resulting $@ is not. If I then comment out the target variable specification and manually set CONFIG := debug (the commented line above), I get what I would expect:

$ make debug
[.build/debug/foo.o] should be [.build/debug/foo.o]
cc -c foo.c -o .build/debug/foo.o
[.build/debug/bar.o] should be [.build/debug/bar.o]
cc -c bar.c -o .build/debug/bar.o

I've tested this with both make-3.81 on Gentoo and MinGW, and make-3.82 on Gentoo. All exhibit the same behavior.

I find it difficult to believe that I would be the first to come across this problem, so I'm guessing I'm probably just doing something wrong -- but I'll be honest: I don't see how I could be. :)

Are there any make gurus out there that might be able to shed some light on this issue? Thanks!

Answer 1

Basically, Make works out the DAG of dependencies and creates a list of rules that must be run before running any rule. Assigning a target-specific value is something that Make does when running a rule, which comes later. This is a serious limitation (which I and others have complained about before), but I wouldn't call it a bug since it is described in the documentation. According to the GNUMake manual:

6.11 Target-specific Variable Values: "As with automatic variables, these values are only available within the context of a target's recipe (and in other target-specific assignments)."

And "the context of a target's recipe" means the commands, not the prereqs:

10.5.3 Automatic variables: "[Automatic variables] cannot be accessed directly within the prerequisite list of a rule."

There are a couple of ways around this. You can use Secondary Expansion, if your version of Make GNUMake has it (3.81 doesn't, I don't know about 3.82). Or you can do without target-specific variables:

DEBUG_OBJS = $(addprefix $(BUILDDIR)/debug/,$(OBJS))
RELEASE_OBJS = $(addprefix $(BUILDDIR)/release/,$(OBJS))

debug: % : $(DEBUG_OBJS)
release: $(RELEASE_OBJS)

$(DEBUG_OBJS): $(BUILDDIR)/debug/%.o : %.cc
$(RELEASE_OBJS): $(BUILDDIR)/release/%.o : %.cc

$(DEBUG_OBJS) $(RELEASE_OBJS):
    @echo making $@ from $^
    @mkdir -p $(dir $@)                                                        
    $(CC) -c $< -o $@ 

Answer 2

As Beta has pointed out, this indeed isn't a bug in make since the limitation is described in the documentation (I guess I must have missed that particular part -- sorry).

In any case, I was actually able to work around this issue by doing something even simpler. Since all I need is to assign a variable based on the goal, I found that I can use the $(MAKECMDGOALS) variable in order to expand the build directory properly. Eliminating the $(CONFIG) variable and rewriting the Makefile as the following does exactly what I need:

all:
        @echo specify configuration 'debug' or 'release'

OBJS := foo.o bar.o

BUILDDIR := .build/$(MAKECMDGOALS)

TARGET := $(addprefix $(BUILDDIR)/,$(OBJS))

debug: $(TARGET)
release: $(TARGET)

clean:
        rm -rf .build

$(BUILDDIR)/%.o: %.c
        @echo [$(BUILDDIR)/$*.o] should be [$@]
        @mkdir -p $(dir $@)
        $(CC) -c $< -o $@

This then gives the proper result:

$ make debug
[.build/debug/foo.o] should be [.build/debug/foo.o]
cc -c foo.c -o .build/debug/foo.o
[.build/debug/bar.o] should be [.build/debug/bar.o]
cc -c bar.c -o .build/debug/bar.o
$ make release
[.build/release/foo.o] should be [.build/release/foo.o]
cc -c foo.c -o .build/release/foo.o
[.build/release/bar.o] should be [.build/release/bar.o]
cc -c bar.c -o .build/release/bar.o
$ make debug
make: Nothing to be done for `debug'.
$ make release
make: Nothing to be done for `release'.

This will of course break if there are multiple goals specified on the command line (since $(MAKECMDGOALS) contains a space-separated list), but dealing with that isn't too much of a problem.

Answer 3

Here is how to solve the problem without MAKECMDGOALS introspection. Tha problem is basically that the rules you specify in Makefile constitute a static graph. The target-specific assignments are used during the execution of rule bodies, but not during their compilation.

The solution to this is to grab control over rule compilation: use GNU Make's macro-like constructs to generate the rules. Then we have full control: we can stick variable material into the target, prerequisite or recipe.

Here is my version of your Makefile

all:
        @echo specify configuration 'debug' or 'release'

OBJS := foo.o bar.o

# BUILDDIR is a macro
# $(call BUILDDIR,WORD) -> .build/WORD
BUILDDIR = .build/$(1)

# target is a macro
# $(call TARGET,WORD) -> ./build/WORD/foo.o ./build/WORD/bar.o
TARGET = $(addprefix $(call BUILDDIR,$(1))/,$(OBJS))

# BUILDRULE is a macro: it builds a release or debug rule
# or whatever word we pass as argument $(1)
define BUILDRULE
$(call BUILDDIR,$(1))/%.o: %.c
        @echo [$(call BUILDDIR,$(1))/$$*.o] should be [$$@]
        @mkdir -p $$(dir $$@)
        $$(CC) -c -DMODE=$(1) $$< -o $$@
endef

debug: $(call TARGET,debug)
release: $(call TARGET,release)

# generate two build rules from macro
$(eval $(call BUILDRULE,debug))
$(eval $(call BUILDRULE,release))

clean:
        rm -rf .build

Now, notice the advantage: I can build both debug and release targets in one go, because I have instantiated both rules from the template!

$ make clean ; make debug release
rm -rf .build
[.build/debug/foo.o] should be [.build/debug/foo.o]
cc -c -DMODE=debug foo.c -o .build/debug/foo.o
[.build/debug/bar.o] should be [.build/debug/bar.o]
cc -c -DMODE=debug bar.c -o .build/debug/bar.o
[.build/release/foo.o] should be [.build/release/foo.o]
cc -c -DMODE=release foo.c -o .build/release/foo.o
[.build/release/bar.o] should be [.build/release/bar.o]
cc -c -DMODE=release bar.c -o .build/release/bar.o

Furthermore, I have taken liberty to add the macro argument into the cc command line also, so that the modules receive a MODE macro which tells them how they are being compiled.

We can use variable indirection to set up different CFLAGS or whatever. Watch what happens if we patch the above like this:

--- a/Makefile
+++ b/Makefile
@@ -3,6 +3,9 @@

 OBJS := foo.o bar.o

+CFLAGS_debug = -O0 -g
+CFLAGS_release = -O2
+
 # BUILDDIR is a macro
 # $(call BUILDDIR,WORD) -> .build/WORD
 BUILDDIR = .build/$(1)
@@ -17,7 +20,7 @@ define BUILDRULE
 $(call BUILDDIR,$(1))/%.o: %.c
        @echo [$(call BUILDDIR,$(1))/$$*.o] should be [$$@]
        @mkdir -p $$(dir $$@)
-       $$(CC) -c -DMODE=$(1) $$< -o $$@
+       $$(CC) -c $$(CFLAGS_$(1)) -DMODE=$(1) $$< -o $$@
 endef

 debug: $(call TARGET,debug)

Run:

$ make clean ; make debug release
rm -rf .build
[.build/debug/foo.o] should be [.build/debug/foo.o]
cc -c -O0 -g -DMODE=debug foo.c -o .build/debug/foo.o
[.build/debug/bar.o] should be [.build/debug/bar.o]
cc -c -O0 -g -DMODE=debug bar.c -o .build/debug/bar.o
[.build/release/foo.o] should be [.build/release/foo.o]
cc -c -O2 -DMODE=release foo.c -o .build/release/foo.o
[.build/release/bar.o] should be [.build/release/bar.o]
cc -c -O2 -DMODE=release bar.c -o .build/release/bar.o

Finally, we can combine that with MAKECMDGOALS. We can inspect MAKECMDGOALS and filter out the build modes which are not specified there. If make release is called, we don't need the debug rules to be expanded. Patch:

--- a/Makefile
+++ b/Makefile
@@ -3,6 +3,11 @@

 OBJS := foo.o bar.o

+# List of build types, but only those mentioned on command line
+BUILD_TYPES := $(filter $(MAKECMDGOALS),debug release)
+
+$(warning "generating rules for BUILD_TYPES := $(BUILD_TYPES)")
+
 CFLAGS_debug = -O0 -g
 CFLAGS_release = -O2

@@ -17,18 +22,15 @@ TARGET = $(addprefix $(call BUILDDIR,$(1))/,$(OBJS))
 # BUILDRULE is a macro: it builds a release or debug rule
 # or whatever word we pass as argument $(1)
 define BUILDRULE
+$(1): $(call TARGET,$(1))
 $(call BUILDDIR,$(1))/%.o: %.c
        @echo [$(call BUILDDIR,$(1))/$$*.o] should be [$$@]
        @mkdir -p $$(dir $$@)
        $$(CC) -c $$(CFLAGS_$(1)) -DMODE=$(1) $$< -o $$@
 endef

-debug: $(call TARGET,debug)
-release: $(call TARGET,release)
-
-# generate two build rules from macro
-$(eval $(call BUILDRULE,debug))
-$(eval $(call BUILDRULE,release))
+$(foreach type,$(BUILD_TYPES),\
+  $(eval $(call BUILDRULE,$(type))))

 clean:
        rm -rf .build

Note that I simplified things by rolling the debug: and release: targets into the BUILDRULE macro.

$ make clean ; make release
Makefile:9: "generating rules for BUILD_TYPES := "
rm -rf .build
Makefile:9: "generating rules for BUILD_TYPES := release"
[.build/release/foo.o] should be [.build/release/foo.o]
cc -c -O2 -DMODE=release foo.c -o .build/release/foo.o
[.build/release/bar.o] should be [.build/release/bar.o]
cc -c -O2 -DMODE=release bar.c -o .build/release/bar.o

$ make clean ; make release debug
Makefile:9: "generating rules for BUILD_TYPES := "
rm -rf .build
Makefile:9: "generating rules for BUILD_TYPES := debug release"
[.build/release/foo.o] should be [.build/release/foo.o]
cc -c -O2 -DMODE=release foo.c -o .build/release/foo.o
[.build/release/bar.o] should be [.build/release/bar.o]
cc -c -O2 -DMODE=release bar.c -o .build/release/bar.o
[.build/debug/foo.o] should be [.build/debug/foo.o]
cc -c -O0 -g -DMODE=debug foo.c -o .build/debug/foo.o
[.build/debug/bar.o] should be [.build/debug/bar.o]
cc -c -O0 -g -DMODE=debug bar.c -o .build/debug/bar.o