Inconsistent expansion by make for the '$?' variable

Question

From the docs:

$?

The names of all the prerequisites that are newer than the target, with spaces between them.

So, given a makefile:

# Force make to search for 'foo' in the VPATH directory
$(shell rm -rf foo)
# If 'D' is a "regular" file, we remove it first.
$(shell rm -rf D)
$(shell mkdir D)
# Suggest a VPATH-file, for Make to "associate" with 'foo'.
$(shell touch D/foo)
$(shell sleep 1)
# Target 'all' is newer than prerequisite 'D/foo'
$(shell touch all)

VPATH = D

all : foo phony
    echo '$?'

foo ::
    touch '$@'

.PHONY: phony

.PRECIOUS : D/foo

And running, I get:

$ make -r
touch 'D/foo'
echo 'D/foo phony'
D/foo phony

# Try again, but this time, with parallel-execution mode.
$ make -r -j
touch 'D/foo'
echo 'phony'
phony

Here, we have 2 serious issues:

Given the simple and explicit recipe to "touch" the prerequisite foo, which Make clearly executes - hence will guarantee that foo will be "newer" than all - Make still does not expand $? to D/foo, at-least in the 2nd case above (i.e. for the parallel-execution (-j) mode). Why?
If you come up with an explanation for the above, shouldn't it also explain, why in the 1st case (non-parallel execution), $? - does indeed - get expanded to D/foo.

I guess, I had an assumption, that parallel vs. non-parallel aside, Make will always pause before executing a target, and first check if all of its prerequisites had already finished their respective builds.

So, shouldn't the $? variable be identically expanded for both cases?

Observation: I don't like the way you are writing your makefiles. I've seen a number of your questions, and I've not answered any of them, in part on those grounds (and in part because the make I use most frequently won't run the makefiles). The $(shell …) material grates on me (or is the principal source of concern). OTOH, I'm just an old fuddy-duddy, so you don't have to pay attention, but … I'm not keen on the notation you're using. — Jonathan Leffler
@JonathanLeffler I've wondered about that myself. I've mostly convinced myself that's just a way to make them self-contained but I'm not sure. I also think, at least in some cases that might affect things, but I just pulled it out of this test and it didn't. Though my results aren't consistent for the non--j case in that I don't always get D/foo listed there either. And with make 3.81 I don't see phony in that list at all. — Etan Reisner
Ohh! Makefile syntax is notorious for its beauty and readability. NOT! And this is the only syntax that Make provides to communicate with the shell. Is it nice? NOT! Is it reliable? NOT! Is there a way to check for erros from the shell? Mostly not! But, there is no other consistent way, to guarantee a reproduction of bugs in makefile, unless you embed those lines, and "hope" for the best. — Ji Cha
You don't need to use only the makefile for this. A tiny wrapper shell script works just fine too. It isn't as if you get points for saving space. — Etan Reisner
Have you tried changing the sleep 0.1 to sleep 10 (or, more sensibly, sleep 2). I wonder if sub-second granularity is part of the problem. — Jonathan Leffler

Etan Reisner Etan Reisner · Accepted Answer · 2015-08-21T04:08:36

I think there are two issues going on here.

The first is that double-colon rules appear to act like phony targets in that they force make to consider the target as "newer" regardless of actual modification time. (This is why the non-parallel version behaves the way it does. Change from foo :: to foo : and you don't get foo in the $? output at all.)

The second thing is that, despite that, using parallel mode seems to force make back into considering modification times of its prerequisites (so the previous behavior is avoided).

This is conjecture and not definitive since I haven't dug through the code to see if this is actually happening but it explains the results here (it also explains the results on the other, nearly identical, question here).

Inconsistent expansion by make for the '$?' variable

1 Answers