phony targets for parallel execution of make

Question

the man page for the make utility says -

Another example of the usefulness of phony targets is in conjunction with recursive invocations of make (for more information, see Recursive Use of make). In this case the makefile will often contain a variable which lists a number of sub-directories to be built. One way to handle this is with one rule whose recipe is a shell loop over the sub-directories, like this:

 SUBDIRS = foo bar baz

 subdirs:
         for dir in $(SUBDIRS); do \
           $(MAKE) -C $$dir; \
         done

There are problems with this method, however. First, any error detected in a sub-make is ignored by this rule, so it will continue to build the rest of the directories even when one fails. This can be overcome by adding shell commands to note the error and exit, but then it will do so even if make is invoked with the -k option, which is unfortunate. Second, and perhaps more importantly, you cannot take advantage of make's ability to build targets in parallel (see Parallel Execution), since there is only one rule.

By declaring the sub-directories as phony targets (you must do this as the sub-directory obviously always exists; otherwise it won't be built) you can remove these problems:

 SUBDIRS = foo bar baz

 .PHONY: subdirs $(SUBDIRS)

 subdirs: $(SUBDIRS)

 $(SUBDIRS):
         $(MAKE) -C $@

 foo: baz

Here we've also declared that the foo sub-directory cannot be built until after the baz sub-directory is complete; this kind of relationship declaration is particularly important when attempting parallel builds.

Could someone explain as to how the second code mentioned above enables parallel execution? I fail to understand how Phony Targets has helped here?

Answer 1

Let me tell in simple words so that you will get some idea

as arved mentioned -j2 option can be given to tell make file run two parallel threads. make checks for the targets and its dependencies if two targets are independent then they can be built parallely.

Let me expand the second makefile and write it simple

SUBDIRS = foo bar baz

 .PHONY: subdirs $(SUBDIRS)

 subdirs: $(SUBDIRS)

 bar:
          $(MAKE) -C bar
 baz:
          $(MAKE) -C baz 

 foo: baz
          $(MAKE) -C foo

Now see the above case

1. bar and baz depends on nothing so they can be built paralelly

2. foo depends on baz so it cant be built parallel to bar or baz

So in above case you ensure that make file uses the ability to built files parallelly.

Now take the first make file you mentioned

 SUBDIRS = foo bar baz

 subdirs:
         for dir in $(SUBDIRS); do \
           $(MAKE) -C $$dir; \
         done

so if you don't mention as .PHONY targets above code is trick for you, but with a penalty of loosing ability to build parallel. Observe closely. There is only one target subdirs which runs three loops. all the three loops will be run one after the other so no parallelism.

Just remember the thumb rule. Make can use the parallel build if the targets are independent of each other so it can build independent target parallel without any dependency issues

Answer 2

If you don't mark the subdirectories as phony make will check if a file named foo exists and immediately mark the target as done because the directory "foo" exists

Marking it PHONY ensures that the target is always remade even if a file exists.

While in example 1 only one make process can be used which sequentially builds all directories, in example 2 make -j2 will result in bar and baz targets being built in parallel.

Answer 3

The idea is making subdirs a target so make can "build" them in parallel. There is no dependency between them, there is a clear rule what has to be done to "build" them, make can process multiple at the same time. For loop would build everything sequentially, one after another, iteration after iteration.

Phony is only a technicality, we need it because otherwise make would see that there is something with the name of subdir and it was modified long time ago so nothing has to be done. Same as you has to declare all, clean etc as phony in case someone creates a file named all.

Answer 4

Parallelism is possible when a target lists multiple prerequisites:

all: a b c

a: a.c
b: b.c
c: c.c

Since a, b, and c are prerequistes listed on the all line, it is possible to run their commands in parallel. When the prequisites are built, then the all rules can be performed. So:

$ make -j3 all

would allow the 3 prerequisites to be built in parallel.

The .PHONY: target declaration prevents:

$ make -t 

from creating a zero-length target.