GNU Makefile rule generating a few targets from a single source file

Question

I am attempting to do the following. There is a program, call it foo-bin, that takes in a single input file and generates two output files. A dumb Makefile rule for this would be:

file-a.out file-b.out: input.in
    foo-bin input.in file-a.out file-b.out

However, this does not tell make in any way that both targets will be generated simultaneously. That is fine when running make in serial, but will likely cause trouble if one tries make -j16 or something equally crazy.

The question is whether there exists a way to write a proper Makefile rule for such a case? Clearly, it would generate a DAG, but somehow the GNU make manual does not specify how this case could be handled.

Running the same code twice and generating only one result is out of the question, because the computation takes time (think: hours). Outputting only one file would also be rather difficult, because frequently it is used as an input to GNUPLOT which doesn't know how to handle only a fraction of a data file.

Answer 1

The trick is to use a pattern rule with multiple targets. In that case make will assume that both targets are created by a single invocation of the command.

all: file-a.out file-b.out
file-a%out file-b%out: input.in
    foo-bin input.in file-a$*out file-b$*out

This difference in interpretation between pattern rules and normal rules doesn't exactly make sense, but it's useful for cases like this, and it is documented in the manual.

This trick can be used for any number of output files as long as their names have some common substring for the % to match. (In this case the common substring is ".")

Answer 2

Make doesn't have any intuitive way to do this, but there are two decent workarounds.

First, if the targets involved have a common stem, you can use a prefix rule (with GNU make). That is, if you wanted to fix the following rule:

object.out1 object.out2: object.input
    foo-bin object.input object.out1 object.out2

You could write it this way:

%.out1 %.out2: %.input
    foo-bin $*.input $*.out1 $*.out2

(Using the pattern-rule variable $*, which stands for the matched part of the pattern)

If you want to be portable to non-GNU Make implementations or if your files can't be named to match a pattern rule, there is another way:

file-a.out file-b.out: input.in.intermediate ;

.INTERMEDIATE: input.in.intermediate
input.in.intermediate: input.in
    foo-bin input.in file-a.out file-b.out

This tells make that input.in.intermediate won't exist before make is run, so its absence (or its timestamp) won't cause foo-bin to be run spuriously. And whether either file-a.out or file-b.out or both are out-of-date (relative to input.in), foo-bin will be only run once. You can use .SECONDARY instead of .INTERMEDIATE, which will instruct make NOT to delete a hypothetical file name input.in.intermediate. This method is also safe for parallel make builds.

The semicolon on the first line is important. It creates an empty recipe for that rule, so that Make knows we will really be updating file-a.out and file-b.out (thanks @siulkiulki and others who pointed this out)

Answer 3

After GNU Make 4.3 (19th Jan. 2020) you can use “grouped explicit targets”. Replace : with &:.

file-a.out file-b.out &: input.in
    foo-bin input.in file-a.out file-b.out

The NEWS file of the GNU Make says:

New feature: Grouped explicit targets

Pattern rules have always had the ability to generate multiple targets with a single invocation of the recipe. It's now possible to declare that an explicit rule generates multiple targets with a single invocation. To use this, replace the ":" token with "&:" in the rule. To detect this feature search for 'grouped-target' in the .FEATURES special variable. Implementation contributed by Kaz Kylheku <[email protected]>

Answer 4

I would solve it as follows :

file-a.out: input.in
    foo-bin input.in file-a.out file-b.out   

file-b.out: file-a.out
    #do nothing
    noop

In this case parallel make will 'serialize' creating a and b but since creating b does not do anything it takes no time.

Answer 5

This is based on @deemer's second answer which does not rely on pattern rules, and it fixes an issue I was experiencing with nested uses of the workaround.

file-a.out file-b.out: input.in.intermediate
    @# Empty recipe to propagate "newness" from the intermediate to final targets

.INTERMEDIATE: input.in.intermediate
input.in.intermediate: input.in
    foo-bin input.in file-a.out file-b.out

---

I would have added this as a comment to @deemer's answer, but I can't because I just created this account and don't have any reputation.

Explanation: The empty recipe is needed in order to allow Make to do the proper bookkeeping to mark file-a.out and file-b.out as having been rebuilt. If you have yet another intermediate target which depends on file-a.out, then Make will choose to not build the outer intermediate, claiming:

No recipe for 'file-a.out' and no prerequisites actually changed.
No need to remake target 'file-a.out'.

Answer 6

This is how I do it. First I always separate pre-requesits from the recipes. Then in this case a new target to do the recipe.

all: file-a.out file-b.out #first rule

file-a.out file-b.out: input.in

file-a.out file-b.out: dummy-a-and-b.out

.SECONDARY:dummy-a-and-b.out
dummy-a-and-b.out:
    echo creating: file-a.out file-b.out
    touch file-a.out file-b.out

The first time:
1. We try to build file-a.out, but dummy-a-and-b.out needs doing first so make runs the dummy-a-and-b.out recipe.
2. We try to build file-b.out, dummy-a-and-b.out is up to date.

The second and subsequent time:
1. We try to build file-a.out: make looks at prerequisites, normal prerequisites are up to date, secondary prerequisites are missing so ignored.
2. We try to build file-b.out: make looks at prerequisites, normal prerequisites are up to date, secondary prerequisites are missing so ignored.

Answer 7

As an extension to @deemer's answer, I have generalised it into a function.

sp :=
sp +=
inter = .inter.$(subst $(sp),_,$(subst /,_,$1))

ATOMIC=\
    $(eval s1=$(strip $1)) \
    $(eval target=$(call inter,$(s1))) \
    $(eval $(s1): $(target) ;) \
    $(eval .INTERMEDIATE: $(target) ) \
    $(target)

$(call ATOMIC, file-a.out file-b.out): input.in
    foo-bin input.in file-a.out file-b.out

Breakdown:

$(eval s1=$(strip $1))

Strip any leading/trailing whitespace from the first argument

$(eval target=$(call inter,$(s1)))

Create a variable target to a unique value to use as the intermediate target. For this case, the value will be .inter.file-a.out_file-b.out.

$(eval $(s1): $(target) ;)

Create an empty recipe for the outputs with the unique target as a depedency.

$(eval .INTERMEDIATE: $(target) ) 

Declare the unique target as an intermediate.

$(target)

Finish with a reference to the unique target so this function can be used directly in a recipe.

Also note the use of eval here is because eval expands to nothing so the full expansion of the function is just the unique target.

Must give credit to Atomic Rules in GNU Make which this function is inspired from.

Answer 8

To prevent parallel multiple execution of a rule with multiple outputs with make -jN, use .NOTPARALLEL: outputs. In your case :

.NOTPARALLEL: file-a.out file-b.out

file-a.out file-b.out: input.in
    foo-bin input.in file-a.out file-b.out