Target wildcards in a shell command

Question

I'm trying to create targets that depend on a list of files in a directory with the name of the target:

bin/%.out: src/%/ $(shell find src/%/* -type f -iname '*.cpp' -o -iname '*.hpp')
#   Build stuff here

However shell find src/%/* ...ends up expanding to shell find src//* .... At first I thought it's because I could only have 1 target wildcard but even after removing the src/%/dependency it ended up with this same problem.

Some more context: my directory contains a 'src' directory, which contains directories. Each sub-directory of 'src' I treat as a "project". When I build, all object files should go in out/src/projname. I'm attempting to use find to recursively get all source and header files per project. All binaries will go in bin/projname.out and as such, the main dependencies are .out files in bin that have the same names as their project name. (if src/abc exists, bin/abc.out is a dependency of all).

So for each sub-directory in src, all source files are compiled and the object files are moved to out/projname which eventually will be linked to bin/projname.out.

• src
  • proj0
    • cpp/hpp files
  • proj1
    • cpp/hpp files
• out
  • src
    • proj0
      • object files
    • proj1
      • object files
• bin
  • proj0.out
  • proj1.out

Currently getting the list of projects and list of output files as follows:

SRCS := $(shell find src/* -maxdepth 0 -type d)
SRCS_OUT := $(patsubst src/%,bin/%.out,$(SRCS))
...
all: out/ bin/ $(SRCS_OUT)

Projects can contain sub-directories and such, which is why I'm using find in the code at the very top.

Answer 1

I do not see how to do what you want with standard, portable, make features. But it is doable with advanced GNU make features:

define PROJECT_rule
CPPS_$(1) := $$(shell find src/$(1) -type f -iname '*.cpp')
HPPS_$(1) := $$(shell find src/$(1) -type f -iname '*.hpp')
OBJS_$(1) := $$(patsubst src/$(1)/%.cpp,out/src/$(1)/%.o,$$(CPPS_$(1)))

$$(OBJS_$(1)): out/src/$(1)/%.o: src/$(1)/%.cpp $$(HPPS_$(1))
    <compile recipe>

bin/$(1).out: $$(OBJS_$(1))
    <link recipe>
endef

PRJS := $(patsubst src/%,%,$(shell find src/* -maxdepth 0 -type d))

$(foreach p,$(PRJS),$(eval $(call PROJECT_rule,$(p))))

We first define a make variable (PROJECT_rule) that will be used as a template for any of your projects; in this template, $(1) will represent the name of your project. Then, we compute the list of projects (PRJS) and, finally, we iterate over the projects and process our template for each of them (foreach-eval-call).

Notes:

1. I assumed that each object file of a project depends on all header files of the same project. If it is not the case you will have to rework this a bit.

2. What is important to understand is that:

   • The PROJECT_rule variable is first processed by call to substitute $(1) with the project's name.

   • Then it is expanded recursively and completely (including the recipes) by eval. Recursively means that if we have:

     FOO = foo
     BAR = FOO
     BAZ = $($(BAR))
     QUX = $$(BAZ)
     

     $($(BAR)) is expanded as foo while $$(BAZ) is expanded as $(BAZ). Completely means that all parts of the PROJET_rule variable are expanded, even what is finally used as recipes (while when make parses a Makefile, it normally defers the expansion of the recipes to a later stage).

   • The result will be instantiated as regular make constructs, that is, it will be expanded once more as regular make constructs (not the recipes), before being evaluated.

3. So, as there will be two expansions of PROJECT_rule, some $ signs must be escaped ($$). Please remember this when writing your own <compile recipe> and <link recipe>. If you want to use the $@ or $^ automatic variables, for instance, do not forget to write $$@. Example: if your compile recipe is:

   $(CPP) $(CPPFLAGS) $(INCLUDES) -c $< -o $@
   

   write:

   $$(CPP) $$(CPPFLAGS) $$(INCLUDES) -c $$< -o $$@
   

   in the definition of PROJECT_rule. The first expansion will transform it in:

   $(CPP) $(CPPFLAGS) $(INCLUDES) -c $< -o $@
   

   While, if you write:

   $(CPP) $(CPPFLAGS) $(INCLUDES) -c $< -o $@
   

   the first expansion will transform it in something like:

   g++ -O3 -Iincludes/foobar -c  -o
   

   Note that, unless CPP, CPPFLAGS or INCLUDES have specific target-dependent definitions, you can also write:

   $(CPP) $(CPPFLAGS) $(INCLUDES) -c $$< -o $$@
   

   because the first expansion will transform it in something like:

   g++ -O3 -Iincludes/foobar -c $< -o $@
   

   which is also correct.

4. A bit more detailed explanation: for each project (e.g. proj0), foreach will produce:

   $(eval $(call PROJECT_rule,proj0))
   

   call will substitute $(1) with proj0 in the PROJECT_rule definition, such that the parameter of eval will be:

   CPPS_proj0 := $$(shell find src/proj0 -type f -iname '*.cpp')
   HPPS_proj0 := $$(shell find src/proj0 -type f -iname '*.hpp')
   OBJS_proj0 := $$(patsubst src/proj0/%.cpp,out/src/proj0/%.o,$$(CPPS_proj0))
   
   $$(OBJS_proj0): out/src/proj0/%.o: src/proj0/%.cpp $$(HPPS_proj0)
       <compile recipe>
   
   bin/proj0.out: $$(OBJS_proj0)
       <link recipe>
   

   eval will expand it recursively and completely, leading to:

   CPPS_proj0 := $(shell find src/proj0 -type f -iname '*.cpp')
   HPPS_proj0 := $(shell find src/proj0 -type f -iname '*.hpp')
   OBJS_proj0 := $(patsubst src/proj0/%.cpp,out/src/proj0/%.o,$(CPPS_proj0))
   
   $(OBJS_proj0): out/src/proj0/%.o: src/proj0/%.cpp $(HPPS_proj0)
       <expanded compile recipe>
   
   bin/proj0.out: $(OBJS_proj0)
       <expanded link recipe>
   

   (each $$ becomes $, even in the recipes). eval will then instantiate the result as regular make constructs and they will be parsed as regular make syntax. That is, the (simple) variables will be expanded immediately, the targets and prerequisites too, but not the recipes. The recipes will be expanded in a second phase, just before being passed to the shell and executed, if they are.

5. The compile rule is a static pattern rule. If you do not know this already you can read this section of the GNU make documentation.

6. Exercise: in the definition of PROJECT_rule you could replace $$(shell... by $(shell... but not $$(patsubst... by $(patsubst.... Why?