Automake: how to handle global and local 'make check' effectively?

Question

In a larger project, I have set up ./tests/Makefile.am to run a number of tests when I call make check. The file global_wrapper.c contains the setup / breakdown code, and it calls test functions implemented in several subdirectories.

TESTS = global_test

check_PROGRAMS = global_test

global_test_SOURCES = global_wrapper.c foo/foo_test.c bar/bar_test.c

Works great. But the tests take a long time, so I would like to be able to optionally execute only tests from a single subdir. This is how I did it at first.

I added the subdirectories:

SUBDIRS = foo bar

In the subdirectories, I added local wrappers and Makefile.am's:

TESTS = foo_test

check_PROGRAMS = foo_test

# the foo_test.c here is of course the same as in the global Makefile.am
foo_test_SOURCES = foo_wrapper.c foo_test.c

This, too, works great - when I call make check in the subdirectory foo, only the foo tests are executed.

However, when I now call make check in ./tests, all tests are executed twice. Once through global_test, and once through the local test programs.

If I omit the SUBDIRS statement in the global Makefile.am, the subdirectory makefiles don't get build. If I omit TESTS from the local Makefile.am's, make check doesn't do anything for the local directories.

I'm not that familiar with automake, but I am pretty sure there is some way to solve this dilemma. Can anybody here give me a hint?

Answer 1

Break your tests up. In your tests/Makefile.am do:

TESTS = foo_test bar_test

and build foo_test bar_test appropriately with something like

foo_test_SOURCES = foo/foo_wrapper.c foo/foo_test.c
bar_test_SOURCES = bar/bar_wrapper.c bar/bar_test.c

Now, if you do a raw 'make check', both tests will be run. If you only want to run one test, you can do that with 'make check TESTS=foo_test' or 'make check TESTS=bar_test' and only the appropriate test will run. Typically, the Makefile.am lists all the tests that will be run by default in TESTS and the user selects alternate tests at make-time. Naturally, if you are running the tests a lot, you can 'export TESTS=foo_test' in your shell session and then only type 'make check'.

Answer 2

Can't you remove from "global_test" any test that is already executed in a subdirectory? (Just so they simply don't get executed twice.)

Answer 3

I think you could maybe overwrite the check rule at the top-level to define an environment variable:

check:
        DISABLE_SUBTESTS=1 make check-recursive

and then test DISABLE_SUBTESTS in your sub-directories to decide whether to actually run the tests or not.

(Personally, I'd rather arrange to work in the existing make check framework by concealing the output of my tests, rather than overwriting the produced rules like this.)