cmake test: Every test is run on each ctest

Question

Strangely, when I build my tests and run them, every test I defined (boost's BOOST_AUTO_TEST_CASE()) will be run on each cmake defined test (cmake's add_test()). I am pretty sure that I did something wrong in the configuration but I cannot for the life of me figure out what it is.

root CMakeLists.txt:

cmake_minimum_required(VERSION 2.8)
project("project")
-- some library findings and other configs --
enable_testing()
subdirs(test)
subdirs(src)

test CMakeLists.txt:

add_test(NAME hash_structors COMMAND projectTest)
add_test(NAME hash_makeHash COMMAND projectTest)
add_test(NAME hash_tree_size_compare COMMAND projectTest)
add_test(NAME hash_tree_size_compare_random COMMAND projectTest)
add_test(NAME hash_tree_compare COMMAND projectTest)
add_test(NAME directory_manual COMMAND projectTest)

include_directories(../include)
add_executable(projectTest testMain.cpp
                       ../src/hash.cpp
                       ../src/hash_tree.cpp
                       ../src/directory.cpp)
target_link_libraries(projectTest ${Boost_LIBRARIES}
                              ${CRYPTO++_LIBRARIES})

testMain.cpp:

#define BOOST_TEST_DYN_LINK
#define BOOST_TEST_MODULE "project tests"
#include <boost/test/unit_test.hpp>
#include "test_hash.hpp"
#include "test_hash_tree.hpp"
#include "test_directory.hpp"

each 'test_NAME.hpp' then contains tests similar to this:

#include "hash.hpp"
BOOST_AUTO_TEST_CASE(hash_structors)
{
  Hash hash;
  BOOST_CHECK_EQUAL("", hash.getHash());
}
BOOST_AUTO_TEST_CASE(hash_makeHash)
{
  Hash hash = Hash("test");
  BOOST_TEST_MESSAGE(hash.getHash());
  // precomputed hash value for "test"
  BOOST_CHECK_EQUAL("7ab383fc29d81f8d0d68e87c69bae5f1f18266d730c48b1d", hash.getHash());
}

ComicSansMS ComicSansMS · Accepted Answer · 2014-10-14T09:26:31

The add_test command is not as smart as you might expect. In particular, it knows nothing about how to configure a test executable to only executing a certain set of tests.

What you are telling CMake now is basically run the full set of tests in projectTest 6 times under different names. You have two options to resolve this.

Either restrict the test command to only execute the correct tests. For Boost Test, this can be done easily with the -t command line parameter:

add_test(NAME hash_structors COMMAND projectTest -t */hash_structors)

The other option is to split the tests at the source level:

add_executable(TestHash testHash.cpp ../src/hash.cpp)
add_test(NAME hash_tests COMMAND TestHash)

add_executable(TestHashTree testHashTree.cpp ../src/hash_tree.cpp)
add_test(NAME hashtree_tests COMMAND TestHashTree)

I personally prefer the second approach, as it is more structured and less tempting to write big unit tests that have too many dependencies on different components. But that's just a personal preference.

cmake test: Every test is run on each ctest

1 Answers