CMake/CTest - Where was defined test XYZ?

Question

After building a CMake/CTest setup with many tests I see a growing problem. If ctest -R test_some_side_corner_item_XYZ fails, how can I systematically track back to the place where somebody added the test test_some_side_corner_item_XYZ?

I understand that there is the problem that we can add macros on top of add_test() which makes it a bit harder to make any clear answer - but still.

It appears cmake does not have any obvious ways to achieve this - and the same seems for ctest.

track back you mean like track back in time? So can't you just grep -r test_some_side_corner_item_XYZ your project? that we can add macros on top of add_test() - so add a macro on top of add_test that logs all of it's uses. And then you'll see. — KamilCuk

squareskittles squareskittles · Accepted Answer · 2020-03-26T12:07:07

You can use the ctest command line options to find the exact line in your CMakeLists.txt hierarchy where the add_test() call was made. We can use the --show-only=json-v1 option to display JSON-formatted meta-data about a test:

ctest -R test_some_side_corner_item_XYZ --show-only=json-v1

An example of what this prints would be:

{
  "backtraceGraph" :
  {
    "commands" :
    [
      "add_test"
    ],
    "files" :
    [
      "C:/workspace/myproject/CMakeLists.txt"
    ],
    "nodes" :
    [
      {
        "file" : 0
      },
      {
        "command" : 0,
        "file" : 0,
        "line" : 34,
        "parent" : 0
      }
    ]
  },
...

This lists the CMakeLists.txt file where add_test() was called for this test, and the line number ("line" : 34) where it was called.

From the CMake documentation, the --show-only option will not actually run the test, but will only display its information:

-N,--show-only[=<format>]

Disable actual execution of tests.

This option tells CTest to list the tests that would be run but not actually run them. Useful in conjunction with the -R and -E options.

Note, the -R option is a regex to match the test(s) you want, so to get an exact match, you can anchor the test name with ^ and $:

ctest -R ^test_some_side_corner_item_XYZ$ --show-only=json-v1

CMake/CTest - Where was defined test XYZ?

1 Answers