CTest Error on MacOS: Test Executable not found

Question

I am working on a CMake project. For CI, I have decided to use Azure Pipelines. However I am facing a small problem on MacOS in the testing phase. The problem is that MacOS fails to find the test executable, even when it is there.

Earlier my project wasn't getting built up properly. But now the entire pipeline runs successfully (thanks to the Stack Overflow community) except for the minor glitch on MacOS. I have updated my question and now it tells what problem was and how I fixed it so that it may be helpful to others like me who are new to the world of CI.

Edit 1:

Earlier my CMake task wasn't triggering the build process. This was because I was providing not providing any command-line arguments to CMake. All I did was this:

- task: CMake@1
  displayName: Generate CMake Cache
  inputs:
    workingDirectory: build

I assumed that the CMake task would drive the build process automatically, as it wasn't specified clearly in the documentation what that task actually does. This did nothing apart from printing the CMake usage. Then I found out that we have to run the CMake task twice (once for project configuration and then for the actual build) using appropriate command-line arguments.

Edit 2:

This is my updated AzurePipelines.yml file

Azure Pipelines CI

stages:
- stage: Build
  displayName: Build

  jobs:
  - job: RunCMakeTask
    displayName: Run CMake Task

    strategy:
      matrix:
        LinuxDebug:
          OS: 'Linux'
          imageName: 'ubuntu-latest'
          BuildConfiguration: 'Debug'
        LinuxRelease:
          OS: 'Linux'
          imageName: 'ubuntu-latest'
          BuildConfiguration: 'RelWithDebInfo'
        MacOSDebug:
          OS: 'MacOS'
          imageName: 'macos-latest'
          BuildConfiguration: 'Debug'
        MacOSRelease:
          OS: 'MacOS'
          imageName: 'macos-latest'
          BuildConfiguration: 'RelWithDebInfo'
        WindowsDebug:
          OS: 'Windows'
          imageName: 'windows-latest'
          BuildConfiguration: 'Debug'
        WindowsRelease:
          OS: 'Windows'
          imageName: 'windows-latest'
          BuildConfiguration: 'RelWithDebInfo'

    pool:
      vmImage: $(imageName)

    steps:
    - script: mkdir $(BuildConfiguration)
      displayName: Create Build Directory
      workingDirectory: $(Build.SourcesDirectory)
    - task: CMake@1
      displayName: Generate CMake Cache
      inputs:
        workingDirectory: $(BuildConfiguration)
        cmakeArgs: '-DCMAKE_BUILD_TYPE=$(BuildConfiguration) ..'
    - task: CMake@1
      displayName: Run Build Process
      inputs:
        workingDirectory: $(BuildConfiguration)
        cmakeArgs: '--build . --config $(BuildConfiguration)'
    - task: PublishPipelineArtifact@1
      displayName: Publish Build Artifact
      inputs:
        targetPath: $(BuildConfiguration)
        artifactName: '$(OS)$(BuildConfiguration)'
    
- stage: Test   
  displayName: Test
  dependsOn: Build

  jobs:
  - job: RunCTestOnWindows
    displayName: Run CTest on Windows

    variables:
      OS: Windows

    strategy:
      matrix:
        Debug:
          BuildConfiguration: 'Debug'
        Release:
          BuildConfiguration: 'RelWithDebInfo'

    pool:
      vmImage: 'windows-latest'

    steps:
    - task: DownloadPipelineArtifact@2
      displayName: Download Build Artifact
      inputs:
        artifact: '$(OS)$(BuildConfiguration)'
        path: $(Build.SourcesDirectory)/$(BuildConfiguration)
    - script: ctest -C $(BuildConfiguration) --output-on-failure
      workingDirectory: $(BuildConfiguration)
      
  - job: RunCTestOnUnixBasedSystems
    displayName: Run CTest on Unix Based Systems

    strategy:
      matrix:
        LinuxDebug:
          OS: 'Linux'
          imageName: 'ubuntu-latest'
          BuildConfiguration: 'Debug'
        LinuxRelease:
          OS: 'Linux'
          imageName: 'ubuntu-latest'
          BuildConfiguration: 'RelWithDebInfo'
        MacOSDebug:
          OS: 'MacOS'
          imageName: 'macos-latest'
          BuildConfiguration: 'Debug'
        MacOSRelease:
          OS: 'MacOS'
          imageName: 'macos-latest'
          BuildConfiguration: 'RelWithDebInfo'

    steps:
    - task: DownloadPipelineArtifact@2
      displayName: Download Build Artifact
      inputs:
        artifact: '$(OS)$(BuildConfiguration)'
        path: $(Build.SourcesDirectory)/$(BuildConfiguration)
    - script: find $(BuildConfiguration)/Tests -type f -name "Test*" ! -name "*.*" ! -exec chmod u+rx {} \;
      displayName: Change File Permissions
    - script: ctest -C $(BuildConfiguration) --output-on-failure
      workingDirectory: $(BuildConfiguration)

The pipeline runs fine on Windows and Linux, but I am facing a small problem on MacOS. On MacOS, ctest fails to find the test executable even when it is there. (If there were any problem in my pipeline or my CMakeLists.txt file, it should also have failed on Windows and Linux)

Edit 3:

And in the Test stage on MacOS, I am getting the error:

Test project /home/vsts/work/1/s/Debug Start 1: StringOperations_CaseIgnore Could not find executable /Users/runner/work/1/s/Debug/Tests/StringOperations/TestStringOperations Looked in the following places: /Users/runner/work/1/s/Debug/Tests/StringOperations/TestStringOperations /Users/runner/work/1/s/Debug/Tests/StringOperations/TestStringOperations /Users/runner/work/1/s/Debug/Tests/StringOperations/Debug/TestStringOperations /Users/runner/work/1/s/Debug/Tests/StringOperations/Debug/TestStringOperations Debug//Users/runner/work/1/s/Debug/Tests/StringOperations/TestStringOperations Debug//Users/runner/work/1/s/Debug/Tests/StringOperations/TestStringOperations Users/runner/work/1/s/Debug/Tests/StringOperations/TestStringOperations Users/runner/work/1/s/Debug/Tests/StringOperations/TestStringOperations Users/runner/work/1/s/Debug/Tests/StringOperations/Debug/TestStringOperations Users/runner/work/1/s/Debug/Tests/StringOperations/Debug/TestStringOperations Debug/Users/runner/work/1/s/Debug/Tests/StringOperations/TestStringOperations Debug/Users/runner/work/1/s/Debug/Tests/StringOperations/TestStringOperations 1/1 Test #1: StringOperations_CaseIgnore ......***Not Run 0.00 sec

0% tests passed, 1 tests failed out of 1

Total Test time (real) = 0.00 sec

The following tests FAILED: 1 - StringOperations_CaseIgnore (Not Run) Unable to find executable: /Users/runner/work/1/s/Debug/Tests/StringOperations/TestStringOperations Errors while running CTest

Edit 4:

I have tried to check whether the test executable is actually there or not by using:

ls -l Debug/Tests/StringOperations

And here's the output:

drwxr-xr-x 3 vsts docker    4096 Aug  6 15:05 CMakeFiles
-rw-r--r-- 1 vsts docker    1208 Aug  6 15:05 cmake_install.cmake
-rw-r--r-- 1 vsts docker     642 Aug  6 15:05 CTestTestfile.cmake
-rw-r--r-- 1 vsts docker    9838 Aug  6 15:05 Makefile
-rwxr--r-- 1 vsts docker 1715072 Aug  6 15:05 TestStringOperations

This confirms that the test executable (TestStringOperations) is there at the same place where it was for Windows and Linux, but still the process fails.

Here is the CMakeLists.txt for this executable should you need it:

Set(SRC StringOperations.cpp)

Add_Executable(TestStringOperations ${SRC})

Target_Include_Directories(TestStringOperations PUBLIC
                           ${HEADER_PATH}/StringOperations
)

Target_Link_Libraries(TestStringOperations
                      PRIVATE ${GTEST_LIBS}
                      PRIVATE StringOperations
)

Add_Test(NAME StringOperations_CaseIgnore COMMAND TestStringOperations)

I have tried looking for help on this issue on Stack Overflow and some other sites, but their solutions aren't benefitting me.

For example: CTest can not find executable file and CMake: How to specify directory where ctest should look for executables?

If you need more info, here is my project on GitHub. You can also refer to the pipeline logs at dev.azure.com.

Can you please help me in fixing this problem? Any suggestions regarding the overall implementation of this file are also welcome.

Obviously this is because the Cmake task is not executed correctly, the build folder is not generated. We need to check the cmake task to make it build correctly. You could check if you could build it correctly on your local machine, and share the cmake build log in your question. — Leo Liu-MSFT
Azure pipelines stages do not share local state, so the build directory does not exist in the Test stage. To transfer files between stages, you need to use artifacts. However, I would suggest to avoid using multiple stages until you are more familiar with Azure pipelines, and just run your build+test in a single stage.Note that unlike most CI services, AZP will continue running commands after a non-zero command exit unless you use set -e (for bash) or similar checks. — Isaiah Norton
Please note that the above comments refer to the time when my build process wasn't starting. Now I have a different problem altogether. — user13088241

DreadedFrost DreadedFrost · Accepted Answer · 2020-08-05T15:06:03

Try running a Publish Pipeline Artifact task after your build. As eluded to in the comments this will publish your build contents and will allow it to be shared across stages.

After doing this then you will also be able to see it as a published artifact in the UI for the pipeline.

# Publish pipeline artifacts
# Publish (upload) a file or directory as a named artifact for the current run
- task: PublishPipelineArtifact@1
  inputs:
    targetPath: '$(Pipeline.Workspace)' 
    artifactName: # 'drop'

Also another gut check that can be done at the end of the stage is to include a powershell script to see the contents of the directory you are working in:

-powershell: Get-ChildItem -Path 'Insert root path' -recurse

CTest Error on MacOS: Test Executable not found

Azure Pipelines CI

1 Answers