2
votes

I am running an azure pipeline as follows. I have set the timeout in minutes in 3 places: the job, the pool and the task however the step run acceptance tests always timeout after 60 minutes with the error message:

An error occurred while provisioning resources (Error Type: Timeout). The operation was canceled.

Otherwise, everything in the pipeline runs ok.

stages: 
- stage: Run_Acceptance_Tests
  jobs:
  - job: Run_Acceptance_Tests
    timeoutInMinutes: 180
    pool:
      name: Hosted VS2017
      timeoutInMinutes: 180
      demands:
      - msbuild
      - visualstudio
      - vstest

    steps:
    - task: NuGetToolInstaller@0
      displayName: 'Use NuGet 4.4.1'
      inputs:
        versionSpec: 4.4.1

    - task: NuGetCommand@2
      displayName: 'NuGet restore'
      inputs:
        restoreSolution: '$(solution)'

    - task: VSBuild@1
      displayName: 'Build solution'
      inputs:
        solution: '$(solution)'
        msbuildArgs: '/p:SkipInvalidConfigurations=true /p:ExcludeApp_Data=true /p:AutoParameterizationWebConfigConnectionStrings=false /p:MarkWebConfigAssistFilesAsExclude=false /p:ProfileTransformWebConfigEnabled=false /p:TransformWebConfigEnabled=false'
        platform: '$(BuildPlatform)'
        configuration: '$(BuildConfiguration)'

    - task: FileTransform@2
      displayName: 'Transform settings for Acceptance tests'
      inputs:
        folderPath: '$(Build.SourcesDirectory)'
        xmlTransformationRules: '-transform **\Selenium.Tests\App.Test.config -xml **\Selenium.Tests\bin\**\Selenium.Tests.dll.config'    

    - task: VSTest@2
      displayName: 'Run ExcelWraps Acceptance tests'
      timeoutInMinutes: 180
      inputs:
        testAssemblyVer2: |
         **\Selenium.Tests.dll
         !**\obj\**
        searchFolder: '$(Build.SourcesDirectory)'
        pathtoCustomTestAdapters: '$(build.sourcesdirectory)\src\packages\NUnit3TestAdapter.3.10.0\build\net35\'
        runInParallel: false
        codeCoverageEnabled: false
        platform: '$(BuildPlatform)'
        configuration: '$(BuildConfiguration)'
        diagnosticsEnabled: false
        uiTests: true
        minimumExpectedTests: 10    

I have read and reread the docs and seem to have done everything they ask. All these questions seem to have been answered by placing the timeout in one of the locations I mentioned above: - https://developercommunity.visualstudio.com/content/problem/375541/60-minutes-time-limit-per-build.html - https://developercommunity.visualstudio.com/content/problem/567714/azure-devops-timeoutinminutes-is-being-ignored-in.html - https://developercommunity.visualstudio.com/content/problem/670377/timeout-not-respected-in-multi-stage-pipeline.html - https://github.com/Microsoft/azure-pipelines-yaml/issues/48 - https://github.com/Microsoft/azure-pipelines-agent/issues/1849

How can I increase the timeout of the testing step?

1
Is your project is private or public? - Shayki Abramczyk
the project is private - johnstaveley
So this is the reason, see my answer :) - Shayki Abramczyk

1 Answers

3
votes

In private projects, the maximum timeout that Microsoft gives in hosted agents is 60 minutes.

See the docs here:

To avoid taking up resources when your job is hung or waiting too long, it's a good idea to set a limit on how long your job is allowed to run. Use the job timeout setting to specify the limit in minutes for running the job. Setting the value to zero means that the job can run:

  • Forever on self-hosted agents
  • For 360 minutes (6 hours) on Microsoft-hosted agents with a public project and public repository
  • For 60 minutes on Microsoft-hosted agents with a private project or private repository