How to get a Chutzpah Test Adapter to run my QUnit tests on an Azure DevOps pipeline build?

Question

It just works on my machine - Prior to using Azure DevOps, I simply installed this Chutzpah VSIX 'Test Adapter' add-on in order to run my QUnit tests. My test.js uses Chutzpah's reference syntax.. so Chutzpah will surface the test in the IDE's Test Explorer and I could run them like any other.

Azure Pipeline Docs - says right here that their VSTest@2 task runner supports running Chutzpah unit tests:

Use this task in a build or release pipeline to run unit and functional tests (Selenium, Appium, Coded UI test, and more) using the Visual Studio Test Runner. Other than MSTest-based tests, test frameworks that have a Visual Studio test adapter, such as xUnit, NUnit, Chutzpah, can also be executed.

But my QUnit / Chutzpah tests aren't discovered - if I upload this solution stack to an Azure DevOps repo, and use the default YAML for ASP.NET.. it discovers several unit tests and runs them successfully but none of my javascript/QUnit tests

I'm guessing that my Chutzpah VSIX add-ons are not present on the Azure DevOps build agent machine and I need to do something to make it work - but, what? I'm not currently using the Chutzpah nuget pkg - should I?

Update...

According to this Art of Simplicity blog post I do need to install the Chutzpah NuGet package.

This NuGet package contains all required DLL’s and tooling to be able to run your JavaScript tests. By using this approach you don’t need to install the Chutzpah test adapter on your build server or specify the location of your test adapter. The TFS Build testing tools will automatically detect the required DLL’s and load the test adapter for you.

I'm new to the world of Azure DevOps (and it's predecessors) and didn't adjust my Google-fu to incorporate terms like 'TFS' - so maybe I need to start searching based on that.

Update... Per the blog post above, I've now added a chutzpah.config file in the root of my web project (it's not in the root of my repo .. my repo has several solutions in it nested at various levels). Still no luck getting it to build on the pipeline server

bkwdesign bkwdesign · Accepted Answer · 2018-11-29T05:22:31

The key for me was that I had to add the Chutzpah NuGet package to my web project, and then tell the build agent / test runner where to find that Chutzpah 'stuff' (see pathtoCustomTestAdapters switch in the yaml below) more here

Here's my .yml file that worked for me:

- task: VSTest@2
  inputs:
    platform: '$(buildPlatform)'
    configuration: '$(buildConfiguration)'
    testAssemblyVer2: '$(Build.SourcesDirectory)/MyApp/trunk/MyWebAppProject/Scripts/tests/test**.js'
    pathtoCustomTestAdapters: '$(Build.SourcesDirectory)/MyApp/trunk/packages/Chutzpah.4.4.3/tools/'

In spite of being told in some places to add a .runsettings file, this did not seem to work for my Azure DevOps build agent.. it actually confused it!

FWIW - This is the chutzpah.json I put in the root of my web project (which is not the root of my repo)

{
  "Framework": "qunit",
  "FrameworkVersion": "1",
  "TestHarnessLocationMode": "TestFileAdjacent", 
  "References": [
    {
      "Path": "Scripts/qunit.js",
      "IsTestFrameworkFile": true
    },
    {
      "Path": "Content/qunit.css",
      "IsTestFrameworkFile": true
    }
  ],
  "Tests": [
    {
      "Path": "Scripts/tests/",
      "ExpandReferenceComments": true,
      "Excludes": [ "fake_razor.js", "fake_razor_search.js","MyCorp.Spa.Routing.Callbacks.js" ]
    }
  ]
}

I don't know if adding this .json was critical to my success or not.

How to get a Chutzpah Test Adapter to run my QUnit tests on an Azure DevOps pipeline build?

1 Answers