Running SpecFlow Scenario Outlines from Test Plans in Azure DevOps (VSTS)

2
votes

I'm using SpecFlow scenario outlines to generate multiple tests based off a single test case in Azure DevOps. Then I am using azure devops release pipeline to deploy the tests and run them using an automated test plan.

The problem I am having is a SpecFlow Scenario Outline represents 1 test case in Azure DevOps, but I can only link 1 of the 4 generated Test Explorer tests to a test case via "Associate to Test Case" to Azure DevOps. When I deploy the pipeline it only runs the single test that is associated, not every test in the Scenario Outline. How can I get it to run all the tests in Scenario Outline?

Example:

I can only right click on AddingUserInformationScenarioOutline("1",null) and link that specific test to VSTS, I cannot link the higher grouping AddingUserInformationScenarioOutline(4). Running the test plan will result in just test AddingUserInformationScenarioOutline("1",null) being run.

SpecFlow Scenario and Scenario Outline

1
azure-devopsspecflow

1 Answers

2
votes

You cannot associate more than one test method with a test case, but you can associate a test method with more than one test case. Please see Associate your test for details.

So, in your scenario you can create multiple test cases and associate each test method with them...

Another way is configuring your build process to execute tests using SpecFlow+ Runner:

SpecFlow+ runner

Smarter integration test runner for SpecFlow

SpecFlow+ Runner is a dedicated test execution engine providing better reporting, faster (parallel) test execution and better VS/TFS integration

Highly recommend to use specflow+ runner to run specflow tests via TFS/Azure DevOps (VSTS).

To configure your build process to execute tests using SpecFlow+ Runner:

  1. Open TFS/VSTS and switch to your build definition.
  2. Click on Add build step to add a new step to the build definition. Click on Test in the list of categories and click on Add next to "Visual Studio Test".

  3. Configure the build step as follows:

    Enter the path to your Test Assembly

    If you are using both MSTest and SpecFlow+ Runner, enter the path to NuGet package folder in the Path to Custom Test Adapters field. If you do not enter the path here, only your MSTest tests will be executed as TFS cannot find the SpecFlow+Runner test adapter.

    Enter the name of your .srprofile file in the Run Settings File field if the name is not TFS.srprofile or default.srprofile (see above).

  4. Save your changes.

More details please refer this tutorial: SpecFlow+ And Build Servers