How to run Specflow feature/scenarios in cmd using vstest.console.exe?

Question

I have a feature file defined like

Feature: Portal Sign in
    In order to login to my portal
    As a User
    I have input user id and password

Scenario: User Login
    Given I have entered username and password 
    When The user clicks on Login button
    Then logged in successfully pop up message should be displayed

I tried this cmd:

vstest.console.exe tests.dll /TestCaseFilter:"FullyQualifiedName~PortalSigninFeature"

but it returns with no test matches for the given test case filter. Am I missing anything?

Also how do you suggest to run scenarios using cmd?

Edit:

In case of scenario Outline like the one below

Scenario Outline: Multiple User Login
    Given I have entered <username> and <password>
    When The user clicks on Login button
    Then logged in successfully pop up message should be displayed

Examples:
|username|password|
|User1   |pwd1    |
|User2   |pwd2    |

When I run the below cmd

SpecRun.exe run D:\SpecFlow\bin\Debug\MySpecFlowTests.dll --filter testpath:"Scenario:Multiple+User+Login"

Discovered tests appear as 0.

Should I change anything to make it work for scenario outline?

Always try a query with no filtering to make sure there is data and the items you are searching for are in the database. — jdweng
Do you actually have a feature and scenario like this? The example feature is a copy and paste from my answer in a previous question. — Greg Burghardt
Well @Greg Burghardt, sorry about that. Went for the lazy way and pasted the same snippet. One of my scenario is as above. To give you more info, I have created several feature files in my project and want to be able to run from cmd based on the feature file. I can provide a tag to each scenario/scenario outline under a feature file and achieve filtering but I wanted to know if there was specifically a way to filter based on feature file/scenarios — mithun

Dazed Dazed · Accepted Answer · 2020-10-12T19:03:10

When I ran this in jenkins from a command line I had the below. I am using SpecRun for a runner. This did require a RunSettings file.

cd "C:\Program Files (x86)\Microsoft Visual Studio\2019\Professional\Common7\IDE\CommonExtensions\Microsoft\TestWindow" vstest.console.exe /settings:E\MyPath\To\RunSettings\ E:\MyPathTo\Test.dll /ResultsDirectory:E:\MyPath\ /TestCaseFilter: "(TestCategory=MyTag)"

I created a folder called Jenkins and the RunSettings and .srprofile both resided in that folder.

RunSettings file pointed to the .srprofile

<RunSettings>
  <!-- Configurations for SpecFlow+ Runner -->
  <SpecRun>
   <Profile>Jenkins/MyTest.srprofile</Profile>
   <GenerateSpecRunTrait>false</GenerateSpecRunTrait>
  <GenerateFeatureTrait>false</GenerateFeatureTrait>
  </SpecRun>
</RunSettings>

If you are using SpecRun as a runner, you can also run this from a cmd line. Update the runner version accordingly.

cd E:\Path\to\packages\SpecRun.Runner.3.3.*\tools\net461
SpecRun.exe run PathTo/My.srprofile --baseFolder E:\Path\To\bin\Debug --filter "@TagL" --log specrun.log

To run by a Feature Name, you would use:

SpecRun.exe run D:\Path\Jenkins\My.srprofile --baseFolder D:\Path\bin\Debug --filter testpath:"Feature:MyFeature*" --log specrun.log

wild card will match anything starting with "MyFeature".

https://docs.specflow.org/projects/specflow-runner/en/latest/Profile/Filter.html

How to run Specflow feature/scenarios in cmd using vstest.console.exe?

1 Answers