When running multiple tags with NUnit Console Runner and SpecFlow I get incorrect results

Question

This is a follow up to my earlier questions on setting up tags: Can I use tags in SpecFlow to determine the right environment to use? and setting up variables from those tags: How to set up a URL variable to be used in NUnit/SpecFlow framework

I've set up some variables to aid in populating my NUnit Tests, but I find that when the NUnit runner finds the test that fits the first tag the test runs it with the settings of the second tag. Since the tags are important to me to not only know what test to run, but what variables to use, this is causing me problems.

So if I have the following tags:

@first

@first @second

@second

If I run @second everything is fine. If I run @first I get any scenario that has only @first fine, but when it comes to scenarios where I have both @first @second the scenario is run, because @first is there, however, it uses the parameters for @second. Since I am running the DLL through the NUnit-Console and the Tests are written through SpecFlow I am not sure where the issue may lie.

Does anyone have advice on setting up tests to run like this?

perfectionist perfectionist · Accepted Answer · 2012-01-26T20:28:58

You've not been very specific, but it sounds like you have a feature file like this:

@first
Scenario: A - Something Specific happens under the first settings
    Given ...etc...

@second
Scenario: B - Something Specific happens under the second settings
    Given ...etc...

@first @second
Scenario: C - Something general happens under the first and second settings
    Given ...etc...

It looks like you are selecting tests to run in NUnit by running all the tests in the "first" category.

If you set up event definitions like this:

[BeforeFeature("first")] 
public static string FirstSettings() 
{ ... }

[BeforeFeature("second")] 
public static string SecondSettings() 
{ ... }

If you execute scenario C then FirstSettings() and SecondSettings() will be executed before it. This is regardless of whether you used the @second category to select the test to run under NUnit.

This is almost certainly the reason that you are seeing the second settings applied to your test with both tags - I expect the second settings overwrite the first ones, right?

My only advice for setting up tests like this, is that binding events and so on to specific tags can be useful but should be used as little as possible. Instead make your individual step definitions reusable, and set up your test environment, where possible, with Given steps.

When running multiple tags with NUnit Console Runner and SpecFlow I get incorrect results

1 Answers