Specflow Contextual Assertions

Question

We use Specflow and currently, most of our assertions are done using Fluent Assertions, ie look something like:
myval.Should().NotBe(null)

All is working well, however there are some cases in which we want the same code to sometimes Assert Inconclusive, and sometimes not.

For example, let us assume we have a step which calls a method to install the AUT.
The test which asserts that the AUT can be installed correctly will use this method as part of a WHEN/THEN step.
In this instance, if installation fails, we want to proceed as normal and fail the test.
GIVEN The AUT is NOT installed WHEN I install the AUT

However, for all other tests, the same method may be called as part of a GIVEN step or as part of a BeforeScenario / BeforeFeature hook, and in that instance, if the installation fails, the test should be failed as inconclusive, because it was not the test itself that failed, but the Setup stage. GIVEN the AUT is installed WHEN I perform function X of the AUT

So in this example, let us assume that the step definitions for WHEN I install the AUT in the first test, and GIVEN the AUT is installed in the second test simply call a helper method with something like AppFacade.Install()

Of course we can litter AppFacade.Install() and all our other helper methods with conditionals etc, but I was wondering if anyone out there has a more elegant solution - ie way to extend an assertion engine so that it automatically applies this logic - I see no barriers on the Specflow side to achieving this insofar as you can examine the Specflow context objects to detect what kind of step you are on.
Ideally we would like to stick with Fluent Assertions, but if this overly complicates things, we would be willing to reconsider.

Answer 1

I was going to suggest you were not asking the right question, since you are implying that when a Specflow step execution fails the test should be inconclusive, but you already realized it.

So to answer the actual question you have:

Is it possible to throw an exception which can mark the test as inconclusive given the above logic?

Yes, you can use the after AfterStep hook (see Specflow hooks here) to check the is there was any exception thrown and execute Assert.Inconclusive() if you're using Nunit or any equivalent method, like Specflow's ScenarioContext.Current.Pending() to not be attached to the testing framework installed, the only caveat is that ScenarioContext.Current.Pending() does not clear ScenarioContext.TestStatus, so your test runner will mark it as red and you'll have to use reflection to set the TestStatus property as described here.

It would look something like this:

[AfterStep]
public void Check()
{
    if (ScenarioContext.Current.TestError is AssertionException &&
        ScenarioContext.Current.CurrentScenarioBlock == ScenarioBlock.Given)
        Assert.Inconclusive(ScenarioContext.Current.TestError.Message);
}

---

Edit

Here is a demo I created.

Answer 2

Seems to me that you have different requirements for the step when it is a given vs a when or then. Specflow already has something built in to handle this, the fact that the engine will call the method tagged as given for given steps and not for when and then steps.

It feels to me like the simplest solution is to not use the same code for the step when its a given and a when, but have dedicated code for the given step which does what you want differently to the when and then steps.

I'm not sure that you are going to be able to make it bend to your desires and be able to reuse the same steps

Answer 3

Not sure if this will help but you can inherit the TechTalk.SpecFlow.Steps class on your steps class like so;

public sealed class MyFeatureSteps : Steps

What does this allow you to do? Well you can do something like this;

[Given(@"Everything is installed")]
public void GivenEverythingIsInstalled()
{
  // Perform Installation with no assertion
  // You can use this step just to do the setup
  // No assertions literally just perform all the actions required to install 
}

[Then(@"its all installed correctly")]
public void ThenItsAllInstalledCorrectly()
{
  // Because you have inherited the TechTalk.SpecFlow.Steps class
  Given("Everything is installed");
  Assert.IsTrue(CorrectInstallCheckMehtod(), "It should have installed");
}

Basically you can make steps as granular as you want now and just put assertions where you need them, and you can reference other steps within each other.

Take note while you can reference a GIVEN step within a WHEN (or THEN) step, if the last thing you do within the step is call another and it is different to its parent I.e a WHEN step references a GIVEN step if you try to use an AND in your feature file its going to be a GIVEN AND. I hope that makes sense

I hope this will help you. I think you should avoid not failing tests even if that's not what you are specifically testing because it still highlights some broken functionality. Alternatively put an assert after the install that checks its installed this will cause the test to fail before it even starts, again this really is not a bad thing.