Trying to understand Specflow relation to unit test frameworks and BDD

Question

Im looking into a BDD solution like SpecFlow and going through various examples and I see references to other TDD frameworks like MsTest and NUnit which Im familiar with. I understand the value of what Specflow and BDD is providing. I read somewhere that Specflow and BDD "wraps" your unit tests. So, with Specflow, does the "step definitions" serve the same purpose as something like MsTest or Nunit would, and these other frameworks are just options to use instead of the Step definitions?

Answer 1

If you go back to the origins of BDD, you'll find out that the first ever tool - JBehave - was originally a replacement for JUnit. Annotations weren't a thing at the time, so JUnit used to look for things starting with the word "test":

testDatabase

Of course, that's pretty meaningless. So JBehave did the same thing, but starting with the word "should":

shouldPersistRecordsUsingHibernate

So now we can see what our class should do, and talk about its behaviour using plain English. "Behaviour" is a more useful word than "test", here.

After a while, and a few conversations between Dan North and Chris Matts, they worked out that what Dan was doing with classes was the same thing that Chris (an analyst at the time) did with whole systems, and so the scenario-running* component was born. Dan ported JBehave to RBehave, which became RSpec's scenario runner, which became SpecFlow and Cucumber and all the rest of them. And of course, JUnit these days uses annotations, so we can start things with the word "should" anyway, and NUnit's pretty much JUnit for .NET.

The point is that you can describe any level of code with BDD. I use BDD for my class-level examples as well as my system-level scenarios.

For the class-level examples, the only audience is technical. It's enough to write comments about how the class behaves in something like NUnit or JUnit. Some frameworks like MSpec or RSpec have other ways of capturing those descriptions, but the audience is still technical. It's still BDD. With class-level examples we mock out dependencies so that we're only looking at one aspect of behaviour at a time. This helps drive good design, same as TDD.

For system-level examples, though, there's another audience; the non-technical stakeholders. Most of them can understand readable code just fine, so you can make a DSL and still use NUnit here too. However, there's benefit to being able to capture natural language a little more directly, without having to worry about making it run. That's where SpecFlow and the like come into play.

The overhead of parsing natural language and matching it to steps isn't small. It wouldn't be worth it at a unit / class level, where behaviours are separated by the Single Responsibility Principle, encapsulation, and other aspects of good design. Natural languages aren't as easy to refactor as code, either.

So we save the natural language scenarios for the outermost layers, where we only want a few examples of how a system behaves. We then end up with a few examples of end-to-end behaviour, more integration / module / library-level scenarios, and still more unit tests (or examples). This is commonly referred to as the "testing pyramid".

For instance, if we want to validate a form, it's enough to have a couple of examples of how our application helps users fill in the form when they make a mistake. We might have a couple of integration tests that check that we validate at both the client and server-side (if that's important to us, for instance). And then we'll have a bunch of unit tests around the actual validation classes, covering all the different possibilities.

To answer your question in shorthand:

• SpecFlow, Cucumber, JBehave and other natural-language frameworks are generally used only for system-level scenarios, often automated against the UI.
• You don't have to use the frameworks for those; it's possible to do it using a DSL in NUnit (or JUnit, etc.).
• Using natural-language framworks at a class level is overkill. Use NUnit here.
• BDDers often have one layer of scenarios running at the system level, and a ton of class-level examples (unit tests) sitting beneath them. This is the "wrapping" that you're talking about.
• There may be intermediate layers too.

*The words "scenario" and "example" are pretty much synonyms, meaning much the same thing. Traditionally we've used "scenario" to refer only to end-to-end examples, with "example" or "unit test" being used at a class level.

The single most important thing about BDD, though, is to make sure you talk through those aspects of behaviour with someone, even if it's in retrospect or with a rubber duck. BDD is much more about the conversation than it is about the tools.

Answer 2

To be short "step definitions" don't serve the same purpose as something like MsTest or Nunit. Actually, I could write a lot of, but there will be too many text. Let me show some simple example: Here is a simple feature file

Feature: BingSearchUi

Scenario: Search in Bing
    Given I open page 'http://www.bing.com'
    And I have entered 'visual studio' into the bing search field
    When I press search button on the bing page
    Then the result should contain 'www.visualstudio.com' on the bing page

It's valid specflow scenario, but by itself it can't do anything. To make it work you gotta write code and code is not only uses Nunit or MsTest. They are could be part of steps implementation, but not nessecary. E.g. Here is Given I open page 'http://www.bing.com' implementation, which atually uses Selenium, not any test frameworks:

[Given(@"I open page '(.*)'")]
 public void GivenIOpenPage(string ulr)
 {            
   driver.Navigate().GoToUrl(ulr);
 }

But to make things more complicated Specflow generates .cs file with steps and if I use NUnit this file is NUnit test class. It's small part of file:

[NUnit.Framework.TestAttribute()]
        [NUnit.Framework.DescriptionAttribute("Search in Bing")]
        [NUnit.Framework.CategoryAttribute("Selenium")]
        public virtual void SearchInBing()
        {
            TechTalk.SpecFlow.ScenarioInfo scenarioInfo = new TechTalk.SpecFlow.ScenarioInfo("Search in Bing", new string[] {
                        "Selenium"});
#line 15
this.ScenarioSetup(scenarioInfo);
#line 16
 testRunner.Given("I open page \'http://www.bing.com\'", ((string)(null)), ((TechTalk.SpecFlow.Table)(null)), "Given ");
#line 17
 testRunner.And("I have entered \'visual studio\' into the bing search field", ((string)(null)), ((TechTalk.SpecFlow.Table)(null)), "And ");
#line 18
 testRunner.When("I press search button on the bing page", ((string)(null)), ((TechTalk.SpecFlow.Table)(null)), "When ");
#line 19
 testRunner.Then("the result should contain \'www.visualstudio.com\' on the bing page", ((string)(null)), ((TechTalk.SpecFlow.Table)(null)), "Then ");
#line hidden
            this.ScenarioCleanup();
        }

and this generated file actually is called by test runner.

I think The simplest way to se how it works altogether - find any public repository on github with specflow in use and play with it.