How do you capture requirements with declarative acceptance tests?

7
votes

Background

I am trying to help my team organize for a new mobile app project. We have chosen to follow BDD (see also BDD definition) in order to capture plain English requirements that form a contract between stakeholders and developers for each individual user story.

We use the acceptance tests to document each user story's requirements. Acceptance tests are written before sprint planning. Developers refine and add to the tests during sprint planning.

We define Acceptance Criteria as a list of rules (eg: input validation, default values, etc) and Acceptance Tests as a list of Cucumber scenarios. We plan on using Calabash for mobile testing.

I feel acceptance criteria/tests are a more agile and therefore better solution to more formal requirements documents.

I feel I have found an effective solution fo, but I would like to understand how others are collecting requirements and writing acceptance tests.

Problem

There is a debate in the Cucumber community of imperative vs declarative test steps. I lean toward imperative, because a developer must know what a deliverable user story looks like.

I do not feel UI coupling aka brittle tests is an issue. There are ways to decouple the UI from the test (eg: page objects). I also do not feel that having detailed steps make it hard for a non-technical stakeholder to understand (unless they don't know how to use a web browser or mobile device, but that's a separate issue).

I may be misappropriating the term "Acceptance Test". In my use, an acceptance test is not on the same level of scope as a unit test. I view an acceptance test as a high level integration test.

The Example

  • As a guest
  • I want to login
  • to access app functionality

Imperative Test

  • Scenario: Valid login
    • Given I am on the "login" screen
    • When I enter "[email protected]" in "email"
      • And I enter "password1" in "password"
      • And I tap "login"
    • Then I see "login successful"

Declarative Test

  • Scenario: Valid login
    • Given I have a valid account
    • Then I can login

These both can cover the same functionality and the latter is shorter, but it does not say if I can login with a username, email or facebook/twitter/google/etc account. It's just not enough to actually code a solution

The Question

How do you capture the requirements for a feature with declarative steps?

2
cucumberbddcalabashacceptance-testing
Dan North (creator of BDD) has some nice thoughts on the matter: dannorth.net/2011/01/31/whose-domain-is-it-anyway. It helps to keep in mind that a scenario should only address 2 domains: the user of the feature and the developer who will implement it. The scenario should clarify to both domains what is to happen how(eg: "user logs in with email and password", not "user logs in with username and password", etc) – Brenden
blog.cyrusinnovation.com/2013/04/… says that imperative is appropriate when a stakeholder is concerned with the user experience of a given scenario. It makes sense to aim for brevity via declarative while being explicit with imperative when it's called for. – Brenden
This question appears to be off-topic because it is about project planning. I think it should be migrated to programmers.stackexchange.com. – Brenden
I don't think it's off topic as the question is about imperative versus declarative BDD tests, rather than about project planning. Might be worth renaming the question title? – Ben Smith

2 Answers

6
votes

Nicely written question!

How do you capture the requirements for a feature with declarative steps?

The requirements for a feature are recorded in the step definitions.

Hence in your imperative example:

When I enter "[email protected]" in "email"
And I enter "password1" in "password"
And I tap "login"

this could be made declarative by re-writing it as:

Given I login using valid credentials

The steps to navigate to a valid account (i.e. implementing the acceptance criteria which defines what "valid" means) can then be implemented in the step definition for this scenario statement. The same will apply for the opposite scenario i.e.

Given I login using invalid credentials

Again, the steps to implement this scenario which satisfy the acceptance criteria can be implemented in the underlying step definition.

Taking this declarative approach means that you lose the (imperative) requirements from the feature (i.e. what exact steps need to be performed), making it harder for the business to see exactly what these scenarios are doing from just reading the feature file. However, what you gain is that the tests become less brittle, as the specific steps to achieve a task are recorded in the step definition, and this step definition can be shared amongst many features.

At my company we wrestle with the same concerns, and we find that in some cases it's better to use imperative rather than declarative, and vice versa. For example, in your case the steps which make up "Given I have a valid account" may be used in many features, so making it declarative is rational. However if you have a feature where many different string values are inputted, then in that case it's probably best to write them imperatively.

"Horses for courses!"

It'll be very interesting to see other answers to this question from the SO community.

2
votes

I recently visited a shop/online to buy a Washing machine and a Dish washer. All I wanted was to buy one that uses less water and has a quick wash. But the details I encountered were overwhelming; such as RPM, Thickness of inner drum, Total connected load in KW etc.

Imperative style may appear suitable from your simple example above but in reality it makes reading scenarios harder and boring. You can experience it by reading 10 scenarios from a project in which you are not directly involved at a technical / day-to-day level.

Given that one of the objective of using cucumber is to bring in transparency across the project, in particular the non-technical users, declarative style is much better at getting management actively involved. I have seen that in my projects.

Here is a fictitious story. Try and implement it in imperative style, come back and read it in couple of days, you will realize that its too boring.

Feature: Delivery 
    Free delivery is offered to customers who order two or more items

  Scenario Outline: Calculate postage for delivery
    Given I am signed-in
    When I "<order>" items
    Then postage should be "<postage>"   

    Examples:
    | order | postage |
    | 1     | 0.99    |
    | 2     | 0       |
    | 3     | 0       |
    | 0     | ?       |

Another link that you may wish to read How to implement UI testing without shooting yourself in the foot