So you start with a failing acceptance test, and build out the feature with unit tests until the acceptance test passes. But as you get passing unit tests, should you be checking in to source control? If you do, do you mark the acceptance test to be ignored (and if so where? in code or on the build server)? How does this fit into continuous integration?
2 Answers
5
votes
No, you should not check in a failing test, in a Continuous Integration environment you should be keeping the code releasable at all times, by definition a failing acceptance test shows that the code is not currently releasable.
While it’s failing, an acceptance test demonstrates that the system does not yet implement that feature; when it passes, we’re done.
Growing Object-Oriented Software Guided by Tests by Steve Freeman and Nat Pryce
If you are worried about losing your progress, or want to persist your changes temporarily store them as a shelveset, that way your changes are on the server and available to another developer if you are not able to get in to work to continue the feature, but equally the team has a working build that another developer can branch from to do other changes, or can integrate their completed feature with.
I wouldn't quite put it this strongly, but this pretty much sums it up -
43. Share code only when ready. Never check in code that’s not ready for others. Deliberately checking in code that doesn’t compile or pass its unit tests should be considered an act of criminal project negligence.
Practices of an Agile Developer by by Venkat Subramaniam and Andy Hunt.
0
votes
I wouldn't push a failing acceptance test to source control, either on a master branch or a feature branch. The failing test would make the branch's build red, making it harder for me to notice failures of other tests that my changes might accidentally cause. But it's also helpful to share work in progress via source control, so we need a way to do that while keeping the build green.
The testing frameworks I use these days address this by providing ways to mark tests as being unimplemented:
- Calling
pendingin a Cucumber step definition skips the rest of the step and scenario and prints a warning to remind you. Cucumber's default configuration also ignores scenarios tagged@wip, although that mechanism doesn't produce a reminder warning. - Similarly, RSpec has several ways to skip examples.
If your testing framework doesn't have any such feature, you can always comment out in-progress tests before committing. I generally don't push in-progress tests to master, but they're fine on in-progress feature branches.
