Does cucumber do away with the need to write unit tests?

Question

I am a little confused by the sheer number of testing frameworks available for Ruby/ROR.

I have recently watched the Cucumber Railscasts and found them very interesting. So I started having a play and then struggled to see conceptually where I would put various tests.

It would seem to be quite possible to do everything that can be done in unit tests within Cucumber, so do I need to write unit tests or should I just write my feature definitions and concentrate on providing as good a coverage as I can get using that.

Should I create my Unit tests using Rspec or Test:Unit? When I'm testing Ajax functionality should I use Selenium or Watir?

There seem to be so many options here I am struggling to see which tools to use and where the boundaries are.

What are other peoples experiences of Cucumber and where to draw the line between writing Cucumber Integration tests and Test:Unit and/or Rspec based unit and functional tests. Is anyone aware of a good write-up on this subject suggesting where to draw lines between testing methods and the strengths and weaknesses of the various tool.

I appreciate that some of this is subjective but common approaches on how to attack this issue would be welcomed.

Answer 1

Use Cucumber at a high level to describe what a user should be able to see and do. Use RSpec, Test:Unit, Shoulda, etc. to write unit tests. Straight from the horse's mouth:

When you decide you want to add a new feature or fix a bug, start by writing a new feature or scenario that describes how the feature should work. Don’t write any code (yet).

...

This is when you start writing code. Start by writing a couple of lines of code to address the failure you got from Cucumber. Run cucumber again. Repeat and rinse until you’re happy with your feature. When you get down to nitty gritty details, drop down one abstraction level and use RSpec, or any Ruby testing framework, to write some specs/tests for your classes.

Cucumber is made to test your whole stack, together, as opposed to 'units'.

You need to decide where to draw the line, but a lot of under the hood stuff probably wouldn't be covered in a cucumber test. Say when signing up, I fill out a form, with my name, email, phone number, etc. A unit test might check to see that a new User will also create a new TelephoneNumber. From the user's perspective, they don't really care that it creates a new TelephoneNumber, they care that once they've signed up, they have an account and can see their telephone number.

I don't have too much experience writing cucumber tests (not yet), but I hope this helps a bit.

Answer 2

When a unit test fails (I mean a real unit test that tests a method in isolation using mocks), it tells you what "unit" has a problem. When an acceptance test fails, it tells you what "feature" has a problem, not where the problem is located.

Answer 3

When you create a rails app you get functional, interation, and unit tests by default. Cucumber is an additional test it is a way to also test the experience that your user will have. When they click the button labeled "go" they should see "success" rendered rather than a 404. This will make sure that nothing you do accidentally messes up the user experience, and that from the top to bottom your app works for the most common use cases you can think of. The other tests are meant to insure that nothing goes wrong, and that you have inspected ever model and method with a microscope. It may be possible to replicate unit tests in their entirety with cucumber, but it would be painful (and crazy slow to execute, especially if you're using selenium). The best time to write tests is when you're developing code, and the quickiest and easiest way to do that, is by using in the built-in rails testing and maybe some additional help such as shoulda, rspec, also i'm a huge fan of factory-girl. If you haven't already checked it out www.railscasts.com has a great intro to cucumber, and rspec, and factory-girl, ... I know this question has already been answered (it's no) but this is my two cents. Good luck coding!!

Answer 4

I have thought/struggled with this question much, and here's where I've arrived.

Cucumber first and Cucumber last. Cucumber will provide the primary test coverage.

The core model methods that do the real business work of the application should also be developed/covered with rspec/unit tests.

Why the unit tests as well?

1) The unit tests will test run much faster. 2) This core business logic may (will probably) be used in several ways beyond the current views (which Cucumber tests through). These methods ought to be hammered with all types of possible inputs and outputs directly calling the method in the test.

Why not unit test the rest of the models, and the controllers and views?

1) Cucumber already has it covered once. 2) I find that the views-controller-some-model-methods all work together to get things done (think everything exercised to log in); so I like to test them together.

Answer 5

I've been practicing Cucumber/RSpec for the past half year or so doing BDD.

First of all BDD is not easy to get into, it will feel unnatural at the beginning.

But once you get into it, there's no other way to do programming.

To answer your question. To test Javascript you'll need a javascript driver that can be used by Capybara which is used by Cucumber.

capybara-webkit is what all the cool kids use now these days

There's one important thing to note.

Integration tests are slow.

And unit tests are fast, but can be slow, so it's important you use the right database cleaner and you write good tests that have good isolation.

My test setup which I'm extremely happy with:

Guard for loading spork Spork for faster tests Cucumber for integration testing capybara-webkit for javascript testing RSpec for unit testing

I don't do view tests and controller tests as these are redundant in my opinion as good knowledge of XPATH willl have you writing remarkable tests that even cover your page layout and structure.

Answer 6

Personally I don't think that you should stop writing unit tests. As an acceptance testing tool, Cucumber should replace your functional tests and, if you writing, view tests.

Cucumber features are supposed to be simple and coupled to the real user's value a given feature has.

Answer 7

From my experience, Cucumber and Rspec have different appeal. Rspec appeals to me from a developer perspective because its easy to write and provides very quick feedback when something breaks. Cucumber does not appeal to me as a developer because it does not run as quickly as Rspec. However, Cucumber does appeal to me as a business stakeholder since it provides full coverage of entire features.

Do yourself a favor and keep writing unit tests.