Unit-Testing Acceptance-Tests?

Question

I am currently making some Acceptance-Tests that will help drive the design of a program I am about to do. Everything seems fine except I've realized that the Acceptance-Tests are kinda complex, that is, although they are conceptually simple, they require quite a bit of tricky code to run. I'll need to make a couple of "helper" classes for my Acceptance-Tests.

My question is in how to develop them:

1. Make Unit-Tests of my Acceptance-Tests(this seems odd -- has anyone done anything like it?)
2. Make Unit-Tests for those help classes. After I have done all the code of those help classes, I can pass and start working on the real Unit-Tests of my System. When using this approach, where would you put the helper classes? In the tests' project or in the real project? They don't necessarily have dependencies on testing/mocking frameworks.
3. Any other idea?

Answer 1

A friend is very keen on the notion that acceptance tests tell you whether your code is broken, while unit tests tell you where it's broken; these are complementary and valuable bits of information. Acceptance tests are better at letting you know when you're done.

But to get done, you need all the pieces along the way; you need them to be working, and that's what unit tests are great at. Done test-first, they'll also lead you to better design (not just code that works). A good approach is to write a big-picture acceptance test, and say to yourself: "when this is passing, I'm done." Then work to make it pass, working TDD: write a small unit test for the next little bit of functionality you need to make the AT pass; write the code to make it pass; refactor; repeat. As you progress, run the AT from time to time; you will probably find it failing later and later in the test. And, as mentioned above, when it's passing, you're done.

I don't think unit testing the acceptance test itself makes much sense. But unit testing its helper classes - indeed, writing them test-first - is a very good way to go. You're likely to find some methods that you write "just for the test" working their way into the production code - and even if you don't, you still want to know that the code your ATs use is working right.

If your AT is simple enough, the old adage of "the test tests the code, and the code tests the test" is probably sufficient - when you have a failing test, it's either because the code is wrong or because the test is wrong, and it should be easy enough to figure out which. But when the test is complicated, it's good to have its underpinnings well tested, too.

Answer 2

If you think of software development like a car production plant, then the act of writing software is like developing a new car. Each component is tested separately because it's new. It's never been done before. (If it has, you can either get people who've done it before or buy it off the shelf.)

Your build system, which builds your software and also which tests it, is like the conveyor belt - the process which churns out car after car after car. Car manufacturers usually consider how they're going to automate production of new components and test their cars as part of creating new ones, and you can bet that they also test the machines which produce those cars.

So, yes, unit-testing your acceptance tests seems perfectly fine to me, especially if it helps you go faster and keep things easier to change.

Answer 3

There is nothing wrong with using unit test framework (like JUnit) to write acceptance tests (or integration tests). People don't like calling them 'unit tests' for many reasons. To me the main reason is that integration/acceptance tests won't run every time one checks in the changes (too long or/and no proper environment).

Your helper classes are rather standard thing that comprise "test infrastructure code". They don't belong anywhere else but test code. It's your choice to test them or not. But without them your tests won't be feasible in big systems.

So, your choice is #2 or no tests of tests at all. There is nothing wrong with refactoring infrastructure code to make it more transparent and simple.

Answer 4

Your option 2 is the way I'd do it: Write helper classes test-first - it sounds like you know what they should do.

The tests are tests, and although the helper classes are not strictly tests, they won't be referenced by your main code, just the tests, so they belong with the tests. Perhaps they could be in a separate package/namespace from regular tests.