how do you specify that an NUnit test should fail?

Question

Is there a way to specify that you want an NUnit test to fail, meaning that a fail should be reported as a pass and a pass should be reported as a fail? This would be useful when testing your own NUnit extensions. Here is an example of something I would like to be able to do:

[Test]
[ExpectFail]
public void TypeOf_fail() {
    string str = "abc";
    str.Should().Be.TypeOf<int>();
}

This does not compile because [ExpectFail] is an imaginary attribute to illustrate what I want to do, but the code inside the method works fine. This problem is specific to testing an NUnit extension in that you can normally just easily write your tests to pass, not fail. In this case you need prove that it is possible to write a failing test using the NUnit extension that you are testing.

Answer 1

Unit tests should be designed so that:
They set up some state
They run the method under test
They assert that one thing is correct after the method under test has completed

(reference: The Art of Unit Testing by Roy Osherove)

Why are tests that are designed to fail a bad thing? They could fail in unexpected ways, and still be marked as a pass because they failed. In your example, assuming that Should() is the method under test (though this point remains even if it isn't), you write the test above and mark it as 'expected to fail'. It fails. Everything is fine. In a couple of months you come back to Should() and realise it needs some refactoring, so you change its implementation.

Now, in your example, Should() throws an exception, because you've accidentally introduced a bug. But your test (which fails because of the exception now, not the logic) is marked as should fail, and it does, so it's still marked as a pass, despite the breaking change.

The test should be designed to pass, not to fail, that way if it fails in another, unexpected, way you'll be notified. So in your example you should write tests with opposite logic:

[Test]
public void TypeOf_ShouldBeString() {
    string str = "abc";
    str.Should().Be.TypeOf<string>();
}

or:

[Test]
public void TypeOf_ShouldNotBeInt() {
    string str = "abc";
    str.Should().Not.Be.TypeOf<int>();
}

(Not sure of the syntax you're using, so .Not probably will need replacing with the correct syntax, but the sentiment holds).

Edit2: If what you're trying to do is ensure that your Should() method fails (by failing an Assert. method) then what you want to do is catch the NUnit AssertionException which the Assert. static methods throw. Try this:

[Test]
[ExpectedException(typeof(AssertionException))]
public void ShouldBeTypeOf_WithInt_Fails() {
    string str = "abc";
    str.Should().Be.TypeOf<int>();
}    

Answer 2

I know this is an old post, but here is what has helped me, using NUnit:

[TestCase("SomeValidValue")]
[TestCase("{3X5XFX9X-7XCX-4XCX-8X3X-3XAXEX9X0XBX}", ExpectedException = typeof(AssertionException))]
public void GetSpecificConfiguration(string key)
{
    Assert.IsNotNull(Config.Instance.GetRecord(key));
}

This approach allows me to have the same test, with two expectations, one succeeding, and one failing.

Answer 3

What about Assert.Throws<XXXException>(x.DoSomething());

The nice thing here is that if the test passes (i.e. the exception was thrown), the return value is the actual exception itself, and you can then interrogate it and Assert further based on that..

Given that in this case you're wanting to test NUnit extensions, which, I'm assuming, function by making Assert calls, you could Assert.Throws<AssertionException>(...), and then do the above.

I'm thinking of writing some similar test plumbing code which I might in turn need tests for, so I'll let you know if I discover anything else in this area.

Answer 4

If you mean that the block of code is expected to throw an exception for test to pass, here's the code:

[Test]
[ExpectedException(typeof(Exception))]
public void TypeOf_fail() {
    string str = "abc";
    str.Should().Be.TypeOf<int>();
}

Of course, replace Exception with the most specific exception possible.