Nunit async test exception assertion

Question

[Edit (May 2020)] - This issue has been reportedly addressed in newer releases of NUnit. Please see Nunit.ThrowsAsync. (Ref this answer, thanks @James-Ross)

---

I have a controller UserController with this action

// GET /blah
public Task<User> Get(string domainUserName)
{
        if (string.IsNullOrEmpty(domainUserName))
        {
            throw new ArgumentException("No username specified.");
        }

        return Task.Factory.StartNew(
            () =>
                {
                    var user = userRepository.GetByUserName(domainUserName);
                    if (user != null)
                    {
                        return user;
                    }

                    throw new HttpResponseException(Request.CreateErrorResponse(HttpStatusCode.NotFound, string.Format("{0} - username does not exist", domainUserName)));
                });
}

I am trying to write a test for the case where I throw a 404 exception.

Here is what I have tried, with the output -

1)

[Test]
public void someTest()
{
        var mockUserRepository = new Mock<IUserRepository>();
        mockUserRepository.Setup(x => x.GetByUserName(It.IsAny<string>())).Returns(default(User));
    var userController = new UserController(mockUserRepository.Object) { Request = new HttpRequestMessage() };

    Assert.That(async () => await userController.Get("foo"), Throws.InstanceOf<HttpResponseException>());
}

Result Test Failed

  Expected: instance of <System.Web.Http.HttpResponseException>
  But was:  no exception thrown

2. [Test] public void someTest() { var mockUserRepository = new Mock(); mockUserRepository.Setup(x => x.GetByUserName(It.IsAny())).Returns(default(User)); var userController = new UserController(mockUserRepository.Object) { Request = new HttpRequestMessage() };

     var httpResponseException = Assert.Throws<HttpResponseException>(() => userController.Get("foo").Wait());
     Assert.That(httpResponseException.Response.StatusCode, Is.EqualTo(HttpStatusCode.NotFound));
   

   }

Result Test failed

  Expected: <System.Web.Http.HttpResponseException>
  But was:  <System.AggregateException> (One or more errors occurred.)

[Test]
public void someTest()
{
        var mockUserRepository = new Mock<IUserRepository>();
        mockUserRepository.Setup(x => x.GetByUserName(It.IsAny<string>())).Returns(default(User));
    var userController = new UserController(mockUserRepository.Object) { Request = new HttpRequestMessage() };

    var httpResponseException = Assert.Throws<HttpResponseException>(async () => await userController.Get("foo"));
    Assert.That(httpResponseException.Response.StatusCode, Is.EqualTo(HttpStatusCode.NotFound));
}

Result Test Failed

  Expected: <System.Web.Http.HttpResponseException>
  But was:  null

[Test]
[ExpectedException(typeof(HttpResponseException))]
public async void ShouldThrow404WhenNotFound()
{            var mockUserRepository = new Mock<IUserRepository>();
        mockUserRepository.Setup(x => x.GetByUserName(It.IsAny<string>())).Returns(default(User));

    var userController = new UserController(mockUserRepository.Object) { Request = new HttpRequestMessage() };

    var task = await userController.Get("foo");
}

Result Test passes

Questions -

1. Why would Assert.Throws not handle HttpResponseException, when ExpectedException does?
2. I don't want to just test that exception is thrown. I want to assert on the Status Code of the response. What's the way to do this?

Any comparision on these behaviour and its cause(s) would be great!

Answer 1

You're seeing problems due to async void.

In particular:

1. async () => await userController.Get("foo") is converted into TestDelegate, which returns void, so your lambda expression is treated as async void. So the test runner will begin executing the lambda but not wait for it to complete. The lambda returns before Get completes (because it's async), and the test runner sees that it returned without an exception.

2. Wait wraps any exceptions in an AggregateException.

3. Again, the async lambda is being treated as async void, so the test runner is not waiting for its completion.

4. I recommend you make this async Task rather than async void, but in this case the test runner does wait for completion, and thus sees the exception.

According to this bug report, there is a fix for this coming in the next build of NUnit. In the meantime, you can build your own ThrowsAsync method; an example for xUnit is here.

Answer 2

I'm not sure when it was added, but the current version of Nunit (3.4.1 at time of writing) includes a ThrowsAsync method

see https://github.com/nunit/docs/wiki/Assert.ThrowsAsync

Example:

[Test]
public void ShouldThrow404WhenNotFound()
{
    var mockUserRepository = new Mock<IUserRepository>();
    mockUserRepository.Setup(x => x.GetByUserName(It.IsAny<string>())).Returns(default(User));
    var userController = new UserController(mockUserRepository.Object) { Request = new HttpRequestMessage() };

    var exception = Assert.ThrowsAsync<HttpResponseException>(() => userController.Get("foo"));

    Assert.That(exception.Response.StatusCode, Is.EqualTo(HttpStatusCode.NotFound));
}

Answer 3

This blog talks about problems similar to mine.

I followed the recommendation proposed there, and have a test like this -

    [Test]
    public void ShouldThrow404WhenNotFound()
    {
        var mockUserRepository = new Mock<IUserRepository>();
        mockUserRepository.Setup(x => x.GetByUserName(It.IsAny<string>())).Returns(default(User));
        var userController = new UserController(mockUserRepository.Object) { Request = new HttpRequestMessage() };

        var aggregateException = Assert.Throws<AggregateException>(() => userController.Get("foo").Wait());
        var httpResponseException = aggregateException.InnerExceptions
            .FirstOrDefault(x => x.GetType() == typeof(HttpResponseException)) as HttpResponseException;

        Assert.That(httpResponseException, Is.Not.Null);
        Assert.That(httpResponseException.Response.StatusCode, Is.EqualTo(HttpStatusCode.NotFound));
    }

I am not too pleased with it, but this works.

EDIT 1

Inspired by @StephenCleary, I added a static helper class that does the asserts that I am looking for. It looks like this -

public static class AssertEx
{
    public static async Task ThrowsAsync<TException>(Func<Task> func) where TException : class
    {
        await ThrowsAsync<TException>(func, exception => { });
    } 

    public static async Task ThrowsAsync<TException>(Func<Task> func, Action<TException> action) where TException : class
    {
        var exception = default(TException);
        var expected = typeof(TException);
        Type actual = null;
        try
        {
            await func();
        }
        catch (Exception e)
        {
            exception = e as TException;
            actual = e.GetType();
        }

        Assert.AreEqual(expected, actual);
        action(exception);
    }
}

I can now have a test like -

    [Test]
    public async void ShouldThrow404WhenNotFound()
    {
        var mockUserRepository = new Mock<IUserRepository>();
        mockUserRepository.Setup(x => x.GetByUserName(It.IsAny<string>())).Returns(default(User));
        var userController = new UserController(mockUserRepository.Object) { Request = new HttpRequestMessage() };

        Action<HttpResponseException> asserts = exception => Assert.That(exception.Response.StatusCode, Is.EqualTo(HttpStatusCode.NotFound));
        await AssertEx.ThrowsAsync(() => userController.Get("foo"), asserts);
    }

Answer 4

If you await a Task then Exceptions that are thrown are aggregated into AggregateException. You can inspect the inner exceptions of AggregateException. This could be the reason why you case 2 does not work.

Unhandled exceptions that are thrown by user code that is running inside a task are propagated back to the joining thread, except in certain scenarios that are described later in this topic. Exceptions are propagated when you use one of the static or instance Task.Wait or Task.Wait methods, and you handle them by enclosing the call in a try-catch statement. If a task is the parent of attached child tasks, or if you are waiting on multiple tasks, then multiple exceptions could be thrown. To propagate all the exceptions back to the calling thread, the Task infrastructure wraps them in an AggregateException instance. The AggregateException has an InnerExceptions property that can be enumerated to examine all the original exceptions that were thrown, and handle (or not handle) each one individually. Even if only one exception is thrown, it is still wrapped in an AggregateException.

Link to MSDN

Answer 5

I have a similar issue that you have in scenario 3 Test case failed due to following result

Expected: <UserDefineException>
But was:  null

by using Assert.ThrowAsync<> problem gets solved

My Web API action method and Unit test case method as below

public async Task<IHttpActionResult> ActionMethod(RequestModel requestModel)
{
   throw UserDefineException();
}


[Test]
public void Test_Contrller_Method()
{
   Assert.ThrowsAsync<UserDefineException>(() => _controller.ActionMethod(new RequestModel()));
}    

Answer 6

This is an example from documentation:

var ex = Assert.ThrowsAsync<ArgumentException>(async () => await MethodThatThrows());

1. Use ThrowsAsync
2. Use async / await

https://docs.nunit.org/articles/nunit/writing-tests/assertions/classic-assertions/Assert.ThrowsAsync.html