Does anyone know whether there is an assert
or something like that which can test whether an exception was thrown in the code being tested?
15 Answers
<?php
require_once 'PHPUnit/Framework.php';
class ExceptionTest extends PHPUnit_Framework_TestCase
{
public function testException()
{
$this->expectException(InvalidArgumentException::class);
// or for PHPUnit < 5.2
// $this->setExpectedException(InvalidArgumentException::class);
//...and then add your test code that generates the exception
exampleMethod($anInvalidArgument);
}
}
expectException() PHPUnit documentation
PHPUnit author article provides detailed explanation on testing exceptions best practices.
You can also use a docblock annotation until PHPUnit 9 is released:
class ExceptionTest extends PHPUnit_Framework_TestCase
{
/**
* @expectedException InvalidArgumentException
*/
public function testException()
{
...
}
}
For PHP 5.5+ (especially with namespaced code), I now prefer using ::class
If you're running on PHP 5.5+, you can use ::class
resolution to obtain the name of the class with expectException
/setExpectedException
. This provides several benefits:
- The name will be fully-qualified with its namespace (if any).
- It resolves to a
string
so it will work with any version of PHPUnit. - You get code-completion in your IDE.
- The PHP compiler will emit an error if you mistype the class name.
Example:
namespace \My\Cool\Package;
class AuthTest extends \PHPUnit_Framework_TestCase
{
public function testLoginFailsForWrongPassword()
{
$this->expectException(WrongPasswordException::class);
Auth::login('Bob', 'wrong');
}
}
PHP compiles
WrongPasswordException::class
into
"\My\Cool\Package\WrongPasswordException"
without PHPUnit being the wiser.
Note: PHPUnit 5.2 introduced
expectException
as a replacement forsetExpectedException
.
Code below will test exception message and exception code.
Important: It will fail if expected exception not thrown too.
try{
$test->methodWhichWillThrowException();//if this method not throw exception it must be fail too.
$this->fail("Expected exception 1162011 not thrown");
}catch(MySpecificException $e){ //Not catching a generic Exception or the fail function is also catched
$this->assertEquals(1162011, $e->getCode());
$this->assertEquals("Exception Message", $e->getMessage());
}
You can use assertException extension to assert more than one exception during one test execution.
Insert method into your TestCase and use:
public function testSomething()
{
$test = function() {
// some code that has to throw an exception
};
$this->assertException( $test, 'InvalidArgumentException', 100, 'expected message' );
}
I also made a trait for lovers of nice code..
The PHPUnit expectException
method is very inconvenient because it allows to test only one exception per a test method.
I've made this helper function to assert that some function throws an exception:
/**
* Asserts that the given callback throws the given exception.
*
* @param string $expectClass The name of the expected exception class
* @param callable $callback A callback which should throw the exception
*/
protected function assertException(string $expectClass, callable $callback)
{
try {
$callback();
} catch (\Throwable $exception) {
$this->assertInstanceOf($expectClass, $exception, 'An invalid exception was thrown');
return;
}
$this->fail('No exception was thrown');
}
Add it to your test class and call this way:
public function testSomething() {
$this->assertException(\PDOException::class, function() {
new \PDO('bad:param');
});
$this->assertException(\PDOException::class, function() {
new \PDO('foo:bar');
});
}
Comprehensive Solution
PHPUnit's current "best practices" for exception testing seem.. lackluster (docs).
Since I wanted more than the current expectException
implementation, I made a trait to use on my test cases. It's only ~50 lines of code.
- Supports multiple exceptions per test
- Supports assertions called after the exception is thrown
- Robust and clear usage examples
- Standard
assert
syntax - Supports assertions for more than just message, code, and class
- Supports inverse assertion,
assertNotThrows
- Supports PHP 7
Throwable
errors
Library
I published the AssertThrows
trait to Github and packagist so it can be installed with composer.
Simple Example
Just to illustrate the spirit behind the syntax:
<?php
// Using simple callback
$this->assertThrows(MyException::class, [$obj, 'doSomethingBad']);
// Using anonymous function
$this->assertThrows(MyException::class, function() use ($obj) {
$obj->doSomethingBad();
});
Pretty neat?
Full Usage Example
Please see below for a more comprehensive usage example:
<?php
declare(strict_types=1);
use Jchook\AssertThrows\AssertThrows;
use PHPUnit\Framework\TestCase;
// These are just for illustration
use MyNamespace\MyException;
use MyNamespace\MyObject;
final class MyTest extends TestCase
{
use AssertThrows; // <--- adds the assertThrows method
public function testMyObject()
{
$obj = new MyObject();
// Test a basic exception is thrown
$this->assertThrows(MyException::class, function() use ($obj) {
$obj->doSomethingBad();
});
// Test custom aspects of a custom extension class
$this->assertThrows(MyException::class,
function() use ($obj) {
$obj->doSomethingBad();
},
function($exception) {
$this->assertEquals('Expected value', $exception->getCustomThing());
$this->assertEquals(123, $exception->getCode());
}
);
// Test that a specific exception is NOT thrown
$this->assertNotThrows(MyException::class, function() use ($obj) {
$obj->doSomethingGood();
});
}
}
?>
Here's all the exception assertions you can do. Note that all of them are optional.
class ExceptionTest extends PHPUnit_Framework_TestCase
{
public function testException()
{
// make your exception assertions
$this->expectException(InvalidArgumentException::class);
// if you use namespaces:
// $this->expectException('\Namespace\MyException');
$this->expectExceptionMessage('message');
$this->expectExceptionMessageRegExp('/essage$/');
$this->expectExceptionCode(123);
// code that throws an exception
throw new InvalidArgumentException('message', 123);
}
public function testAnotherException()
{
// repeat as needed
$this->expectException(Exception::class);
throw new Exception('Oh no!');
}
}
Documentation can be found here.
/**
* @expectedException Exception
* @expectedExceptionMessage Amount has to be bigger then 0!
*/
public function testDepositNegative()
{
$this->account->deposit(-7);
}
Be very carefull about "/**"
, notice the double "*". Writing only "**"(asterix) will fail your code.
Also make sure your using last version of phpUnit. In some earlier versions of phpunit @expectedException Exception is not supported. I had 4.0 and it didn't work for me, I had to update to 5.5 https://coderwall.com/p/mklvdw/install-phpunit-with-composer to update with composer.
TLDR; scroll to: Use PHPUnit's Data Provider
PHPUnit 9.5 offers following methods to test exceptions:
$this->expectException(string $exceptionClassName);
$this->expectExceptionCode(int|string $code);
$this->expectExceptionMessage(string $message);
$this->expectExceptionMessageMatches(string $regularExpression);
$this->expectExceptionObject(\Exception $exceptionObject);
However the Documentation is vague about the order of any of the above methods in the test code.
If you get used to using assertions for example:
<?php
class SimpleAssertionTest extends \PHPUnit\Framework\TestCase
{
public function testSimpleAssertion(): void
{
$expected = 'bar';
$actual = 'bar';
$this->assertSame($expected, $actual);
}
}
output:
✔ Simple assertion
OK (1 test, 1 assertion)
you may be surprised by failing the exception test:
<?php
use PHPUnit\Framework\TestCase;
final class ExceptionTest extends TestCase
{
public function testException(): void
{
throw new \InvalidArgumentException();
$this->expectException(\InvalidArgumentException::class);
}
}
output:
✘ Exception
├ InvalidArgumentException:
ERRORS!
Tests: 1, Assertions: 0, Errors: 1.
The error is because:
Once an exception is thrown the PHP can not return to the line of code that comes after the line that thrown the exception. Catching an exception changes nothing in this regard. Throwing an exception is a one way ticket.
Unlike errors, exceptions don't have a capability to recover from them and make PHP continue code execution as if there was no exception at all.
Therefore PHPUnit does not even reach the place:
$this->expectException(\InvalidArgumentException::class);
if it was preceded by:
throw new \InvalidArgumentException();
Moreover, PHPUnit will never be able to reach that place, no matter its exception catching capabilities.
Therefore using any of the PHPUnit's exception testing methods:
$this->expectException(string $exceptionClassName);
$this->expectExceptionCode(int|string $code);
$this->expectExceptionMessage(string $message);
$this->expectExceptionMessageMatches(string $regularExpression);
$this->expectExceptionObject(\Exception $exceptionObject);
must be before a code where an exception is expected to be thrown in contrary to an assertion that is placed after an actual value is set.
A proper order of using exception test:
<?php
use PHPUnit\Framework\TestCase;
final class ExceptionTest extends TestCase
{
public function testException(): void
{
$this->expectException(\InvalidArgumentException::class);
throw new \InvalidArgumentException();
}
}
Because call to PHPUnit internal methods for testing exceptions must be before an exception is thrown it makes sense that PHPUnit methods related to test exceptions start from $this->excpect
instead of $this->assert
.
Knowing already that:
Once an exception is thrown the PHP can not return to the line of code that comes after the line that thrown the exception.
You should be able to easily spot a bug in this test:
<?php
namespace VendorName\PackageName;
class ExceptionTest extends \PHPUnit\Framework\TestCase
{
public function testThrowException(): void
{
# Should be OK
$this->expectException(\RuntimeException::class);
throw new \RuntimeException();
# Should Fail
$this->expectException(\RuntimeException::class);
throw new \InvalidArgumentException();
}
}
The first $this->expectException()
should be OK, it expects an exception class before an exact exception class as expected is thrown so nothing wrong here.
The second that should fail expects RuntimeException
class before a completely different exception is thrown so it should fail but will PHPUnit execution reach that place at all?
The test's output is:
✔ Throw exception
OK (1 test, 1 assertion)
OK
?
No it is far from OK
if the test passes and it should Fail
on the second exception. Why is that?
Note that the output has:
OK (1 test, 1 assertion)
where the count of tests is right but there is only 1 assertion
.
There should be 2 assertions = OK
and Fail
that makes test not passing.
That's simply because PHPUnit is done with executing testThrowException
after the line:
throw new \RuntimeException();
that is a one way ticket outside the scope of the testThrowException
to somewhere where PHPUnit catches the \RuntimeException
and does what it needs to do, but whatever it could do we know it will not be able to jump back into testThrowException
hence the code:
# Should Fail
$this->expectException(\RuntimeException::class);
throw new \InvalidArgumentException();
will never be executed and that's why from PHPUnit point of view the test result is OK
instead of Fail
.
That's not a good news if you would like to use multiple $this->expectException()
or a mix of $this->expectException()
and $this->expectExceptionMessage()
calls in the same test method:
<?php
namespace VendorName\PackageName;
class ExceptionTest extends \PHPUnit\Framework\TestCase
{
public function testThrowException(): void
{
# OK
$this->expectException(\RuntimeException::class);
throw new \RuntimeException('Something went wrong');
# Fail
$this->expectExceptionMessage('This code will never be executed');
throw new \RuntimeException('Something went wrong');
}
}
gives wrong:
OK (1 test, 1 assertion)
because once an exception is thrown all other $this->expect...
calls related to testing exceptions will not be executed and the PHPUnit test case result will only contain the result of the first expected exception.
How to test multiple Exceptions?
Split multiple exceptions into separate tests:
<?php
namespace VendorName\PackageName;
class ExceptionTest extends \PHPUnit\Framework\TestCase
{
public function testThrowExceptionBar(): void
{
# OK
$this->expectException(\RuntimeException::class);
throw new \RuntimeException();
}
public function testThrowExceptionFoo(): void
{
# Fail
$this->expectException(\RuntimeException::class);
throw new \InvalidArgumentException();
}
}
gives:
✔ Throw exception bar
✘ Throw exception foo
┐
├ Failed asserting that exception of type "InvalidArgumentException" matches expected exception "RuntimeException". Message was: "" at
FAILURES!
Tests: 2, Assertions: 2, Failures: 1.
FAILURES
as it should.
This method however has a downside at its fundamental approach - for every exception thrown you need a separate test. That will make a flood of tests just to check exceptions.
Catch an exception and check it with an assertion
If you can't continue script execution after an exception was thrown you may simply catch an expected exception and later get all data about it with the methods an exception provides and use that with combination of expected values and assertions:
<?php
namespace VendorName\PackageName;
class ExceptionTest extends \PHPUnit\Framework\TestCase
{
public function testThrowException(): void
{
# OK
unset($className);
try {
$location = __FILE__ . ':' . (string) (__LINE__ + 1);
throw new \RuntimeException('Something went wrong');
} catch (\Exception $e) {
$className = get_class($e);
$msg = $e->getMessage();
$code = $e->getCode();
}
$expectedClass = \RuntimeException::class;
$expectedMsg = 'Something went wrong';
$expectedCode = 0;
if (empty($className)) {
$failMsg = 'Exception: ' . $expectedClass;
$failMsg .= ' with msg: ' . $expectedMsg;
$failMsg .= ' and code: ' . $expectedCode;
$failMsg .= ' at: ' . $location;
$failMsg .= ' Not Thrown!';
$this->fail($failMsg);
}
$this->assertSame($expectedClass, $className);
$this->assertSame($expectedMsg, $msg);
$this->assertSame($expectedCode, $code);
# ------------------------------------------
# Fail
unset($className);
try {
$location = __FILE__ . ':' . (string) (__LINE__ + 1);
throw new \InvalidArgumentException('I MUST FAIL !');
} catch (\Exception $e) {
$className = get_class($e);
$msg = $e->getMessage();
$code = $e->getCode();
}
$expectedClass = \InvalidArgumentException::class;
$expectedMsg = 'Something went wrong';
$expectedCode = 0;
if (empty($className)) {
$failMsg = 'Exception: ' . $expectedClass;
$failMsg .= ' with msg: ' . $expectedMsg;
$failMsg .= ' and code: ' . $expectedCode;
$failMsg .= ' at: ' . $location;
$failMsg .= ' Not Thrown!';
$this->fail($failMsg);
}
$this->assertSame($expectedClass, $className);
$this->assertSame($expectedMsg, $msg);
$this->assertSame($expectedCode, $code);
}
}
gives:
✘ Throw exception
┐
├ Failed asserting that two strings are identical.
┊ ---·Expected
┊ +++·Actual
┊ @@ @@
┊ -'Something·went·wrong'
┊ +'I·MUST·FAIL·!'
FAILURES!
Tests: 1, Assertions: 5, Failures: 1.
FAILURES
as it should but oh my lord, did you read all that above? You need to take care for clearing variables unset($className);
to detect if an exception was thrown, then this creature $location = __FILE__ ...
to have a precise location of the exception in case it was not thrown, then checking if the exception was thrown if (empty($className)) { ... }
and using $this->fail($failMsg);
to signal if the exception was not thrown.
Use PHPUnit's Data Provider
PHPUnit has a helpful mechanism called a Data Provider. A data provider is a method that returns the data (array) with data sets. A single data set is used as the argument(s) when a test method - testThrowException
is called by PHPUnit.
If the data provider returns more than one data set then the test method will be run multiple times, each time with another data set. That is helpful when testing multiple exceptions or/and multiple exception's properties like class name, message, code because even though:
Once an exception is thrown the PHP can not return to the line of code that comes after the line that thrown the exception.
PHPUnit will run the test method multiple times, each time with different data set so instead of testing for example multiple exceptions in a single test method run (that will fail).
That's why we may make a test method responsible for testing just one exception at the time but run that test method multiple times with different input data and expected exception by using the PHPUnit's data provider.
Definition of the data provider method can be done by doing @dataProvider
annotation to the test method that should be supplied by the data provider with a data set.
<?php
class ExceptionCheck
{
public function throwE($data)
{
if ($data === 1) {
throw new \RuntimeException;
} else {
throw new \InvalidArgumentException;
}
}
}
class ExceptionTest extends \PHPUnit\Framework\TestCase
{
public function ExceptionTestProvider() : array
{
$data = [
\RuntimeException::class =>
[
[
'input' => 1,
'className' => \RuntimeException::class
]
],
\InvalidArgumentException::class =>
[
[
'input' => 2,
'className' => \InvalidArgumentException::class
]
]
];
return $data;
}
/**
* @dataProvider ExceptionTestProvider
*/
public function testThrowException($data): void
{
$this->expectException($data['className']);
$exceptionCheck = new ExceptionCheck;
$exceptionCheck->throwE($data['input']);
}
}
gives result:
✔ Throw exception with RuntimeException
✔ Throw exception with InvalidArgumentException
OK (2 tests, 2 assertions)
Note that even there is just a one test method in the entire ExceptionTest
the output of the PHPUnit is:
OK (2 tests, 2 assertions)
So even the line:
$exceptionCheck->throwE($data['input']);
threw the exception at the first time that was no problem for testing another exception with the same test method because PHPUnit ran it again with different data set thanks to the data provider.
Each data set returned by the data provider can be named, you just need to use a string as a key under which a data set is stored. Therefore the expected exception class name is used twice. As a key of data set array and as a value (under 'className' key) that is later used as an argument for $this->expectException()
.
Using strings as key names for data sets makes that pretty and self explanatory summary:
✔ Throw exception with RuntimeException
✔ Throw exception with InvalidArgumentException
and if you change the line:
if ($data === 1) {
to:
if ($data !== 1) {
of the public function throwE($data)
to get wrong exceptions thrown and run the PHPUnit again you'll see:
✘ Throw exception with RuntimeException
├ Failed asserting that exception of type "InvalidArgumentException" matches expected exception "RuntimeException". Message was: "" at (...)
✘ Throw exception with InvalidArgumentException
├ Failed asserting that exception of type "RuntimeException" matches expected exception "InvalidArgumentException". Message was: "" at (...)
FAILURES!
Tests: 2, Assertions: 2, Failures: 2.
as expected:
FAILURES! Tests: 2, Assertions: 2, Failures: 2.
with exactly pointed out the data sets' names that caused some problems:
✘ Throw exception with RuntimeException
✘ Throw exception with InvalidArgumentException
Making public function throwE($data)
not throwing any exceptions:
public function throwE($data)
{
}
and running PHPUnit again gives:
✘ Throw exception with RuntimeException
├ Failed asserting that exception of type "RuntimeException" is thrown.
✘ Throw exception with InvalidArgumentException
├ Failed asserting that exception of type "InvalidArgumentException" is thrown.
FAILURES!
Tests: 2, Assertions: 2, Failures: 2.
It looks that using a data provider has several advantages:
- The Input data and/or expected data is separated from the actual test method.
- Every data set can have a descriptive name that clearly points out what data set caused test to pass or fail.
- In case of a test fail you get a proper failure message mentioning that an exception was not thrown or a wrong exception was thrown instead of an assertion that x is not y.
- There is only a single test method needed for testing a single method that may throw multiple exceptions.
- It is possible to test multiple exceptions and/or multiple exception's properties like class name, message, code.
- No need for any non-essential code like try catch block, instead just using the built in PHPUnit's feature.
Testing Exceptions Gotchas
exception of type "TypeError"
With PHP7 datatype support this test:
<?php
declare(strict_types=1);
class DatatypeChat
{
public function say(string $msg)
{
if (!is_string($msg)) {
throw new \InvalidArgumentException('Message must be a string');
}
return "Hello $msg";
}
}
class ExceptionTest extends \PHPUnit\Framework\TestCase
{
public function testSay(): void
{
$this->expectException(\InvalidArgumentException::class);
$chat = new DatatypeChat;
$chat->say(array());
}
}
fails with the output:
✘ Say
├ Failed asserting that exception of type "TypeError" matches expected exception "InvalidArgumentException". Message was: "Argument 1 passed to DatatypeChat::say() must be of the type string, array given (..)
FAILURES!
Tests: 1, Assertions: 1, Failures: 1.
even though there is in the method say
:
if (!is_string($msg)) {
throw new \InvalidArgumentException('Message must be a string');
}
and the test passes an array instead of a string:
$chat->say(array());
PHP does not reach the code:
throw new \InvalidArgumentException('Message must be a string');
because the exception is thrown earlier due to the type typing string
:
public function say(string $msg)
therefore the TypeError
is thrown instead of InvalidArgumentException
exception of type "TypeError" again
Knowing that we don't need if (!is_string($msg))
for checking data type because PHP already takes care about that if we specify the data type in the method declaration say(string $msg)
we may want to throw InvalidArgumentException
if the message is too long if (strlen($msg) > 3)
.
<?php
declare(strict_types=1);
class DatatypeChat
{
public function say(string $msg)
{
if (strlen($msg) > 3) {
throw new \InvalidArgumentException('Message is too long');
}
return "Hello $msg";
}
}
class ExceptionTest extends \PHPUnit\Framework\TestCase
{
public function testSayTooLong(): void
{
$this->expectException(\Exception::class);
$chat = new DatatypeChat;
$chat->say('I have more than 3 chars');
}
public function testSayDataType(): void
{
$this->expectException(\Exception::class);
$chat = new DatatypeChat;
$chat->say(array());
}
}
Modifying also ExceptionTest
so we have two cases (test methods) where an Exception
should be thrown - first testSayTooLong
when the message is too long and second testSayDataType
when the message is a wrong type.
In both tests we expect instead of a specific exception class like InvalidArgumentException
or TypeError
just a generic Exception
class by using
$this->expectException(\Exception::class);
the test result is:
✔ Say too long
✘ Say data type
├ Failed asserting that exception of type "TypeError" matches expected exception "Exception". Message was: "Argument 1 passed to DatatypeChat::say() must be of the type string, array given (..)
FAILURES!
Tests: 2, Assertions: 2, Failures: 1.
testSayTooLong()
expecting a generic Exception
and using
$this->expectException(\Exception::class);
passes with OK
when the InvalidArgumentException
is thrown
but
testSayDataType()
using the same $this->expectException(\Exception::class);
Fails
with the description:
Failed asserting that exception of type "TypeError" matches expected exception "Exception".
It looks confusing that PHPUnit complained that exception TypeError
was not an Exception
, otherwise it would not have had any problem with $this->expectException(\Exception::class);
inside the testSayDataType()
as it did not have any problem with the testSayTooLong()
throwing InvalidArgumentException
and expecting: $this->expectException(\Exception::class);
The problem is that PHPUnit misleads you with the description as above because TypeError
is not an exception. TypeError
does not extends from the Exception
class nor from any other its children.
TypeError
implements Throwable
interface see documentation
whereas
InvalidArgumentException
extends LogicException
documentation
and LogicException
extends Exception
documentation
thus InvalidArgumentException
extends Exception
as well.
That's why throwing the InvalidArgumentException
passes test with OK and $this->expectException(\Exception::class);
but throwing TypeError
will not (it does not extend Exception
)
However both Exception
and TypeError
implement Throwable
interface.
Therefore changing in both tests
$this->expectException(\Exception::class);
to
$this->expectException(\Throwable::class);
makes test green:
✔ Say too long
✔ Say data type
OK (2 tests, 2 assertions)
See the list of Errors and Exception classes and how they are related to each other.
Just to be clear: it is a good practice to use a specific exception or error for unit test instead of a generic Exception
or Throwable
but if you ever encounter that misleading comment about exception now you will know why PHPUnit's exception TypeError
or other exception Errors are not in fact Exception
s but Throwable
For PHPUnit 5.7.27 and PHP 5.6 and to test multiple exceptions in one test, it was important to force the exception testing. Using exception handling alone to assert the instance of Exception will skip testing the situation if no exception occurs.
public function testSomeFunction() {
$e=null;
$targetClassObj= new TargetClass();
try {
$targetClassObj->doSomething();
} catch ( \Exception $e ) {
}
$this->assertInstanceOf(\Exception::class,$e);
$this->assertEquals('Some message',$e->getMessage());
$e=null;
try {
$targetClassObj->doSomethingElse();
} catch ( Exception $e ) {
}
$this->assertInstanceOf(\Exception::class,$e);
$this->assertEquals('Another message',$e->getMessage());
}
PhpUnit is an amazing library, but this specific point is a bit frustrating. This is why we can use the turbotesting-php opensource library which has a very convenient assertion method to help us testing exceptions. It is found here:
And to use it, we would simply do the following:
AssertUtils::throwsException(function(){
// Some code that must throw an exception here
}, '/expected error message/');
If the code we type inside the anonymous function does not throw an exception, an exception will be thrown.
If the code we type inside the anonymous function throws an exception, but its message does not match the expected regexp, an exception will also be thrown.