Why is my custom test not being launched by PHPUnit?

Question

As described in the PHPUnit doc, you can implement PHPUnit\Framework\Test to write custom tests.

Sound great, but how can I launch those tests as part of my testsuite?

Given the following test directory:

+tests/
   |- NormalTestExtendingTestCaseTest.php
   |- CustomTestImplementingTest.php
phpunit.xml

And my phpunit.xml file:

<phpunit xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:noNamespaceSchemaLocation="https://schema.phpunit.de/6.3/phpunit.xsd"
         convertErrorsToExceptions="true"
         convertNoticesToExceptions="true"
         convertWarningsToExceptions="true"
>
    <testsuites>
        <testsuite name="My Test Suite">
            <directory>./tests</directory>
        </testsuite>
    </testsuites>
</phpunit>

Only NormalTestExtendingTestCaseTest.php is executed by PHPUnit.

---

CustomTestImplementingTest.php

<?php
use PHPUnit\Framework\TestCase;

class CustomTestImplementingTest implements PHPUnit\Framework\Test
{

    public function count()
    {
        return 1;
    }

    public function run(PHPUnit\Framework\TestResult $result = null)
    {
        if ($result === null) {
            $result = new PHPUnit\Framework\TestResult;
        }

        $result->startTest($this);
        // do something
        $result->endTest($this, $stopTime);

        return $result;
    }
}

Answer 1

Check that your classes has the correct inheritance, the correct name methods (starting with 'test'), and the correct namespace.

You maybe have to run the composer dump-autoload.

Answer 2

I found a solution for this problem:

1. The custom test class should implement a suite() method so PHPUnit run the resulting TestSuite (with our test(s))
2. The test class should declare a getter method getName() (even if it's not declared in the interface PHPUnit\Framework\Test or an exception is thrown at execution. Moreover, the name should not be null or an other exception is throw at execution.

So the final class is

<?php

use PHPUnit\Framework\TestResult;
use PHPUnit\Framework\TestSuite;
use SebastianBergmann\Timer\Timer;
use PHPUnit\Framework\Test;

class CustomTestImplementingTest implements Test {

    protected $name = '';

    /**
     * This method create a TestSuite for this Test
     *
     * @return TestSuite
     * @throws ReflectionException
     */
    public static function suite() {
        $classname = (new \ReflectionClass(static::class))->getShortName();
        $suite = new TestSuite();
        $suite->setName($classname);
        $suite->addTest(new static());
        return $suite;
    }

    /**
     * @return mixed
     */
    public function getName() {
        return $this->name;
    }

    /**
     * @param mixed $name
     */
    public function setName($name): void {
        $this->name = $name;
    }

    public function count() {
        return 1;
    }

    public function run(TestResult $result = null): TestResult {
        if ($result === null) {
            $result = new PHPUnit\Framework\TestResult;
        }
        Timer::start();
        $stopTime = null;
        $result->startTest($this);

        try {
            // do something
        } catch (PHPUnit\Framework\AssertionFailedError $e) {
            $result->addFailure($this, $e, $stopTime);
        } catch (Exception $e) {
            $result->addError($this, $e, $stopTime);
        } finally {
            $stopTime = Timer::stop();
        }

        $result->endTest($this, $stopTime);

        return $result;
    }
}