PHPUnit : Execute testSuites in specific order

Question

I have some testsuites (PHPUnit-Selenium, PHPUnit), I want to run them in a specific order. So I want to run "install (selenium driven)" test suite then "unit tests" test suite. I know I should avoid dependencies in Unit testing, but my question is not about this (I work with an old application with many dependencies, I need an installation and test this intallation with selenium then run unit tests, etc.).

So, I don't need a specific order for my "test cases", it's okay for this, but only for my "test suites". Here is my phpunit.xml configuration file :

<phpunit backupGlobals="false">

    <selenium>
      <browser name="Firefox" browser="*firefox" timeout="600" />
    </selenium>

    <testsuites>
        <testsuite name="install (selenium driven)">
            <file>./_install.php</file>
            <exclude>./bin</exclude>
        </testsuite>
        <testsuite name="unit tests">
            <directory>./</directory>
            <exclude>./selenium</exclude>
        </testsuite>
    </testsuites>
</phpunit>

So when I run phpunit, it seems it does not wait for "install (selenium driven)" result, so "unit tests" fails (MySQL error, but nevermind). How can I deal with this ? I would like two separate steps :

Run "install (selenium driven)" test suite
Run "unit tests" test suite

Other information :

I have many testCases files so I don't want to specify them
I know I should avoid backupGlobals=false too, I'm sorry, I can't :( ...
I know the bootstrap option, but I need to "test" installation (so it's a test suite)

Thanks a lot if you have a clue ! (I've tried to find a solution, but I did not find any for now...)

Maybe use a batch file to call each test suite one after the other, launching PHPUnit with command line switches or different XML configuration files to run just 1 test suite, so you could arrange the order manually. A pain to maintain, but would work for now. — Steven Scott

Darren Cook Darren Cook · Accepted Answer · 2013-11-26T00:32:05

Steven Scott's comment to use a batch file, and two configuration files, would be what I would do. It follows the Unix philosophy of using simple tools to string simple tools together, rather than one big complex tool that does everything.

Here is an alternative, based on a guess that "install" is not just doing functional tests, but also setting up the DB and other fixtures that the unit tests need. So, how about moving the functional tests to a setupBeforeClass() call inside your unit test suite. Use shell_exec() to run phpunit to run the functional tests, and if they failed, then call die or similar, so that the unit tests do not run.

P.S. You sound like you know what you are doing, but for anyone coming later: when you get chance, the unit tests should be refactored to use a mock DB, so that they can be run first, and frequently. Still keep your existing unit tests suite, but merge it into the functional test suite, as it sounds like that is where it belongs. In fact that (merging the two test suites then using explicit @depends everywhere) is another approach.

PHPUnit : Execute testSuites in specific order

1 Answers