phpunit restarting tests randomly

73
votes

I am trying to test my symfony2 application using PHPUnit. I got one project where everything works as expected, but on my other project I have this strange behaviour that PHPUnit either stops executing the Test Suite randomly near the end of all tests and restarts or restarts the tests after finishing the Test Suite and writing the code coverage. Other times it runs normally.

Here is some output to make visible what is happening ( Test is restarting over and over):

PHPUnit 3.6.10 by Sebastian Bergmann.

Configuration read from C:\workspace\cllctr\app\phpunit.xml

................................................................. 65 / 83 ( 78%)
...........PHPUnit 3.6.10 by Sebastian Bergmann.

Configuration read from C:\workspace\cllctr\app\phpunit.xml

................................................................. 65 / 83 ( 78%)
...PHPUnit 3.6.10 by Sebastian Bergmann.

Configuration read from C:\workspace\cllctr\app\phpunit.xml

................................................................. 65 / 83 ( 78%)
............PHPUnit 3.6.10 by Sebastian Bergmann.

Configuration read from C:\workspace\cllctr\app\phpunit.xml

................................................................. 65 / 83 ( 78%)
............PHPUnit 3.6.10 by Sebastian Bergmann.

Configuration read from C:\workspace\cllctr\app\phpunit.xml

................................................................. 65 / 83 ( 78%)
..................

Time: 01:03, Memory: 43.00Mb

OK (83 tests, 145 assertions)

Writing code coverage data to XML file, this may take a moment.

Generating code coverage report, this may take a moment.

Here is an example of the Test Suite restarting after executing all tests:

PHPUnit 3.6.10 by Sebastian Bergmann.

Configuration read from C:\workspace\cllctr\app\phpunit.xml

................................................................. 65 / 83 ( 78%)
..................

Time: 01:29, Memory: 53.25Mb

OK (83 tests, 145 assertions)

Writing code coverage data to XML file, this may take a moment.

Generating code coverage report, this may take a moment.
PHPUnit 3.6.10 by Sebastian Bergmann.

Configuration read from C:\workspace\cllctr\app\phpunit.xml

................................................................. 65 / 83 ( 78%)
............PHPUnit 3.6.10 by Sebastian Bergmann.

As my other project runs without any problems, there must be some problem within my code. But I cannot figure out what possibly triggers this behaviour! The logs don't show nothing unexpected/strange.

EDIT

Yesterday, I noticed something strange: I decided to switch from MongoDB to MySQL because of some unrelated reasons. After the transition was done, all tests run without any problems. I tried it many times and I'm not able to reproduce it anymore. As this only happened with my functional tests, I tend to think that the problem was my WebTestCase class, which runs some commands to clear and rebuild the database. Maybe someone who also uses MongoDB can reproduce this behaviour?

1
unit-testingsymfonyphpunit
It looks like it's always happening in the same place, 65/83. Can you find out which test it's getting to? Try using the --verbose option when running. – Jon Cairns
65/83 means that in line 1, 65 tests where executed (each dot is one test). In the second line you see that the number of tests executed is not the same, so I doubt it's one test that's failing. As the tests sometimes work and sometimes loop, I doubt it's because of one test. But I will investigate further! – Sgoettschkes
My bad! I did have that knowledge somewhere in my brain. Here's a question - are you running each test in an isolated PHP process? That could be causing this unusual activity. We use the same version of PHPUnit as you all the time, and haven't seen anything as mad as what you're seeing! – Jon Cairns
phpunit --debug will show you which test it is running. – Rygu
phpunit --testdox will show you the test names - instead of dots. – Jens A. Koch

1 Answers

1
votes

I'd suggest to check the database servers connection limits and pools.

For example if you've got a max limit of 100 connections, and some of the tests leaves connections open ("leaks"), you'd hit the limits there.

That would also explain why sometimes it works and sometimes it hits the limit, as your database could handle other tasks simultaneously, so sometimes you hit the ceiling, other times when nothing else runs, you can successfully run your tests.

Check for persistent network connections and other external resources.