PHPUnit reports different code coverage % for the same test when run in isolation

Question

First, let me say that I've been through Reaching 100% Code Coverage with PHPUnit^SO as well as How to get 100% Code Coverage with PHPUnit^SO. I've not been able to solve the issue with the aid of either.

My problem is that PHPUnit is reporting a different code coverage % for the same test when run in isolation as compared to when the entire test suite is run. So, when I run only the specific test in question using phpunit test/src/artax/UsesRequestTraitTest I get the following coverage:

enter image description here

However, if I run my full suite of tests using phpunit without specifying a specific test, I receive the following for the same file:

enter image description here

Here's the source code for the test:

<?php

class UsesRequestTraitTest extends PHPUnit_Framework_TestCase
{
  public function testIsInitiallyEmpty()
  {
    $traitObj = $this->getObjectForTrait('artax\UsesRequestTrait');
    $this->assertAttributeEmpty('request', $traitObj);
    return $traitObj;
  }

  /**
   * @depends testIsInitiallyEmpty
   * @covers artax\UsesRequestTrait::setRequest
   * @covers artax\UsesRequestTrait::getRequest
   */
  public function testSetterAssignsRequestProperty($traitObj)
  {
    $r = new artax\blocks\http\HttpRequest();
    $traitObj->setRequest($r);
    $this->assertEquals($r, $traitObj->getRequest());
  }
}

If you paid attention you'll see that I'm testing a PHP5.4 trait. I'm using PHP5.4RC6 and the bleeding edge 2.2 xdebug trunk version because the release version of xdebug doesn't yet support PHP5.4.

It's possible (likely?) that this is a bug in xdebug or phpunit that hasn't yet been worked out because of still-evolving support for the new PHP version, but I wanted to get other opinions before wasting the concerned parties' time with an unnecessary bug report. Does anyone have any idea what might cause this or how to correct it so my geek OCD doesn't drive me insane over this gap in 100% coverage?

P.S. Heres the actual graphical coverage report for the 60% coverage version. The lines shown as uncovered below are green (covered) when the test is run in isolation:

enter image description here

UPDATE

I've also tried adding @covers annotations for the trait methods to tests of concrete classes that use the trait, but this has not helped either ...

could it be that the coverage is for the last isolated test on that file only? — hakre
@hakre I'm not sure I understand what you're asking. For what it's worth, I experience the same behavior for another trait file that's identical except that it deals with a different type of object (than a $request) ... what's really strange is that there are two other trait files in the same format with the same tests where the full test suite reports the coverage correctly. I've looked over and over for the difference in the tests that would cause one to report correctly while another does not, but to no avail. — rdlowrey
If you run tests in isolation, I'm just thinking that the code-coverage is isolated, too. When it gets written, probably only the last coverage data is written to HTML (overwrites previous HTML dumps with the same file-name). That's what I meant. — hakre
@hakre I've been able to resolve the issue by using a concrete test class implementation instead of the new getObjectForTrait method and have posted an answer describing the fix, if you're interested. — rdlowrey

rdlowrey rdlowrey · Accepted Answer · 2012-02-01T17:35:58

This seems to be an issue with how PHPUnit's new getObjectForTrait method reports code coverage for a trait file. You can read about getObjectForTrait over at Sebastian Bergmann's blog.

The error disappears when I use a concrete implementation class for testing the trait in place of the built-in getObjectForTrait method. So, the following changes to the above posted code resolve the issue:

<?php

class UsesRequestTraitTest extends PHPUnit_Framework_TestCase
{
  public function testIsInitiallyEmpty()
  {
    // trait code coverage not reported correctly using getObjectForTrait
    //$traitObj = $this->getObjectForTrait('artax\UsesRequestTrait');

    // use a concrete implementation instead (see bottom for class definition)
    $traitObj = new TraitImplementationClass();
    $this->assertAttributeEmpty('request', $traitObj);
    return $traitObj;
  }

  /**
   * @depends testIsInitiallyEmpty
   * @covers artax\UsesRequestTrait::setRequest
   * @covers artax\UsesRequestTrait::getRequest
   */
  public function testSetterAssignsRequestProperty($traitObj)
  {
    $r = new artax\blocks\http\HttpRequest();
    $traitObj->setRequest($r);
    $this->assertEquals($r, $traitObj->getRequest());
  }
}

// using this class to test instead of getObjectForTrait resolves the issue
class TraitImplementationClass
{
  use artax\UsesRequestTrait;
}

PHPUnit reports different code coverage % for the same test when run in isolation

1 Answers