junit plugin not showing results from all tests in jenkins

Question

I have a test suite, where some of the XML files are generated after running unit tests (junit). And there are some other protractor automation tests which uses jasmine junit xml reporter (https://github.com/larrymyers/jasmine-reporters) to create XML file. I am generating both sets of XML files in same directory in my jenkins job, I have used junit plugin (Publish JUnit test result report) to publish report. So, now, after the job completes, 9 xmls are generated from unit tests and 2 XMLs are generated by jasmine-reporter, so as stated below from my jenkins job console output, it is able to recognize that there are 11 xml files from the pattern I have specified in job config.

22:49:52 [xUnit] [INFO] - [JUnit] - 11 test report file(s) were found with the 
pattern '**/tests/target/failsafe-reports/junitreports/*.xml' relative to 
'/home/jenkins/workspace/projectName' for the testing framework 
'JUnit'.

But, problem is, when i click on "Test Result" link from jenkins job UI, it only shows all tests related to xml generated by unit test, and it doesn't show any result related to XML which was generated from jasmine report. Need help to know why jenkins is not showing any test result related to jasmine XML.

Below is a sample XML generated by unit test (which jenkins is showing in ui, when test result link is clicked).

<!--
Generated by org.testng.reporters.JUnitReportReporter 
-->
<testsuite hostname="someHostName" 
name="packageName.Class1" tests="22" failures="0" timestamp="16 
Apr 2018 05:47:59 GMT" time="69.663" errors="0">
  <testcase name="test1" time="2.488" 
classname="packageName.Class1"/>
  <testcase name="test2" time="5.808" 
classname="packageName.Class1"/>
</testsuite>

Below is a sample XML generated by jasmine junit xml reporter (result of this is not shown by jenkins in it's UI, when clicking test result link)

<testsuites>
  <testsuite name="chromeTestItemDetailsPage" timestamp="2018-04-16T05:48:43" 
hostname="localhost" time="29.357" errors="0" tests="22" skipped="0" 
disabled="0" failures="0">
   <testcase classname="chromeTestItemDetailsPage" name="ItemTitleDisplayed" 
time="0.895"/>
   <testcase classname="chromeTestItemDetailsPage" name="ItemPriceDisplayed" 
time="0.966"/>
 </testsuite>
</testsuites>

Can you please try to let Jenkins parse the jasmine JUnit XML reports only. Do they then show up in the WebUI? I see a subtle difference in your example XML reports between the unit tests and the jasmine report, but I am not sure, whether that is the root cause. — Blue
If I don't run the unit tests and just run the protractor tests, then jenkins is able to show the report in web ui using the xml created by jasmine reporter. — stackoverflow

Blue Blue · Accepted Answer · 2018-04-21T22:52:17

I think you are encountering a Jenkins issue here.

Jenkins uses the attributes classname as well as testname of a <testcase> within a JUnit XML report to classify test results on the UI. Furthermore Jenkins expects the classname to be composed of a package name followed by a single dot and then a class name:

<testcase classname="packagename.classname" name="testname">

If no dot is present, Jenkins assumes the package "root". If multiple dots are present, only the last dot is recognized as separator. See this question for a better explanation.

If you look at your XML reports above you can notice, that the XML of the unit tests specify a package name in the classname attribute, while the XML of the jasmine reporter does not. I think the missing package name is the reason, why the test results are not showing up. Apparently Jenkins uses the package "root" only, if there are no packages present at all. If there is at least one test case with a package name, all other test cases without a package are "lost".

There are two possible solutions:

Report a bug for Jenkins since above behavior is unexpected
Modify the jasmine reporter to construct a classnameof the pattern package.class. There is a pull request at https://github.com/larrymyers/jasmine-reporters/pull/186/files that would allow modifications of the classname by a user function.

junit plugin not showing results from all tests in jenkins

1 Answers