How to compare jenkins results from different jobs

3
votes

In my current project I'm testing one web application against different browsers. so I've got one selenium/maven project and 16 jobs in jenkins: tests in IE8, 9, FF, Chrome, windows 7, windows xp, 32 and 64 bit environments

Jobs results are generating sourcefire reports and I've got nice and clean overall charts.

But the problem is comparison of single tests results: for example I'd like to know which tests are failing under Internet Explorer and not in Firefox or which browser is stable and which is not.

Currently I'm forced to open X windows with different results and manually compare results

enter image description here

is there a plugin that compares results? or my jenkins configuration is wrong?

2
jenkinscompare

2 Answers

1
votes

You should probably be using the "matrix" job type, called "Build multi-configuration project" in the New Job page.

There you configure the job once, and it gets executed automatically multiple times across all the combinations of axes you supply

I'm not sure about the specifics of Selenium with Jenkins, but you would probably want to set up two different axes:

  • User-defined axis: browser (FF, IE...)
  • Slave axis: build slave (XP_32, XP_64, W7_32...)

Overall, documentation is a bit thin for this feature, but see this answer for some info:
https://stackoverflow.com/a/424343/234938

0
votes

Have you tried Test Results Analyzer plugin in Jenkins?

https://wiki.jenkins-ci.org/display/JENKINS/Test+Results+Analyzer+Plugin