Creating copy of Robot Framework output.xml in Jenkins pipeline

Question

I am in the process of creating a Groovy email template for a Jenkins pipeline running Robot Framework tests. I intend to use Groovy's XMLSlurper to parse the output.xml created by Jenkins to extract the information I need. However, the template also relies on using Robot Publisher which I've now realized automatically deletes the output.xml. I would rather not have to archive the artifacts and access them that way, so is there a way to create a copy of the output.xml in the Jenkins pipeline before the Robot Publisher stage, that will not be deleted by Robot Publisher, that I can parse in my email stage?

Please bear with me as I'm relatively new to Jenkins (and stackoverflow for that matter), so apologies if I've excluded vital information, but any ideas would be much appreciated! Thanks

Hi and welcome to SO. Please read How to ask then try to improve your question by editing it, focusing on a specific problem and giving details about it (code, configuration, error messages, exact problem your are facing. — Sourav Atta

Bence Kaulics Bence Kaulics · Accepted Answer · 2020-09-19T12:26:28

I would approach your problem from a different angle. First of all I do not suggest using Groovy's XMLSlurper or any other XML parser to extract the information you need from Robot Framework's output.xml.

What you should use is Robot Framework's own API that already implements the parsers you need. You could easily access any information described in the robot.result.model module. You can find everything here, suites, tests and keywords with all thier attributes like, test messages, failure messages, execution times, test results, etc. All in all this would be the most future proof parsing solution as this parser will always match the version of the framework. Make sure to use the API documentation that matches your current framework version.

Now back to your task, you should utilize the above mentioned API via Robot Framework's listener interface. Implementing the output_file listener method you can access the output.xml (you can even make a copy of it here) file before the Robot Publisher plugin moves the file. The output_file will be automatically called once the output.xml is ready. The method will get the path to the xml file as an input. You can pass this path straight to the ExecutionResult class from the API, then you could "visit" the results by your ResultVisitor and acquire the information needed.

Last step would be to write the data into a file that would serve as an input to your e-mail stage. Note that this file won't be touched by the Robot Publisher by default as it is not a standard output, but a custom you just made using Robot Framework's API.

As it could sound a lot, here is an example to demonstrate the idea. The listener and the result visitor in EmailInputProvider.py:

from robot.api import ExecutionResult, ResultVisitor

class MyTestResultVisitor(ResultVisitor):

    def __init__(self):
        self.test_results = dict()

    def visit_test(self, test):
        self.test_results[test.longname] = test.status

        
class EmailInputProvider:

    ROBOT_LISTENER_API_VERSION = 3

    def output_file(self, path):
        output = 'EmailInput.txt'
        
        visitor = MyTestResultVisitor()    # Instantiate result visitor
        result = ExecutionResult(path)     # Parse up execution result using robot API
        result.visit(visitor)              # Visit and top level suite to retrive needed metadata
        
        with open(output, 'w') as f:       # Write retrived data into a file
            for testname, result in visitor.test_results.items():
                print(f'{testname} - {result}', file=f)

        # You can make a copy of the output.xml here as well
        print(f'Email:   Input saved into {output}') # Log about custom output to console

globals()[__name__] = EmailInputProvider

This would give the following results for this dummy suite (SO2.robot):

*** Test Cases ***
Test A
    No Operation

Test B
    No Operation

Test C
    No Operation

Test D
    No Operation

Test E
    No Operation

Test F
    Fail

Console output:

$ robot --listener EmailInputProvider SO2.robot
==============================================================================
SO2
==============================================================================
Test A                                                                | PASS |
------------------------------------------------------------------------------
Test B                                                                | PASS |
------------------------------------------------------------------------------
Test C                                                                | PASS |
------------------------------------------------------------------------------
Test D                                                                | PASS |
------------------------------------------------------------------------------
Test E                                                                | PASS |
------------------------------------------------------------------------------
Test F                                                                | FAIL |
AssertionError
------------------------------------------------------------------------------
SO2                                                                   | FAIL |
6 critical tests, 5 passed, 1 failed
6 tests total, 5 passed, 1 failed
==============================================================================
Email:   Input saved into EmailInput.txt
Output:  ..\output.xml
Log:     ..\log.html
Report:  ..\report.html

Custom output file:

SO2.Test A - PASS
SO2.Test B - PASS
SO2.Test C - PASS
SO2.Test D - PASS
SO2.Test E - PASS
SO2.Test F - FAIL

Creating copy of Robot Framework output.xml in Jenkins pipeline

1 Answers