python - Parse Robot Framework's output xml

Question

Robot framework spits out an output XML which is then used to build the HTML reports (with rebot), re-run failures, etc.

I need to parse this file to generate other reports, in particular, I need to parse the test case's documentation for a regex, get the test case result and then build a report (this is for integration with a legacy system).

Does Robot Framework provide functionality to easily parse the output XML file or we just use standard XML parsing libraries?

Answer 1

Ok, found 3 viable responses:

1. Use DbBot and query the DB created.
2. Parse the XML file directly. Using xml.etree.ElementTree is pretty simple.
3. Use Robot Framework's ExecutionResult, which follows the Visitor pattern (in ResultVisitor, which we have to extend) and that allows you to do what you need in the visit_test method.

Ended up with going with option 3 as that is part of Robot Framework and less likely to break (or easier to fix) if the format of the XML file changes.

Answer 2

I had a similar problem as yours and for me using the Robot Framework's Listener Interface was the most convenient solution.

The output_file method from the Listener version 3 API will be invoked when writing to an output file is ready. The argument of the method is the absolute path to the output XML file, and that is all needed to create any kind of new reports.

Example:

import os

"""Listener that parses the output XML when it is ready and creates a unique log."""

ROBOT_LISTENER_API_VERSION = 3

def output_file(path):
    # parse xml with etree or lxml
    log_dir = os.path.split(path)[0]
    print('Extra log: %s' % (log_dir + '\\extra.log'))

Console log of a test execution:

D:\robot_framework>robot --listener my_listener.py my_test.robot
==============================================================================
My Test
==============================================================================
Some Test                                                             | PASS |
------------------------------------------------------------------------------
My Test                                                               | PASS |
1 critical test, 1 passed, 0 failed
1 test total, 1 passed, 0 failed
==============================================================================
Extra log: D:\robot_framework\extra.log
Output:  D:\robot_framework\output.xml
Log:     D:\robot_framework\log.html
Report:  D:\robot_framework\report.html

The additional log file is listed among the default outputs.

Answer 3

This answer borrows from Bence's example, but adds functionality of echoing grand total summary lines.

#!/usr/bin/env python3

# invoke with robot --listener API-Libs/RF_Listener.py

import xml.etree.ElementTree as xmlElementTree

ROBOT_LISTENER_API_VERSION = 3

class RF_Listener:  # class should be same as filename

    def __init__(self):
        self.ROBOT_LISTENER_API_VERSION = 3

    def output_file(self, path):  # Listener that parses the output XML when it is ready
      root = xmlElementTree.parse(path).getroot()
      for type_tag in root.findall('./statistics/total/stat'):
      # <stat pass="1" fail="2">Critical Tests</stat>
      # <stat pass="3" fail="4">All Tests</stat>
        cntPassed = int(type_tag.attrib.get("pass"))  # attrib is dict-like (except for 'text')
        cntFailed = int(type_tag.attrib.get("fail"))
        cntTests = cntPassed + cntFailed
        pct_pass = cntPassed / cntTests * 100
        fmt_str = "{}: {} tests, {} passed, {} failed, {:.3g}% pass rate (--listener summary)"
        print(fmt_str.format(type_tag.text,cntTests, cntPassed, cntFailed,pct_pass))
      # optionally write grand total results summary to a file

This version prints to STDOUT instead of writing to a file, thus inserting these two lines near the end of the output. Suitable for grep (esp. for use in Jenkins job description-setter plugin.)

Critical Tests: 3 tests, 1 passed, 2 failed, 33.3% pass rate (--listener summary)
All Tests: 7 tests, 3 passed, 4 failed, 42.9% pass rate (--listener summary)