Call googletest test cases from Robot Framework

Question

In my current project I have to run test cases using the beautiful Robot Framework. Now, there are already some very detailed test cases, that are already implemented in Google-Test and cannot be rewritten to Robot Framework.

Now, in our continuous integration setting (Jenkins) it is necessary to start all test cases using the Robot Framework and also the Google-Test tests need to be triggered by Robot Framework.

Is there an easy way to call all tests in google test using a Robot Framework extension library or is it necessary to write one?

In case, that there is no such extension, what would be good way to start writing some.

Answer 1

gtest based tests are just executable binaries. Robot Framework has builtin library Process that can be used to run external processes. Documentation here.

So, essentially you could just do use Run Process keyword to trigger the execution of the gtest binary.

Answer 2

If you would like to see the individual tests in Robot Framework you should process the stdout of the Google Test, which is available in the returned object of the Run Process keyword.

In Robot Framework, there are ways to dynamically generate test cases in runtime. The idea below was inspired and it is based on this blog post: Dynamically create test cases with Robot Framework.

Disclaimer, it is not a full solution as it does not handle every issues that could happen during building or running a gtest binary. For example build errors, segmentation faults, etc. Also the effort required might not worth the end result. It would also need continuous maintenance and development to follow changes in Google Test framework. So this is only a small example with demonstrative and inspirational purposes.

gtest.py is a library that also acts as a listener. This way it can have a start_suite method that will be invoked and it will get the suite(s) as Python object(s), robot.running.model.TestSuite. Then you could use this object along with Robot Framework's API to create new test cases (or remove existing ones). (With multiple gtest classes, it is possible to create new child suites per gtest class as well, so the Robot Framework test structure could follow the original gtest structure.)

from robot.running.model import TestSuite
from robot.libraries.BuiltIn import BuiltIn
import re


class TestCase():

    TESTNAME_PATTERN = re.compile(r"\[ RUN      \]\s(.*)\.(.*)")

    def __init__(self):
        self.name = None
        self.log = ''
        self.status = False

    def set_name(self, line):
        match = TestCase.TESTNAME_PATTERN.search(line)
        if match:
            self.name = f'{match.group(1)}.{match.group(2)}'
            self.log = line
        else:
            raise RuntimeError("Error parsing unit test name")


    def set_status(self, lastline):
        if 'OK' in lastline:
            self.status = True


class gtest(object):
    ROBOT_LISTENER_API_VERSION = 3
    ROBOT_LIBRARY_SCOPE = 'GLOBAL'
    ROBOT_LIBRARY_VERSION = 0.1

    def __init__(self):
        self.ROBOT_LIBRARY_LISTENER = self
        self.top_suite = None
        self.cases = []

    def _start_suite(self, suite, result):
        self.top_suite = suite

    def add_test_cases(self, gtest_output):
        self.top_suite.tests.clear() # remove placeholder test
        case = TestCase()
        case_started = False

        for line in gtest_output.splitlines():

            if line.startswith('[ RUN'):
                case.set_name(line)
                case_started = True
            elif case_started and (line.startswith('[       OK ]') or line.startswith('[  FAILED  ]')):
                case.log += f'\n{line}'
                case.set_status(line)
                self.cases.append(case)
                case = TestCase()
                case_started = False
            elif case_started:
                case.log += f'\n{line}'

        for case in self.cases:
            self._add_test_case(case)

    def _add_test_case(self, case):
        tc = self.top_suite.tests.create(name=case.name)
        tc.keywords.create(name="Test", args=[case])

    def test(self, case):
        if case.status is False:
            BuiltIn().fail(case.log)
        else:
            BuiltIn().log(case.log)


globals()[__name__] = gtest

The library is used in the following way, (all gtest related task should be implemented in the Suite Setup phase),

*** Settings ***
Library         gtest
Variables       gtest_output.py                # should come from Run Process keyword's output
Suite Setup     Add Test Cases    ${stdout}    # build and run gtest binaries here

*** Test Cases ***
Placeholder
    [Documentation]    This is needed to avoid empty suite error.
    ...    It will be removed during the run.
    No Operation

and provides the following outputs.

The input was a string literal as the focus was on the parsing and dynamic test cases creation rather than on building/running gtests itself.

gtest_output.py:

stdout= \
"""
Running main() from user_main.cpp
[==========] Running 2 tests from 1 test case.
[          ] Global test environment setup.
[          ] 2 tests from SquareRootTest
[ RUN      ] SquareRootTest.PositiveNos
../user_sqrt.cpp(6862): error: Value of: sqrt (2533.310224)
Actual: 50.332
Expected: 50.3321
[  FAILED  ] SquareRootTest.PositiveNos (9 ms)
[ RUN      ] SquareRootTest.ZeroAndNegativeNos
[       OK ] SquareRootTest.ZeroAndNegativeNos (0 ms)
[          ] 2 tests from SquareRootTest (0 ms total)

[          ] Global test environment teardown
[==========] 2 tests from 1 test case ran. (10 ms total)
[  PASSED  ] 1 test.
[  FAILED  ] 1 test, listed below:
[  FAILED  ] SquareRootTest.PositiveNos

1 FAILED TEST
"""