how to auto generate test cases for parameter-less methods?

Question

To reduce boiler plate code i came upon the idea to generate test cases in the class Tester for all parameter-less methods.

On running py.test, it only recognizes the statically written test cases (test_a, test_b) but not the dynamically created test cases using setattr(Tester,'test_' + name, member)

Perhaps py.test has already inspected the class Tester for methods with 'test_*' before setUpClass is called? Any hints how to get this running?

import inspect
import unittest


class Testee:
    def a(self):
        print('a')

    def b(self):
        print('b')    

    #...

    #...
    def z(self):
        print('z')

class Tester(unittest.TestCase):

    @classmethod
    def setUpClass(cls):
        testee = Testee()
        for name, member in inspect.getmembers(object=testee, predicate=inspect.ismethod or inspect.iscoroutine):
            if len(inspect.signature(member).parameters):
                print(str(inspect.signature(member).parameters))
                setattr(Tester,'test_' + name, member)
            if inspect.isfunction(member) or inspect.ismethod(member):
                setattr(Tester,'test_' + name, member)
            elif inspect.iscoroutinefunction(member):
                setattr(Tester,'test_' + name, functools.partialmethod(TestInstrument.run_coro, member))
            else:
                print(member)
        return super().setUpClass()

    def test_a(self):
        Tester.testee.a()

    def test_b(self):
        Tester.testee.b()

============================= test session starts ============================= platform win32 -- Python 3.5.1, pytest-2.9.2, py-1.4.31, pluggy-0.3.1 -- c:\program files\python35\python.exe cachedir: .cache rootdir: C:\tests, inifile: collected 2 items

sandbox.py::Tester::test_a PASSED sandbox.py::Tester::test_b PASSED

========================== 2 passed in 0.03 seconds ===========================

EDIT: If i move the code in setupClass to the global scope (outside the class), then py.test detects and runs the auto-generated test cases.

Can you please post where you're instantiating the test class? Also, please use cls instead of Tester in your setattr. — the_constant

goldcode goldcode · Accepted Answer · 2017-01-16T16:11:26

To elaborate on my 'EDIT', one option would be so. I'm unhappy with the solution because global code execution would be open to side-effects and other subtle bugs. any suggestions on how to get this into class Tester scope?

import inspect
import unittest
import functools
def auto_generate_test_cases(Tester, Testee):
    def run(self, fn):
        fn(Tester._testee)
    for name, member in inspect.getmembers(
        object=Testee, predicate=inspect.isfunction or inspect.iscoroutine):
        if len(inspect.signature(member).parameters) == 1:            
            setattr(Tester,'test_' + name, functools.partialmethod(run, member))

class Testee:
    def __init__(self):
        self._a = 'a'
    def a(self):
        print(self._a)
    def ab(self, a, b):
        print('a')
    def b(self):
        print('b')    
    def h(self):
        print('h')    
    async def q(self):
        print('async q')
    #...

    #...
    def z(self):
        print('z')

class Tester(unittest.TestCase):
    _testee = Testee()
auto_generate_test_cases(Tester, Testee)

py.test. output:

C:\tests>py.test sandbox.py --verbose
============================= test session starts =============================
platform win32 -- Python 3.5.1, pytest-2.9.2, py-1.4.31, pluggy-0.3.1 -- c:\program files\python35\python.exe
cachedir: .cache
rootdir: C:\\tests, inifile:
collected 5 items

sandbox.py::Tester::test_a PASSED
sandbox.py::Tester::test_b PASSED
sandbox.py::Tester::test_h PASSED
sandbox.py::Tester::test_q PASSED
sandbox.py::Tester::test_z PASSED

========================== 5 passed in 0.07 seconds ===========================

how to auto generate test cases for parameter-less methods?

1 Answers