Does ITest work when we use TestNG dataproviders rather than Factory

Question

I am trying to use the ITest interface to get custom names (for Jenkins reports) for my test coming from a data-provider. I tried a sample program like below to understand at what points the gettestname() method gets called by TestNG.

Pasting the output and the program.

As you can see below the output is confusing and there are places where the test name is coming as 'null' too. So my Q is whether Itest is designed to be used with a dataprovider ? Also when we run the below dataprovider using parallel=true, it gets even more confusing as the member 'testName' is shared across threads and only one instance of class is running.

I can see how ITest can be used with a @factory as every testmethod run will invoke a new TestClass when we are using factories, so it would not be a problem there.

Output:

TestNGThreadingTest:31 - DataProvider running on Thread id = 1
TestNGThreadingTest:53 - GetTestname Method Thread id = 1 TestName = null
TestNGThreadingTest:43 - Before Method Thread id = 1 TestName = test1
TestNGThreadingTest:53 - GetTestname Method Thread id = 1 TestName = test1
TestNGThreadingTest:37 - Run Method Thread id = 1 TestName = test1
TestNGThreadingTest:53 - GetTestname Method Thread id = 1 TestName = test1
TestNGThreadingTest:48 - After Method Thread id = 1 TestName = test1
TestNGThreadingTest:53 - GetTestname Method Thread id = 1 TestName = test1
TestNGThreadingTest:43 - Before Method Thread id = 1 TestName = test2
TestNGThreadingTest:53 - GetTestname Method Thread id = 1 TestName = test2
TestNGThreadingTest:37 - Run Method Thread id = 1 TestName = test2
TestNGThreadingTest:53 - GetTestname Method Thread id = 1 TestName = test2
TestNGThreadingTest:48 - After Method Thread id = 1 TestName = test2
TestNGThreadingTest:53 - GetTestname Method Thread id = 1 TestName = test2
TestNGThreadingTest:43 - Before Method Thread id = 1 TestName = test3
TestNGThreadingTest:53 - GetTestname Method Thread id = 1 TestName = test3
TestNGThreadingTest:37 - Run Method Thread id = 1 TestName = test3
TestNGThreadingTest:53 - GetTestname Method Thread id = 1 TestName = test3
TestNGThreadingTest:48 - After Method Thread id = 1 TestName = test3
TestNGThreadingTest:53 - GetTestname Method Thread id = 1 TestName = test3
TestNGThreadingTest:43 - Before Method Thread id = 1 TestName = test4
TestNGThreadingTest:53 - GetTestname Method Thread id = 1 TestName = test4
TestNGThreadingTest:37 - Run Method Thread id = 1 TestName = test4
TestNGThreadingTest:53 - GetTestname Method Thread id = 1 TestName = test4
TestNGThreadingTest:48 - After Method Thread id = 1 TestName = test4
TestNGThreadingTest:53 - GetTestname Method Thread id = 1 TestName = test4
TestNGThreadingTest:43 - Before Method Thread id = 1 TestName = test5
TestNGThreadingTest:53 - GetTestname Method Thread id = 1 TestName = test5
TestNGThreadingTest:37 - Run Method Thread id = 1 TestName = test5
TestNGThreadingTest:53 - GetTestname Method Thread id = 1 TestName = test5
TestNGThreadingTest:48 - After Method Thread id = 1 TestName = test5
TestNGThreadingTest:53 - GetTestname Method Thread id = 1 TestName = test5
TestNGThreadingTest:53 - GetTestname Method Thread id = 1 TestName = test5
TestNGThreadingTest:53 - GetTestname Method Thread id = 1 TestName = test5
TestNGThreadingTest:53 - GetTestname Method Thread id = 1 TestName = test5
TestNGThreadingTest:53 - GetTestname Method Thread id = 1 TestName = test5
PASSED: test1("test1")
PASSED: test2("test2")
PASSED: test3("test3")
PASSED: test4("test4")
PASSED: test5("test5")

Output [when parallel=true]

TestNGThreadingTest:31 - DataProvider running on Thread id = 1
TestNGThreadingTest:53 - GetTestname Method Thread id = 9 TestName = null
TestNGThreadingTest:53 - GetTestname Method Thread id = 11 TestName = null
TestNGThreadingTest:53 - GetTestname Method Thread id = 10 TestName = null
TestNGThreadingTest:53 - GetTestname Method Thread id = 13 TestName = null
TestNGThreadingTest:53 - GetTestname Method Thread id = 12 TestName = null
TestNGThreadingTest:43 - Before Method Thread id = 11 TestName = test3
TestNGThreadingTest:43 - Before Method Thread id = 13 TestName = test5
TestNGThreadingTest:43 - Before Method Thread id = 9 TestName = test1
TestNGThreadingTest:43 - Before Method Thread id = 12 TestName = test4
TestNGThreadingTest:43 - Before Method Thread id = 10 TestName = test2
TestNGThreadingTest:53 - GetTestname Method Thread id = 12 TestName = test2
TestNGThreadingTest:53 - GetTestname Method Thread id = 9 TestName = test2
TestNGThreadingTest:53 - GetTestname Method Thread id = 13 TestName = test2
TestNGThreadingTest:53 - GetTestname Method Thread id = 11 TestName = test2
TestNGThreadingTest:53 - GetTestname Method Thread id = 10 TestName = test2
TestNGThreadingTest:37 - Run Method Thread id = 11 TestName = test3
TestNGThreadingTest:37 - Run Method Thread id = 12 TestName = test4
TestNGThreadingTest:37 - Run Method Thread id = 9 TestName = test1
TestNGThreadingTest:37 - Run Method Thread id = 10 TestName = test2
TestNGThreadingTest:37 - Run Method Thread id = 13 TestName = test5
TestNGThreadingTest:53 - GetTestname Method Thread id = 12 TestName = test2
TestNGThreadingTest:53 - GetTestname Method Thread id = 11 TestName = test2
TestNGThreadingTest:53 - GetTestname Method Thread id = 9 TestName = test2
TestNGThreadingTest:53 - GetTestname Method Thread id = 10 TestName = test2
TestNGThreadingTest:48 - After Method Thread id = 12 TestName = test4
TestNGThreadingTest:48 - After Method Thread id = 10 TestName = test2
TestNGThreadingTest:48 - After Method Thread id = 9 TestName = test1
TestNGThreadingTest:53 - GetTestname Method Thread id = 13 TestName = test2
TestNGThreadingTest:48 - After Method Thread id = 11 TestName = test3
TestNGThreadingTest:48 - After Method Thread id = 13 TestName = test5
TestNGThreadingTest:53 - GetTestname Method Thread id = 1 TestName = test2
TestNGThreadingTest:53 - GetTestname Method Thread id = 1 TestName = test2
TestNGThreadingTest:53 - GetTestname Method Thread id = 1 TestName = test2
TestNGThreadingTest:53 - GetTestname Method Thread id = 1 TestName = test2
TestNGThreadingTest:53 - GetTestname Method Thread id = 1 TestName = test2
PASSED: test2("test3")
PASSED: test2("test4")
PASSED: test2("test1")
PASSED: test2("test2")
PASSED: test2("test5")

Code:

public class TestNGThreadingTest implements ITest {

public String testName;

final static Logger logger = Logger.getLogger(TestNGThreadingTest.class);

@DataProvider(name="dp")
public Iterator<Object[]> getTests() {
    Collection<Object[]> tests = new ArrayList<Object[]>();

    tests.add(new Object[]{"test1"});
    tests.add(new Object[]{"test2"});
    tests.add(new Object[]{"test3"});
    tests.add(new Object[]{"test4"});
    tests.add(new Object[]{"test5"});

    logger.info("DataProvider running on Thread id = " + Thread.currentThread().getId());
    return tests.iterator();
}

@Test(dataProvider="dp")
public void run(String testName) {
    logger.info("Run Method Thread id = " + Thread.currentThread().getId() + " TestName = "+ testName );
}

@BeforeMethod
public void init(Object[] testArgs) {
    this.testName = (String) testArgs[0];
    logger.info("Before Method Thread id = " + Thread.currentThread().getId() + " TestName = " + testArgs[0]);
}

@AfterMethod
public void tearDown(Object[] testArgs) {
    logger.info("After Method Thread id = " + Thread.currentThread().getId() + " TestName = " + testArgs[0]);
}

@Override
public String getTestName() {
    logger.info("GetTestname Method Thread id = " + Thread.currentThread().getId() + " TestName = " + testName);
    return testName;
}

}

I don't see any problems here. What is the expected behavior? — juherr
my question is whether we can use this ITest feature of giving custom names along with dataproviders ?.......I can understand that they function properly with @factory as for each entry a new instance of test class is used. Also I see that a call is made to the gettestname() before the beforeMethod() in which i initialize the test name. And there are a few additional calls before the aftermethod....which calls is exactly used by TestNG to determine the name of a test ? — sandee609
Your output shows us that ITest is working with data provider: PASSED: test1("test1"). Without ITest, the display won't be the same. — juherr
ah, my bad.....however I was running with parallel=true in the dataprovider annotation and the result is different. I updated the question section with the output. — sandee609
Ok, I see the problem. But now, we need all the output to see when testng is calling what ;) — juherr

juherr juherr · Accepted Answer · 2015-10-30T20:08:33

It's a concurency issue (maybe a TestNG one?).

Remplacing String testName by ThreadLocal<String> testName = new ThreadLocal() should fix your problem.

Or you can use a @Factory linked to the @DataProvider too, but @Factory may not work in parallel.

Does ITest work when we use TestNG dataproviders rather than Factory

1 Answers