0
votes

I am using dataprovider method and a test method (with ITestContext parameter in the test method). A simplified example is as follows :

@DataProvider(name="Dataprovider")
public Object[][] dataprovider(){

    return new Object[][]{{1},{2,},{3}};
}

@Test(dataProvider="Dataprovider")
public void test(int data, ITestContext itx){

    System.out.println(data);
    org.testng.Assert.assertEquals(data, 3);
}

My Retry class and RetryListener classes are below :

public class RetryListener implements IAnnotationTransformer {

@Override
public void transform(ITestAnnotation testannotation, Class testClass,
        Constructor testConstructor, Method testMethod) {

    IRetryAnalyzer retry = testannotation.getRetryAnalyzer();

    if (retry == null)  {
        testannotation.setRetryAnalyzer(Retry.class);
    }
}
}

public class Retry implements IRetryAnalyzer {

private static int retryCount = 0;
private int maxRetryCount = 1;

// Below method returns 'true' if the test method has to be retried else 'false' 
//and it takes the 'Result' as parameter of the test method that just ran
    public boolean retry(ITestResult result) {
        if (retryCount < maxRetryCount) {
            System.out.println("Retrying test " + result.getName() + " with status "
                    + getResultStatusName(result.getStatus()) + " for the " + (retryCount+1) + " time(s).");
            retryCount++;
            return true;
        }
        retryCount = 0;
        return false;
    }

    public String getResultStatusName(int status) {
        String resultName = null;
        if(status==1)
            resultName = "SUCCESS";
        if(status==2)
            resultName = "FAILURE";
        if(status==3)
            resultName = "SKIP";
        return resultName;
    }

}

Expected : When test fails, the retry is called by TestNG, then the dataprovider should return the same values to the test method.

Observed : Dataprovider returns the same value but test method doesn't run and the retry terminates and next test starts (new values will now be returned by dataprovider)

But my retry does not enter the test method ( It is not expecting the (int data, ITestContext itx) for test method). If I remove ITestContext, the retry works.

ITestContext is a must for maintaining the test case context. So how to perform retry along with keeping the ITestContext in the test method.

1
"my retry does not enter the test method" -> what does it mean?juherr
When retry is called, the data provider is called again with the same values to return to the test method, this time it doesn't run the test method, the test simply passes (retry passes) and goes over to next testAswin Suresh
Currently it will not compile because of the additional parameter ITestContext itx.Grzegorz Górkiewicz
@GrzegorzGórkiewicz you mean, it will not "run" correct ?Krishnan Mahadevan
Ok so there is no way to do it right? I will have to remove the ITestContext for the test method to run, is it?Aswin Suresh

1 Answers

0
votes

Within a @Test method that's powered by a data provider, I think its difficult to put the ITestContext as an explicit parameter and expect it to be injected by TestNG, because TestNG wouldn't know how to inject it (especially at which position).

Please see if the below would work for you (If you notice, I am now explicitly also passing in the ITestContext via the data provider )

import org.testng.IRetryAnalyzer;
import org.testng.ITestContext;
import org.testng.ITestResult;
import org.testng.annotations.DataProvider;
import org.testng.annotations.Test;

public class RetryWithDataProvider {
    @DataProvider(name = "Dataprovider")
    public Object[][] dataprovider(ITestContext ctx) {

        return new Object[][]{{1, ctx}, {2, ctx}, {3, ctx}};
    }

    @Test(dataProvider = "Dataprovider", retryAnalyzer = Retry.class)
    public void test(int data, ITestContext itx) {
        System.out.println(data);
        System.out.println("Current Test name is : " + itx.getName());
        org.testng.Assert.assertEquals(data, 3);
    }

    public static class Retry implements IRetryAnalyzer {

        private static int retryCount = 0;
        private int maxRetryCount = 1;

        public boolean retry(ITestResult result) {
            if (retryCount < maxRetryCount) {
                System.out.println("Retrying test " + result.getName() + " with status "
                        + getResultStatusName(result.getStatus()) + " for the " + (retryCount + 1) + " time(s).");
                retryCount++;
                return true;
            }
            retryCount = 0;
            return false;
        }

        String getResultStatusName(int status) {
            String resultName = null;
            if (status == 1)
                resultName = "SUCCESS";
            if (status == 2)
                resultName = "FAILURE";
            if (status == 3)
                resultName = "SKIP";
            return resultName;
        }
    }
}

Here's my console output

[TestNG] Running:
  /Users/krmahadevan/Library/Caches/IntelliJIdea2016.3/temp-testng-customsuite.xml
1
Retrying test test with status FAILURE for the 1 time(s).

Test ignored.
1

java.lang.AssertionError: expected [3] but found [1]
Expected :3
Actual   :1
 <Click to see difference>


    at org.testng.Assert.fail(Assert.java:94)
    at org.testng.Assert.failNotEquals(Assert.java:513)
    at org.testng.Assert.assertEqualsImpl(Assert.java:135)
    at org.testng.Assert.assertEquals(Assert.java:116)
    at org.testng.Assert.assertEquals(Assert.java:389)
    at org.testng.Assert.assertEquals(Assert.java:399)
    at com.rationaleemotions.stackoverflow.RetryWithDataProvider.test(RetryWithDataProvider.java:20)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:104)
    at org.testng.internal.Invoker.invokeMethod(Invoker.java:645)
    at org.testng.internal.Invoker.retryFailed(Invoker.java:998)
    at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1200)
    at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:129)
    at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:112)
    at org.testng.TestRunner.privateRun(TestRunner.java:756)
    at org.testng.TestRunner.run(TestRunner.java:610)
    at org.testng.SuiteRunner.runTest(SuiteRunner.java:387)
    at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:382)
    at org.testng.SuiteRunner.privateRun(SuiteRunner.java:340)
    at org.testng.SuiteRunner.run(SuiteRunner.java:289)
    at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:52)
    at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:86)
    at org.testng.TestNG.runSuitesSequentially(TestNG.java:1293)
    at org.testng.TestNG.runSuitesLocally(TestNG.java:1218)
    at org.testng.TestNG.runSuites(TestNG.java:1133)
    at org.testng.TestNG.run(TestNG.java:1104)
    at org.testng.IDEARemoteTestNG.run(IDEARemoteTestNG.java:72)
    at org.testng.RemoteTestNGStarter.main(RemoteTestNGStarter.java:127)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at com.intellij.rt.execution.application.AppMain.main(AppMain.java:147)

2
Retrying test test with status FAILURE for the 1 time(s).

Test ignored.
2

java.lang.AssertionError: expected [3] but found [2]
Expected :3
Actual   :2
 <Click to see difference>


    at org.testng.Assert.fail(Assert.java:94)
    at org.testng.Assert.failNotEquals(Assert.java:513)
    at org.testng.Assert.assertEqualsImpl(Assert.java:135)
    at org.testng.Assert.assertEquals(Assert.java:116)
    at org.testng.Assert.assertEquals(Assert.java:389)
    at org.testng.Assert.assertEquals(Assert.java:399)
    at com.rationaleemotions.stackoverflow.RetryWithDataProvider.test(RetryWithDataProvider.java:20)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:104)
    at org.testng.internal.Invoker.invokeMethod(Invoker.java:645)
    at org.testng.internal.Invoker.retryFailed(Invoker.java:998)
    at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1200)
    at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:129)
    at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:112)
    at org.testng.TestRunner.privateRun(TestRunner.java:756)
    at org.testng.TestRunner.run(TestRunner.java:610)
    at org.testng.SuiteRunner.runTest(SuiteRunner.java:387)
    at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:382)
    at org.testng.SuiteRunner.privateRun(SuiteRunner.java:340)
    at org.testng.SuiteRunner.run(SuiteRunner.java:289)
    at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:52)
    at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:86)
    at org.testng.TestNG.runSuitesSequentially(TestNG.java:1293)
    at org.testng.TestNG.runSuitesLocally(TestNG.java:1218)
    at org.testng.TestNG.runSuites(TestNG.java:1133)
    at org.testng.TestNG.run(TestNG.java:1104)
    at org.testng.IDEARemoteTestNG.run(IDEARemoteTestNG.java:72)
    at org.testng.RemoteTestNGStarter.main(RemoteTestNGStarter.java:127)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at com.intellij.rt.execution.application.AppMain.main(AppMain.java:147)

3

===============================================
Default Suite
Total tests run: 5, Failures: 2, Skips: 2
===============================================