Cucumber rerun failed tests and cocatenate results using a testNG runner for cucumber 4.x

Question

I am trying to execute a 2nd round of running only the failed scenarios in order to avoid false alarms because of environment/network/test unstabilities. I am wondering if there is way to run only a maven command that will perform the following: 1. run all the scenarios that my runner includes in its configuration 2. create the cucumber.json report for this run 3. create a list of the failed scenarios 4. run the list of the failed scenarios 5. change the result of the report created in 1rst run depending on the results of failed tests in 2nd run, so that at the end, only the scenarios that failed in both runs are marked as failed and also there are no duplicates in the final report for those tests that failed at 1rst run (i.e. 2nd run overides the result of 1st run).

I am using cucumber 4.4.0 and testNG runner.

I have tried to use @ExtendedCucumberOptions (http://mkolisnyk.github.io/cucumber-reports/extended-cucumber-runner) and the 1rst run was ok, but the 2nd run was never happened, throwing the following exception: "Method public void com.github.mkolisnyk.cucumber.runner.ExtendedTestNGRunner.feature(cucumber.api.testng.CucumberFeatureWrapper) requires a @DataProvider named : feature"

I found this issue: https://github.com/mkolisnyk/cucumber-reports/issues/138 So, it seems that this cucumber-reports library does not support later versions of cucumber and I cannot use @ExtendedCucumberOptions that seemed to provide exactly what I need.

===> My maven dependecies include:

<dependency>
    <groupId>com.github.mkolisnyk</groupId>
    <artifactId>cucumber-runner</artifactId>
    <version>1.3.4</version>
</dependency>


<dependency>
    <groupId>io.cucumber</groupId>
    <artifactId>cucumber-java</artifactId>
    <version>4.4.0</version>
</dependency>

<dependency>
    <groupId>io.cucumber</groupId>
    <artifactId>cucumber-testng</artifactId>
    <version>4.4.0</version>
</dependency>

<dependency>
    <groupId>io.cucumber</groupId>
    <artifactId>cucumber-junit</artifactId>
    <version>4.4.0</version>
</dependency>

===> And my runner class :

package runners;

import cucumber.api.CucumberOptions;
import cucumber.api.testng.AbstractTestNGCucumberTests;
import org.testng.annotations.DataProvider;


@CucumberOptions(
        features = {"src/test/resources/features/"},
        glue = {"stepDefinitions"},
        tags = {"@magic"},
        plugin = {
                "pretty"
                , "html:target/cucumber"
                , "json:target/cucumber/cucumber.json"
                , "rerun:target/cucumber/failedTests.txt"
        },
        monochrome = false
)


public class AreaRunner extends AbstractTestNGCucumberTests { 

        @Override
        @DataProvider(parallel = true)
        public Object[][] scenarios() {
                return super.scenarios();
        }

}

Is there any solution other than @ExtendedCucumberOptions?

Anoop Singh Anoop Singh · Accepted Answer · 2019-11-07T02:26:34

The reason the 2nd run never happened is because Retry functionality is applicable for JUnit runner only, as per https://github.com/mkolisnyk/cucumber-reports/issues/167.

Cucumber rerun failed tests and cocatenate results using a testNG runner for cucumber 4.x

2 Answers