Driver behavior executing parallel TestNG selenium tests with dataprovider

Question

I want to run selenium tests in TestNg in parallel that use the @dataprovider. Ideally tests are parallel by method (one test = one method) and not simple suite parallelism by browser. I have read somewhere that about 5 instances of ChromeDriver can be controlled at a time so I thought this should be possible. Later I plan to move to grid2. For developement I'm running things with IntelliJ Idea test runner by right-click + run on the XML config file.

I had problems running my tests in parallel (on grid2 and locally) so I created a sample of more or less what I want to do.

Here is my test class

package tests;

import org.openqa.selenium.By;
import org.openqa.selenium.Keys;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.firefox.FirefoxDriver;
import org.openqa.selenium.interactions.Actions;
import org.testng.annotations.*;
import java.util.concurrent.TimeUnit;
import static org.testng.Assert.assertNotNull;

public class ParallelTest {
    public static final String SEARCH_TERMS = "search-terms";
    private WebDriver driver;

    @BeforeMethod
    @Parameters({"browser"})
    public void beforeMethod(@Optional("chrome") String browser){
        driver = getBrowser(browser);
        driver.manage().deleteAllCookies();
        driver.manage().timeouts().implicitlyWait(10, TimeUnit.SECONDS);
    }

    private WebDriver getBrowser(String browser) {
        if(browser.equals("chrome")){
            System.setProperty("webdriver.chrome.driver", "webdrivers\\chromedriver.exe");
            return new ChromeDriver();
        }
        return new FirefoxDriver();
    }

    @AfterMethod
    public void afterMethod(){
        driver.quit();
    }

    @Test(description = "Check parallel selenium works.",
          dataProvider = SEARCH_TERMS)
    public void parallelSeleniumTest(String searchTerm){
        driver.get("http://google.com");
        WebElement search = driver.findElement(By.id("gbqfq"));
        new Actions(driver)
                .sendKeys(search, searchTerm)
                .sendKeys(search, Keys.ENTER)
                .perform();
        String firstResult = driver.findElements(By.className("r")).get(0).getText();
        assertNotNull(firstResult);
        System.out.println(firstResult);
    }

    @DataProvider(name = SEARCH_TERMS, parallel = true)
    public Object[][] getSearchTerms(){
        return new Object[][]{
                {"google"},
                {"microsoft"},
                {"facebook"},
                {"amazon"},
                {"apple"},
                {"oracle"},
                {"yahoo"},
                {"jetbrains"},
                {"intellij idea"},
                {"selenium"},
                {"java"},
                {"testng"},
                {"code"}
        };
    }
}

I threw in some native events since I use them heavily in my test suite.

And here is the TestNg xml config file

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite thread-count="4" name="vfr6-ui-tests" parallel="methods">
    <test name="parallel-test-firefox">
        <parameter name="browser" value="firefox"/>
        <classes>
            <class name="tests.ParallelTest"/>
        </classes>
    </test>
    <test name="parallel-test-chrome">
        <parameter name="browser" value="chrome"/>
        <classes>
            <class name="tests.ParallelTest"/>
        </classes>
    </test>
</suite>

I read instantiating one driver per test tends to be the most maintainable. The problem is that the firefox test runs in serial while the chrome test spits out all of the data points as test cases, attempts to open a trove of browser instances, then everything fails. My tests will have either 10-25 or 300-500 data points (cycling between either clients or clients x products).

What is the best way to set up the driver, dataprovider, and test runner to achieve the best parallelism in running tests?

Amit Chaudhari Amit Chaudhari · Accepted Answer · 2013-09-24T05:56:46

I had the same experience about dataProvider. In my case I used dataProvider's (parallel=true) attribute though. There are two solutions to your problem.

Use dataProvider and in test class and use factory annotation for your constructor. In the factory annotation's attribute, use dataProvider="Your dataProvider's name". In the testng.xml, instead of parallel=methods, use parallel=instances.

The drawback of the above approach is that when you get the report; may be it is maven- surefire, testng Eclipse report or reportNG report, you do not see parameters passed up front. To overcome this, you can use the following approach.
Create a factory class and instantiate your test class in the factory method using a for loop. (Start for loop from 0.) In the test class define a constructor which receives a parameter from factory class. Define a dataProvider in this test class which can use the parameter (data-point) received in the constructor. Define a BeforeMethod or BeforeClass which can use that parameter or data point and Your test methods should have the "dataProvider" attribute pointing to the desired dataProvider. Again, in testng.xml use parallel="instances".

Also, use try/catch block for instantiating driver object and closing the browser. This will help you in avoiding skips due to failure of setUp of tearDown method.

Driver behavior executing parallel TestNG selenium tests with dataprovider

2 Answers