Running Parallel Tests using Selenium WebDriver, Selenium Grid and testNG

Question

This is my very first post here in SO; I am a newbie Selenium User with beginner skill in Java.

To give you a background of our work; we are using Page Object Model. All of our tests uses a single DataProvider method that fetches its data from an ".xlsx" file based on the test case name that calls/uses the DataProvider.

We are unsure however if we have declared our ThreadLocal the way it should be. And also, we are not sure if the declaration of our getDriver() method is correct. Another issue is that we are unsure if we should be using "@BeforeTest"/"@AfterTest" or "@BeforeClass"/"@AfterClass" on our setup and tearDown methods.

The following problems are being encountered:

1. One test fails, the succeeding test also fails.
2. There are times when the test data being fetched is inaccurate (more column data thrown than expected).

Here is our CONFIGTESTBASE class:

public class ConfigTestBase {

    private static ThreadLocal<RemoteWebDriver> threadedDriver = null;
    private static XSSFSheet ExcelWSheet;
    private static XSSFWorkbook ExcelWBook;
    private static XSSFCell Cell;
    private static XSSFRow Row;
    private static final String Path_TestData = GlobalConstants.testDataFilePath;
    private static final String File_TestData = GlobalConstants.testDataFileName;

    @Parameters({"objectMapperFile"})
    @BeforeSuite
    public void setupSuite(String objectMapperFile) throws Exception {
        GlobalConstants.objectMapperDefPath = new File(objectMapperFile).getAbsolutePath();
        new Common().OverrideSSLHandshakeException();
    }

    @Parameters({"browserName"})
    @BeforeClass
    public void setup(String browserName) throws Exception {

        threadedDriver = new ThreadLocal<>();

        DesiredCapabilities capabilities = new DesiredCapabilities();

        if (browserName.toLowerCase().contains("firefox")) {
            capabilities = DesiredCapabilities.firefox();
            capabilities.setCapability(CapabilityType.ACCEPT_SSL_CERTS, true);
            capabilities.setBrowserName("firefox");
            capabilities.setPlatform(Platform.WINDOWS);
        }

        if (browserName.toLowerCase().contains("ie")) {
            System.setProperty("webdriver.ie.driver","C:\\selenium\\IEDriverServer.exe");
            capabilities = DesiredCapabilities.internetExplorer();
            capabilities.setCapability(InternetExplorerDriver.INTRODUCE_FLAKINESS_BY_IGNORING_SECURITY_DOMAINS, true);
            capabilities.setCapability(InternetExplorerDriver.FORCE_CREATE_PROCESS, false);
            capabilities.setBrowserName("internet explorer");
            capabilities.setPlatform(Platform.WINDOWS);
        }

        if (browserName.toLowerCase().contains("chrome")) {
            System.setProperty("webdriver.chrome.driver","C:\\selenium\\chromedriver.exe");
            capabilities = DesiredCapabilities.chrome();
            capabilities.setCapability(CapabilityType.ACCEPT_SSL_CERTS, true);
            capabilities.setBrowserName("chrome");
            capabilities.setPlatform(Platform.WINDOWS);
        }

        if (browserName.toLowerCase().contains("safari")) {
            SafariOptions options = new SafariOptions();
            options.setUseCleanSession(true);
            capabilities = DesiredCapabilities.safari();
            capabilities.setCapability(SafariOptions.CAPABILITY, options);
            capabilities.setBrowserName("safari");
            capabilities.setPlatform(Platform.WINDOWS);
        }

        threadedDriver.set(new RemoteWebDriver(new URL(GlobalConstants.GRIDHUB), capabilities));

    }

    protected static RemoteWebDriver getDriver(){
        RemoteWebDriver driver = null;

        try {
            driver = threadedDriver.get();
        } catch (Exception e) {
            e.printStackTrace();
        }
        return driver;
    }

    @AfterClass
    public void tearDown() throws Exception {
        getDriver().quit();
    }

    //This method is to set the File path and to open the Excel file, Pass Excel Path and Sheetname as Arguments to this method
    public void setExcelFile(String Path,String SheetName) throws Exception {
        try {
            // Open the Excel file
            FileInputStream ExcelFile = new FileInputStream(Path);

            // Access the required test data sheet
            ExcelWBook = new XSSFWorkbook(ExcelFile);
            ExcelWSheet = ExcelWBook.getSheet(SheetName);
        } catch (Exception e) {
            throw (e);
        }
    }

    //This method is to read the test data from the Excel cell, in this we are passing parameters as Row num and Col num
    @SuppressWarnings("static-access")
    public String getCellData(int RowNum, int ColNum) throws Exception{
        try{
            Cell = null;
            Cell = ExcelWSheet.getRow(RowNum).getCell(ColNum);
            Cell.setCellType(Cell.CELL_TYPE_STRING);
            return Cell.getStringCellValue();
        } catch (Exception e) {
            return "";
        }
    }

    //This method is to write in the Excel cell, Row num and Col num are the parameters
    @SuppressWarnings("static-access")
    public void setCellData(String textValue,  int RowNum, int ColNum) throws Exception    {
        try{
            Row  = ExcelWSheet.getRow(RowNum);
            Cell = Row.getCell(ColNum, Row.RETURN_BLANK_AS_NULL);
            if (Cell == null) {
                Cell = Row.createCell(ColNum);
                Cell.setCellValue(textValue);
            } else {
                Cell.setCellValue(textValue);
            }

            // Constant variables Test Data path and Test Data file name
            FileOutputStream fileOut = new FileOutputStream(Path_TestData + File_TestData);
            ExcelWBook.write(fileOut);
            fileOut.flush();
            fileOut.close();
        } catch (Exception e) {
            throw (e);
        }
    }

    @DataProvider(name="getDataFromFile")
    public Object[][] getDataFromFile(Method testMethod, ITestContext context) throws Exception {

        String[][] tabArray;
        int intCounter;
        int intRowCount = 0;
        int intRowCounter = 0;
        int intColCount = 0;
        int intColCounter;
        int intColDataCount = 0;
        int intColDataCounter = 0;
        String temp;

        String testName = testMethod.getName();
        String banner = context.getCurrentXmlTest().getParameter("banner");
        setExcelFile(Path_TestData + File_TestData, banner);

        //get number of data to be returned
        for(intCounter=0;intCounter<ExcelWSheet.getLastRowNum()+1;intCounter++){
            if(getCellData(intCounter, 1).equals(testName)) {
                if (intColCount == 0) {
                    intColCount = ExcelWSheet.getRow(intCounter).getLastCellNum() - 2;
                }
                intRowCount++;
            }
        }

        if(intRowCount == 0){
            System.out.println("\n*** Data for '" + testName + "' was not found.");
            throw new AssertionError("Data for '" + testName + "' was not found.");
        }

        for(intCounter=0;intCounter<ExcelWSheet.getLastRowNum()+1;intCounter++){
            if(getCellData(intCounter, 1).equals(testName)) {
                for(intColCounter=2;intColCounter<intColCount+2;intColCounter++) {
                    temp = getCellData(intCounter,intColCounter);
                    if(temp != null && !temp.isEmpty()){
                        intColDataCount++;
                    }
                }
                //to exit FOR loop
                intCounter = ExcelWSheet.getLastRowNum()+1;
            }
        }

        //set data array dimension
        tabArray = new String[intRowCount][intColDataCount];

        for(intCounter=0;intCounter<ExcelWSheet.getLastRowNum()+1;intCounter++){
            if(getCellData(intCounter, 1).equals(testName)) {
                intRowCounter++;
                for(intColCounter=2;intColCounter<intColCount+2;intColCounter++) {
                    temp = getCellData(intCounter,intColCounter);
                    if(temp != null && !temp.isEmpty()){
                        tabArray[intRowCounter-1][intColDataCounter] = getCellData(intCounter,intColCounter);
                        intColDataCounter++;
                    }
                }
            }
        }

        return tabArray;

    }

}

Here is a sample test class that we have; extends CONFIGTESTBASE class... one class per test:

public class Google6 extends ConfigTestBase {

    private Generic generic = null;
    private HomePage homePage = null;

    @BeforeMethod
    public void MethodInit(ITestResult result) throws Exception {
        generic = new Generic(getDriver());
        homePage = new HomePage(getDriver());
    }

    @Test(dataProvider="getDataFromFile")
    public void Google6(String execute, String url, String searchString) throws Exception {

        if(execute.toUpperCase().equals("YES")) {

            //navigate to application page
            generic.navigateToURL(url);

            //search
            homePage.search(searchString);

        } else {
            generic.log("Execute variable <> 'YES'. Skipping execution...");
            throw new SkipException("Execute variable <> 'YES'. Skipping execution...");
        }

    }
}

And here is our suite file; 'banner' parameter is used for finding specific sheet on ".xlsx" file:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">

<suite name="Full Test Suite - Firefox" parallel="classes" thread-count="5" verbose="1">
    <parameter name="objectMapperFile" value="mom.objectdef.properties" />

    <test name="Regression - Firefox">
        <parameter name="browserName" value="firefox" />
        <parameter name="banner" value="SAMPLE" />
        <classes>
            <class name="com.company.automation.web.app.testsuites.Google1" />
            <class name="com.company.automation.web.app.testsuites.Google2" />
            <class name="com.company.automation.web.app.testsuites.Google3" />
            <class name="com.company.automation.web.app.testsuites.Google4" />
            <class name="com.company.automation.web.app.testsuites.Google5" />
            <class name="com.company.automation.web.app.testsuites.Google6" />
        </classes>
    </test>

</suite>

Any suggestions on how we could make it work?

Answer 1

1) You are instantiating a new ThreadLocal for each test class since it's in the BeforeClass annotation. You should be declaring and instantiating the ThreadLocal at the same time:

private static ThreadLocal<RemoteWebDriver> threadedDriver = new ThreadLocal<>();

2) That Excel parsing logic is insane. At the very least, you need to separate that into it's own class. If at all possible, use .csv files instead of .xlsx files and use something like opencsv to parse the file; it will rock your world. After parsing you can convert it to something that TestNG's DataProvider expects. Something like this:

public class CSVDataReader {
    private static final char DELIMITER = ',';
    private static final char QUOTE_CHAR = '\"';
    private static final char ESCAPE_CHAR = '\\';

    private static List<String[]> read(String filePath, boolean hasHeader) throws IOException {
        CSVReader reader;

        if (hasHeader) {
            // If file has a header, skip the header (line 1)
            reader = new CSVReader(new FileReader(filePath), DELIMITER, QUOTE_CHAR, ESCAPE_CHAR, 1);
        } else {
            reader = new CSVReader(new FileReader(filePath), DELIMITER, QUOTE_CHAR, ESCAPE_CHAR);
        }

        List<String[]> rows = reader.readAll();
        reader.close();

        return rows;
    }

    public static String [][] readCSVFileToString2DArray(String filePath, boolean hasHeader) throws IOException {
        List<String[]> rows = read(filePath, hasHeader);

        // Store all rows/columns in a two-dimensional String array, then return it
        String [][] csvData = new String[rows.size()][];

        int r = 0;
        for (String[] row : rows) {
            csvData[r++] = row;
        }

        return csvData;
    }
}

Answer 2

one thing at a time..I believe you've been handed over this code by someone..please check on how to initialize threadlocal first here

try to narrow down your issue one at a time with small snippet of codes to understand root cause

Answer 3

Couple of suggestions.

1. Try separating what you logically seperate tasks. for eg. initializing driver and data provider are two seperate tasks. You can create dataprovider in a separate class and refer to that inside any test eg. "By default, the data provider will be looked for in the current test class or one of its base classes. If you want to put your data provider in a different class, it needs to be a static method and you specify the class where it can be found in the dataProviderClass attribute:"

2. Your driver instantiation can be a separate class, since it is all static. The instantiation of the threadlocal should be at the point of declaration. The setting of values should be in the threads. So move your new ThreadLocal to your declaration.

   3. The choice of beforetest or class would depend on how you plan to run your tests parallely. If you plan to use the same driver instance across all methods of your class, then beforeclass should be used. If you plan to run test as testtags in xml parallely then beforetest should be used.
      Since you are using parallel = classes, choice of beforeclass seems to be correct.