Selenium Grid multi browser and multi platform right approach

Question

My goal is to a run test cases parallel on below combination and produce a extent report for each combination, total 8 combination

Windows (Chrome, Firefox, IE)
Linux (Chrome, Firefox)
Mac (Safari, Chrome, Firefox)

Have come up with this after searching over net. This <suite> run all <test> tags parallel and each <test> represents a OS & browser combination that again run test classes parallel. Each test class has a RemoteWebDriver instance.

Whether this solution is correct or do i need to make any changes?
Complexity increases once test classes start increasing I want to add / remove test classes from all 8 test tags.
If i want run just one combination(os+browser) do i need to have another testng.xml file and edit the parameter values?
How to get the report for each combination. Since all test are running parallel and multi threaded how to track each test?

TestNg.xml

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="Suite" parallel="tests">
    <test name="Windows_Chrome" parallel="classes" thread-count="3">
        <parameter name="os" value="windows" />
        <parameter name="browser" value="chrome" />
        <classes>
            <class name="com.mag.SeleniumGrid.Test_001" />
            <class name="com.mag.SeleniumGrid.Test_002" />
            <class name="com.mag.SeleniumGrid.Test_00N" />
            <!--Each class create RemoteWebDriver instance based on parameters-->
        </classes>
    </test>
    <!-- ..... all 8 combinations -->
    
    <test name="Linux_Chrome" parallel="classes" thread-count="3">
        <parameter name="os" value="linux" />
        <parameter name="browser" value="chrome" />
        <classes>
            <class name="com.mag.SeleniumGrid.Test_001" />
            <class name="com.mag.SeleniumGrid.Test_002" />
            <class name="com.mag.SeleniumGrid.Test_00N" />
        </classes>
    </test>
</suite>

Greg Greg · Accepted Answer · 2020-11-11T21:41:16

Can't say I have used this method so apologies but I use maven to run the tests and have found it works fine. You can add a test to a group by doing:

@Test(groups = {"WindowsChrome", "LinuxChrome"})
public class Test123 {

You can run tests with maven like:

mvn test -Dgroups=WindowsChrome

If you want to run this in a pipeline then I would checkout the code and clone it for each environment. In the pipeline you can add .env files that each environment will use containing variables like "OS_TYPE" and "BROWSER_TYPE". Now you can do:

stage('Windows Chrome Tests') {
     steps {
          bat "mvn test -f WindowsChrome/pom.xml -Dgroups=WindowsChrome"
     }
}
stage('Linux Chrome Tests') {
     steps {
          bat "mvn test -f LinuxChrome/pom.xml -Dgroups=LinuxChrome"
     }
}

In order to have logs making any sense with parallel tests you need to use ThreadLocal with Logger which will create a unique instance for each thread. Ensure you remove this when done or a following test using the same thread will continue on using the same instance. I would put the OS/browser in the log file name.

Selenium Grid multi browser and multi platform right approach

1 Answers