I am using soft assertion in my test class and getting test data from data provider. Below is the sample code:
@Test(dataProvider = "userPages")
public void validatePages(String pageName) {
try {
Reporter.log("Logged In User : " + userType, true);
//Page1 takes extra checks
if (pageName.equalsIgnoreCase("Page1")) {
System.out.println("Running in Page1 bloc ");
softAssert.assertEquals(page.isValidPage1(), true,
"Failed to loada : " + pageName);
} else {
System.out.println("Running in NON Page1 bloc .. Page Name is :: "+pageName);
softAssert.assertEquals(page.isValid(), true,
"Failed to load : " + pageName);
}
softAssert.assertAll(); ---> Appends page1 failures to all the subsequent passed page tests thus marking all of them failed.
} catch (InterruptedException e) {
Reporter.log("Test failed for : " + pageName);
}
//
}
TestNG dependency is ::
<dependency>
<groupId>org.testng</groupId>
<artifactId>testng</artifactId>
<version>${testng.version}</version>
</dependency>
In my case, if a single page fails then all the subsequent pages are marked as failed even if they are passed. I have tried to debug the issue and found that, in SoftAssert class's assertAll() method, the error message of previously failed test-data is getting appended
public void assertAll() {
if (!m_errors.isEmpty()) {
StringBuilder sb = new StringBuilder("The following asserts failed:");
boolean first = true;
for (Map.Entry<AssertionError, IAssert<?>> ae : m_errors.entrySet()) {
if (first) {
first = false;
} else {
sb.append(",");
}
sb.append("\n\t");
sb.append(ae.getKey().getMessage());
}
throw new AssertionError(sb.toString());
}}
This issue is marking false errors in the testNG reports. Please help and advice.
Edit1:
Data Provider
@DataProvider(name = "userPageNames")
public Object[] creatingObjects() {
Map<String, String> pagesList = null;
try {
pagesList = //Populate Map
} catch (Exception e) {
Reporter.log("Failed to get Pages List for " + userType, true);
}
Object[] object = new Object[pagesList.keySet().size()];
Integer i = 0;
for (String pageName : pagesList.keySet()) {
object[i] = pageName;
i++;
}
return object;
}
Report: Genuine Failure
Logged In User : user1
Running in Page1 bloc ..
Page verified : Page1
java.lang.AssertionError: The following asserts failed:
Failed to loada : Page1 expected [true] but found [false]
Expected :true
Actual :false
at org.testng.asserts.SoftAssert.assertAll(SoftAssert.java:43)
at web.dishaPages.ValidatePagesTestNew.validatePages(ValidatePagesTestNew.java:88)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:567)
at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:124)
at org.testng.internal.MethodInvocationHelper$1.runTestMethod(MethodInvocationHelper.java:230)
at org.springframework.test.context.testng.AbstractTestNGSpringContextTests.run(AbstractTestNGSpringContextTests.java:175)
at org.testng.internal.MethodInvocationHelper.invokeHookable(MethodInvocationHelper.java:242)
at org.testng.internal.Invoker.invokeMethod(Invoker.java:576)
at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:716)
at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:988)
at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:125)
at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:109)
at org.testng.TestRunner.privateRun(TestRunner.java:648)
at org.testng.TestRunner.run(TestRunner.java:505)
at org.testng.SuiteRunner.runTest(SuiteRunner.java:455)
at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:450)
at org.testng.SuiteRunner.privateRun(SuiteRunner.java:415)
at org.testng.SuiteRunner.run(SuiteRunner.java:364)
at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:52)
at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:84)
at org.testng.TestNG.runSuitesSequentially(TestNG.java:1208)
at org.testng.TestNG.runSuitesLocally(TestNG.java:1137)
at org.testng.TestNG.runSuites(TestNG.java:1049)
at org.testng.TestNG.run(TestNG.java:1017)
at org.testng.IDEARemoteTestNG.run(IDEARemoteTestNG.java:73)
at org.testng.RemoteTestNGStarter.main(RemoteTestNGStarter.java:123)
Report: Bogus Failures
Logged In User : user1
Running in NON Page1 bloc .. Page Name is :: ViewDMPage
Page verified : ViewDMPage
java.lang.AssertionError: The following asserts failed:
Failed to loada : Page1 expected [true] but found [false]
Expected :true
Actual :false
at org.testng.asserts.SoftAssert.assertAll(SoftAssert.java:43)
at web.dishaPages.ValidatePagesTestNew.validatePages(ValidatePagesTestNew.java:88)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:567)
at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:124)
at org.testng.internal.MethodInvocationHelper$1.runTestMethod(MethodInvocationHelper.java:230)
at org.springframework.test.context.testng.AbstractTestNGSpringContextTests.run(AbstractTestNGSpringContextTests.java:175)
at org.testng.internal.MethodInvocationHelper.invokeHookable(MethodInvocationHelper.java:242)
at org.testng.internal.Invoker.invokeMethod(Invoker.java:576)
at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:716)
at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:988)
at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:125)
at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:109)
at org.testng.TestRunner.privateRun(TestRunner.java:648)
at org.testng.TestRunner.run(TestRunner.java:505)
at org.testng.SuiteRunner.runTest(SuiteRunner.java:455)
at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:450)
at org.testng.SuiteRunner.privateRun(SuiteRunner.java:415)
at org.testng.SuiteRunner.run(SuiteRunner.java:364)
at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:52)
at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:84)
at org.testng.TestNG.runSuitesSequentially(TestNG.java:1208)
at org.testng.TestNG.runSuitesLocally(TestNG.java:1137)
at org.testng.TestNG.runSuites(TestNG.java:1049)
at org.testng.TestNG.run(TestNG.java:1017)
at org.testng.IDEARemoteTestNG.run(IDEARemoteTestNG.java:73)
at org.testng.RemoteTestNGStarter.main(RemoteTestNGStarter.java:123)
