Selenium test not failing correctly - Pass with configuration fail rather than “proper” fail

Question

Hi I'm still newbie to Selenium/Scripting/Java and I'me still at the stage of hacking together code from elsewhere to get stuff work( tutorials and recorders mainly)

Anyway Im trying to write a script to check a particular 'element' is present (I will want to the the reverse as well) I can get the script to pass correctly when it finds the 'element' but if change the element details so I know it should fail (as it don't exist) TestNG still pass's the test but gives a configuration fail ?

I presume I'm missing something to cover the fail aspect of the test but no sure how to go about it, every time I try I and get it to run into this.

package Links;

import org.testng.annotations.*;
import static org.testng.Assert.*;
import org.openqa.selenium.*;
import org.openqa.selenium.firefox.FirefoxDriver;

public class TestArea {
    private WebDriver driver;
    private StringBuffer verificationErrors = new StringBuffer();

@BeforeClass(alwaysRun = true)
public void setUp() throws Exception {
    System.setProperty("webdriver.gecko.driver", "C:\\Automation\\SeleniumFiles\\Browser Drivers\\geckodriver.exe");
    driver = new FirefoxDriver();

}

@Test
public void Example() throws Exception {
    driver.get(
            "http://MyWebsite");
    try {
        assertTrue(isElementPresent(
                By.xpath("The Element I want look for ")));

    } catch (Error e) {
        verificationErrors.append(e.toString());
    }
}
// -------------------------------------------------------------------------------------

@AfterClass(alwaysRun = true)
public void tearDown() throws Exception {
    driver.quit();
    String verificationErrorString = verificationErrors.toString();
    if (!"".equals(verificationErrorString)) {
        fail(verificationErrorString);
    }
}

private boolean isElementPresent(By by) {
    try {
        driver.findElement(by);
        return true;
    } catch (NoSuchElementException e) {
        return false;
    }
}
}

An example of a "passed" test but with a failed configuration.

FAILED CONFIGURATION: @AfterClass tearDown java.lang.AssertionError: java.lang.AssertionError: expected [true] but found [false] at org.testng.Assert.fail(Assert.java:96) at Links.TestArea.tearDown(TestArea.java:39) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.lang.reflect.Method.invoke(Unknown Source) at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:124) at org.testng.internal.MethodInvocationHelper.invokeMethodConsideringTimeout(MethodInvocationHelper.java:59) at org.testng.internal.Invoker.invokeConfigurationMethod(Invoker.java:455) at org.testng.internal.Invoker.invokeConfigurations(Invoker.java:222) at org.testng.internal.Invoker.invokeConfigurations(Invoker.java:142) at org.testng.internal.TestMethodWorker.invokeAfterClassMethods(TestMethodWorker.java:214) at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:111) at org.testng.TestRunner.privateRun(TestRunner.java:648) at org.testng.TestRunner.run(TestRunner.java:505) at org.testng.SuiteRunner.runTest(SuiteRunner.java:455) at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:450) at org.testng.SuiteRunner.privateRun(SuiteRunner.java:415) at org.testng.SuiteRunner.run(SuiteRunner.java:364) at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:52) at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:84) at org.testng.TestNG.runSuitesSequentially(TestNG.java:1208) at org.testng.TestNG.runSuitesLocally(TestNG.java:1137) at org.testng.TestNG.runSuites(TestNG.java:1049) at org.testng.TestNG.run(TestNG.java:1017) at org.testng.remote.AbstractRemoteTestNG.run(AbstractRemoteTestNG.java:114) at org.testng.remote.RemoteTestNG.initAndRun(RemoteTestNG.java:251) at org.testng.remote.RemoteTestNG.main(RemoteTestNG.java:77)

PASSED: Example

=============================================== Default test Tests run: 1, Failures: 0, Skips: 0

Configuration Failures: 1, Skips: 0

=============================================== Default suite Total tests run: 1, Failures: 0, Skips: 0 Configuration Failures: 1, Skips:

0

I don't get the configuration issue when the test can find the element.

Thanks very much in advance

Asserting that an element is present or not is kind of useless. One state or another doesn't guaranty that the rendered page is the expected one (an element can be present but not displayed or your locator could no longer match the expected element). Instead leave the assertion to the action (click, sendKeys...) or assert the presence / non presence of text which will match what an end-user sees. — Florent B.
I had originally set it up to look for text, and though the doing via element would be more robust, I was looking to test that the expected breadcrumb was present, so you are saying I should go back to my original idea ? On the website I'm trying to test I was also going to use similar code to click though links and check participial set of errors wasn't present but this "error" has many parts to it image/text/elements, should I just stick to search for it via text again ? — Leon Tilbrook
It depends on how the page is constructed, especially if you are trying to validate some images. Keep in mind that Selenium will only help you validate a use case. It doesn't provide any method out of the box to validate the layout. — Florent B.
In this particular case I just wanted to confirm that the breadcrumb navigation bar is present as part of a PLP (product listing page) and use a completely different script to test the functionally of the breadcrumb. — Leon Tilbrook

Krishnan Mahadevan Krishnan Mahadevan · Accepted Answer · 2018-04-28T04:30:29

There are a few issues in your test code.

TestNG by default fails a @Test method when :

An assertion fails
test method raises an Exception.

So you don't need to wrap assertTrue() call within a try..catch block. If you would like to run through all assertions and have the test method fail at the end, you should be using something called as Soft Assertion in TestNG.

Below is a cleaned-up version of your test code.

import org.testng.annotations.*; 
import static org.testng.Assert.*; 
import org.openqa.selenium.*; 
import org.openqa.selenium.firefox.FirefoxDriver;

public class TestArea { 
    private WebDriver driver; 

    @BeforeClass(alwaysRun = true) 
    public void setUp() throws Exception { 
        System.setProperty("webdriver.gecko.driver", "C:\\Automation\\SeleniumFiles\\Browser Drivers\\geckodriver.exe"); 
        driver = new FirefoxDriver();
    }

    @Test 
    public void Example() throws Exception { 
        driver.get( "http://MyWebsite"); 
        assertTrue(isElementPresent( By.xpath("The Element I want look for ")));
    } 

    @AfterClass(alwaysRun = true) 
    public void tearDown() throws Exception { 
        driver.quit(); 
    }

    private boolean isElementPresent(By by) { 
        try { 
            driver.findElement(by); 
            return true; 
        } catch (NoSuchElementException e) { 
            return false; 
        } 
    } 
}

Selenium test not failing correctly - Pass with configuration fail rather than “proper” fail

Configuration Failures: 1, Skips: 0

0

1 Answers