Selenium wait until document is ready

141

votes

Can anyone let me how can I make selenium wait until the time the page loads completely? I want something generic, I know I can configure WebDriverWait and call something like 'find' to make it wait but I don't go that far. I just need to test that the page loads successfully and move on to next page to test.

I found something in .net but couldn't make it work in java ...

IWait<IWebDriver> wait = new OpenQA.Selenium.Support.UI.WebDriverWait(driver, TimeSpan.FromSeconds(30.00));
wait.Until(driver1 => ((IJavaScriptExecutor)driver).ExecuteScript("return document.readyState").Equals("complete"));

Any thoughts anyone?

seleniumloaddocumentwait

Why is that you dont want to use wait? – Amey

You mean explicit wait ? It is time consuming , I'm testing some 10k pages. – Girish

I mean , adding a fix wait may not be a good idea if I'm testing a large number of links , right ? – Girish

Waiting for a fixed number of seconds is of no use. That is guessing. – Anders Lindén

Given that a page's javascript can run any generic code, writing a program to wait for its completion is impossible. It's a form of the Halting Problem (en.wikipedia.org/wiki/Halting_problem). Any solution here will need to make compromises or be based on assumptions of the underlying webpage. – speedplane

96

votes

Your suggested solution only waits for DOM readyState to signal complete. But Selenium by default tries to wait for those (and a little bit more) on page loads via the driver.get() and element.click() methods. They are already blocking, they wait for the page to fully load and those should be working ok.

Problem, obviously, are redirects via AJAX requests and running scripts - those can't be caught by Selenium, it doesn't wait for them to finish. Also, you can't reliably catch them via readyState - it waits for a bit, which can be useful, but it will signal complete long before all the AJAX content is downloaded.

There is no general solution that would work everywhere and for everyone, that's why it's hard and everyone uses something a little bit different.

The general rule is to rely on WebDriver to do his part, then use implicit waits, then use explicit waits for elements you want to assert on the page, but there's a lot more techniques that can be done. You should pick the one (or a combination of several of them) that works best in your case, on your tested page.

See my two answers regarding this for more information:

84

votes

Try this code:

  driver.manage().timeouts().pageLoadTimeout(10, TimeUnit.SECONDS);

The above code will wait up to 10 seconds for page loading. If the page loading exceeds the time it will throw the TimeoutException. You catch the exception and do your needs. I am not sure whether it quits the page loading after the exception thrown. i didn't try this code yet. Want to just try it.

This is an implicit wait. If you set this once it will have the scope until the Web Driver instance destroy.

See the documentation for WebDriver.Timeouts for more info.

67

votes

This is a working Java version of the example you gave :

void waitForLoad(WebDriver driver) {
    new WebDriverWait(driver, 30).until((ExpectedCondition<Boolean>) wd ->
            ((JavascriptExecutor) wd).executeScript("return document.readyState").equals("complete"));
}

Example For c#:

public static void WaitForLoad(IWebDriver driver, int timeoutSec = 15)
{
    IJavaScriptExecutor js = (IJavaScriptExecutor)driver;
    WebDriverWait wait = new WebDriverWait(driver, new TimeSpan(0, 0, timeoutSec));
    wait.Until(wd => js.ExecuteScript("return document.readyState").ToString() == "complete");
}

Example for PHP:

final public function waitUntilDomReadyState(RemoteWebDriver $webDriver): void
{
    $webDriver->wait()->until(function () {
        return $webDriver->executeScript('return document.readyState') === 'complete';
    });
}

12

votes

Here's my attempt at a completely generic solution, in Python:

First, a generic "wait" function (use a WebDriverWait if you like, I find them ugly):

def wait_for(condition_function):
    start_time = time.time()
    while time.time() < start_time + 3:
        if condition_function():
            return True
        else:
            time.sleep(0.1)
    raise Exception('Timeout waiting for {}'.format(condition_function.__name__))

Next, the solution relies on the fact that selenium records an (internal) id-number for all elements on a page, including the top-level <html> element. When a page refreshes or loads, it gets a new html element with a new ID.

So, assuming you want to click on a link with text "my link" for example:

old_page = browser.find_element_by_tag_name('html')

browser.find_element_by_link_text('my link').click()

def page_has_loaded():
    new_page = browser.find_element_by_tag_name('html')
    return new_page.id != old_page.id

wait_for(page_has_loaded)

For more Pythonic, reusable, generic helper, you can make a context manager:

from contextlib import contextmanager

@contextmanager
def wait_for_page_load(browser):
    old_page = browser.find_element_by_tag_name('html')

    yield

    def page_has_loaded():
        new_page = browser.find_element_by_tag_name('html')
        return new_page.id != old_page.id

    wait_for(page_has_loaded)

And then you can use it on pretty much any selenium interaction:

with wait_for_page_load(browser):
    browser.find_element_by_link_text('my link').click()

I reckon that's bulletproof! What do you think?

More info in a blog post about it here

7

votes

I had a similar problem. I needed to wait until my document was ready but also until all Ajax calls had finished. The second condition proved to be difficult to detect. In the end I checked for active Ajax calls and it worked.

Javascript:

return (document.readyState == 'complete' && jQuery.active == 0)

Full C# method:

private void WaitUntilDocumentIsReady(TimeSpan timeout)
{
    var javaScriptExecutor = WebDriver as IJavaScriptExecutor;
    var wait = new WebDriverWait(WebDriver, timeout);            

    // Check if document is ready
    Func<IWebDriver, bool> readyCondition = webDriver => javaScriptExecutor
        .ExecuteScript("return (document.readyState == 'complete' && jQuery.active == 0)");
    wait.Until(readyCondition);
}

7

votes

WebDriverWait wait = new WebDriverWait(dr, 30);
wait.until(ExpectedConditions.jsReturnsValue("return document.readyState==\"complete\";"));

4

votes

For C# NUnit, you need to convert WebDriver to JSExecuter and then execute the script to check if document.ready state is complete or not. Check below code for reference:

 public static void WaitForLoad(IWebDriver driver)
    {
        IJavaScriptExecutor js = (IJavaScriptExecutor)driver;
        int timeoutSec = 15;
        WebDriverWait wait = new WebDriverWait(driver, new TimeSpan(0, 0, timeoutSec));
        wait.Until(wd => js.ExecuteScript("return document.readyState").ToString() == "complete");
    }

This will wait until the condition is satisfied or timeout.

3

votes

For initial page load I have noticed that "Maximizing" the browser window practically waits until page load is completed (including sources)

Replace:

AppDriver.Navigate().GoToUrl(url);

With:

public void OpenURL(IWebDriver AppDriver, string Url)
            {
                try
                {
                    AppDriver.Navigate().GoToUrl(Url);
                    AppDriver.Manage().Window.Maximize();
                    AppDriver.SwitchTo().ActiveElement();
                }
                catch (Exception e)
                {
                    Console.WriteLine("ERR: {0}; {1}", e.TargetSite, e.Message);
                    throw;
                }
            }

than use:

OpenURL(myDriver, myUrl);

This will load the page, wait until completed, maximize and focus on it. I don't know why its like this but it works.

If you want to wait for page load after click on next or any other page navigation trigger other then "Navigate()", Ben Dyer's answer (in this thread) will do the work.

2

votes

In Nodejs you can get it via promises...

If you write this code, you can be sure that the page is fully loaded when you get to the then...

driver.get('www.sidanmor.com').then(()=> {
    // here the page is fully loaded!!!
    // do your stuff...
}).catch(console.log.bind(console));

If you write this code, you will navigate, and selenium will wait 3 seconds...

driver.get('www.sidanmor.com');
driver.sleep(3000);
// you can't be sure that the page is fully loaded!!!
// do your stuff... hope it will be OK...

From Selenium documentation:

this.get( url ) → Thenable

Schedules a command to navigate to the given URL.

Returns a promise that will be resolved when the document has finished loading.

Selenium Documentation (Nodejs)

1

votes

Have a look at tapestry web-framework. You can download source code there.

The idea is to signalize that page is ready by html attribute of body. You can use this idea ignore complicated sue cases.

<html>
<head>
</head>
<body data-page-initialized="false">
    <p>Write you page here</p>

    <script>
    $(document).ready(function () {
        $(document.body).attr('data-page-initialized', 'true');
    });
    </script>  
</body>
</html>

And then create extension of Selenium webdriver (according to tapestry framework)

public static void WaitForPageToLoad(this IWebDriver driver, int timeout = 15000)
{
    //wait a bit for the page to start loading
    Thread.Sleep(100);

    //// In a limited number of cases, a "page" is an container error page or raw HTML content
    // that does not include the body element and data-page-initialized element. In those cases,
    // there will never be page initialization in the Tapestry sense and we return immediately.
    if (!driver.ElementIsDisplayed("/html/body[@data-page-initialized]"))
    {                
        return;
    }

    Stopwatch stopwatch = Stopwatch.StartNew();

    int sleepTime = 20;

    while(true)
    {
        if (driver.ElementIsDisplayed("/html/body[@data-page-initialized='true']"))
        {
            return;
        }

        if (stopwatch.ElapsedMilliseconds > 30000)
        {
            throw new Exception("Page did not finish initializing after 30 seconds.");
        }

        Thread.Sleep(sleepTime);
        sleepTime *= 2; // geometric row of sleep time
    }          
}

Use extension ElementIsDisplayed written by Alister Scott.

public static bool ElementIsDisplayed(this IWebDriver driver, string xpath)
{
    try
    {
        return driver.FindElement(By.XPath(xpath)).Displayed;
    }
    catch(NoSuchElementException)
    {
        return false;
    }
}

And finally create test:

driver.Url = this.GetAbsoluteUrl("/Account/Login");            
driver.WaitForPageToLoad();

1

votes

Ben Dryer's answer didn't compile on my machine ("The method until(Predicate<WebDriver>) is ambiguous for the type WebDriverWait").

Working Java 8 version:

Predicate<WebDriver> pageLoaded = wd -> ((JavascriptExecutor) wd).executeScript(
        "return document.readyState").equals("complete");
new FluentWait<WebDriver>(driver).until(pageLoaded);

Java 7 version:

Predicate<WebDriver> pageLoaded = new Predicate<WebDriver>() {

        @Override
        public boolean apply(WebDriver input) {
            return ((JavascriptExecutor) input).executeScript("return document.readyState").equals("complete");
        }

};
new FluentWait<WebDriver>(driver).until(pageLoaded);

1

votes

I tried this code and it works for me. I call this function every time I move to another page

public static void waitForPageToBeReady() 
{
    JavascriptExecutor js = (JavascriptExecutor)driver;

    //This loop will rotate for 100 times to check If page Is ready after every 1 second.
    //You can replace your if you wants to Increase or decrease wait time.
    for (int i=0; i<400; i++)
    { 
        try 
        {
            Thread.sleep(1000);
        }catch (InterruptedException e) {} 
        //To check page ready state.

        if (js.executeScript("return document.readyState").toString().equals("complete"))
        { 
            break; 
        }   
      }
 }

1

votes

The wait for the document.ready event is not the entire fix to this problem, because this code is still in a race condition: Sometimes this code is fired before the click event is processed so this directly returns, since the browser hasn't started loading the new page yet.

After some searching I found a post on Obay the testing goat, which has a solution for this problem. The c# code for that solution is something like this:

 IWebElement page = null;
 ...
 public void WaitForPageLoad()
 {
    if (page != null)
    {
       var waitForCurrentPageToStale = new WebDriverWait(driver, TimeSpan.FromSeconds(10));
       waitForCurrentPageToStale.Until(ExpectedConditions.StalenessOf(page));
    }

    var waitForDocumentReady = new WebDriverWait(driver, TimeSpan.FromSeconds(10));
    waitForDocumentReady.Until((wdriver) => (driver as IJavaScriptExecutor).ExecuteScript("return document.readyState").Equals("complete"));

    page = driver.FindElement(By.TagName("html"));

}

` I fire this method directly after the driver.navigate.gotourl, so that it gets a reference of the page as soon as possible. Have fun with it!

1

votes

normaly when selenium open a new page from a click or submit or get methods, it will wait untell the page is loaded but the probleme is when the page have a xhr call (ajax) he will never wait of the xhr to be loaded, so creating a new methode to monitor a xhr and wait for them it will be the good.

public boolean waitForJSandJQueryToLoad() {
    WebDriverWait wait = new WebDriverWait(webDriver, 30);
    // wait for jQuery to load
    ExpectedCondition<Boolean> jQueryLoad = new ExpectedCondition<Boolean>() {
      @Override
      public Boolean apply(WebDriver driver) {
        try {
            Long r = (Long)((JavascriptExecutor)driver).executeScript("return $.active");
            return r == 0;
        } catch (Exception e) {
            LOG.info("no jquery present");
            return true;
        }
      }
    };

    // wait for Javascript to load
    ExpectedCondition<Boolean> jsLoad = new ExpectedCondition<Boolean>() {
      @Override
      public Boolean apply(WebDriver driver) {
        return ((JavascriptExecutor)driver).executeScript("return document.readyState")
        .toString().equals("complete");
      }
    };

  return wait.until(jQueryLoad) && wait.until(jsLoad);
}

if $.active == 0 so the is no active xhrs call (that work only with jQuery). for javascript ajax call you have to create a variable in your project and simulate it.

0

votes

You can write some logic to handle this. I have write a method that will return the WebElement and this method will be called three times or you can increase the time and add a null check for WebElement Here is an example

public static void main(String[] args) {
        WebDriver driver = new FirefoxDriver();
        driver.get("https://www.crowdanalytix.com/#home");
        WebElement webElement = getWebElement(driver, "homekkkkkkkkkkkk");
        int i = 1;
        while (webElement == null && i < 4) {
            webElement = getWebElement(driver, "homessssssssssss");
            System.out.println("calling");
            i++;
        }
        System.out.println(webElement.getTagName());
        System.out.println("End");
        driver.close();
    }

    public static WebElement getWebElement(WebDriver driver, String id) {
        WebElement myDynamicElement = null;
        try {
            myDynamicElement = (new WebDriverWait(driver, 10))
                    .until(ExpectedConditions.presenceOfElementLocated(By
                            .id(id)));
            return myDynamicElement;
        } catch (TimeoutException ex) {
            return null;
        }
    }

0

votes

I executed a javascript code to check if the document is ready. Saved me a lot of time debugging selenium tests for sites that has client side rendering.

public static boolean waitUntilDOMIsReady(WebDriver driver) {
def maxSeconds = DEFAULT_WAIT_SECONDS * 10
for (count in 1..maxSeconds) {
    Thread.sleep(100)
    def ready = isDOMReady(driver);
    if (ready) {
        break;
    }
}

}

public static boolean isDOMReady(WebDriver driver){
    return driver.executeScript("return document.readyState");
}

0

votes

public boolean waitForElement(String zoneName, String element, int index, int timeout) {
        WebDriverWait wait = new WebDriverWait(appiumDriver, timeout/1000);
        wait.until(ExpectedConditions.visibilityOfElementLocated(By.xpath(element)));
        return true;
    }

0

votes

Like Rubanov wrote for C#, i write it for Java, and it is:

    public void waitForPageLoaded() {
    ExpectedCondition<Boolean> expectation = new
            ExpectedCondition<Boolean>() {
                public Boolean apply(WebDriver driver) {
                    return (((JavascriptExecutor) driver).executeScript("return document.readyState").toString().equals("complete")&&((Boolean)((JavascriptExecutor)driver).executeScript("return jQuery.active == 0")));
                }
            };
    try {
        Thread.sleep(100);
        WebDriverWait waitForLoad = new WebDriverWait(driver, 30);
        waitForLoad.until(expectation);
    } catch (Throwable error) {
        Assert.fail("Timeout waiting for Page Load Request to complete.");
    }
}

0

votes

In Java it will like below :-

  private static boolean isloadComplete(WebDriver driver)
    {
        return ((JavascriptExecutor) driver).executeScript("return document.readyState").equals("loaded")
                || ((JavascriptExecutor) driver).executeScript("return document.readyState").equals("complete");
    }

0

votes

The following code should probably work:

WebDriverWait wait = new WebDriverWait(driver, 10);
wait.until(ExpectedConditions.presenceOfAllElementsLocated(By.xpath("//*")));

0

votes

If you have a slow page or network connection, chances are that none of the above will work. I have tried them all and the only thing that worked for me is to wait for the last visible element on that page. Take for example the Bing webpage. They have placed a CAMERA icon (search by image button) next to the main search button that is visible only after the complete page has loaded. If everyone did that, then all we have to do is use an explicit wait like in the examples above.

-1

votes

public void waitForPageToLoad()
  {
(new WebDriverWait(driver, DEFAULT_WAIT_TIME)).until(new ExpectedCondition<Boolean>() {
      public Boolean apply(WebDriver d) {
        return (((org.openqa.selenium.JavascriptExecutor) driver).executeScript("return document.readyState").equals("complete"));
      }
    });//Here DEFAULT_WAIT_TIME is a integer correspond to wait time in seconds

-1

votes

Here's something similar, in Ruby:

wait = Selenium::WebDriver::Wait.new(:timeout => 10)
wait.until { @driver.execute_script('return document.readyState').eql?('complete') }

-1

votes

You can have the thread sleep till the page is reloaded. This is not the best solution, because you need to have an estimate of how long does the page take to load.

driver.get(homeUrl); 
Thread.sleep(5000);
driver.findElement(By.xpath("Your_Xpath_here")).sendKeys(userName);
driver.findElement(By.xpath("Your_Xpath_here")).sendKeys(passWord);
driver.findElement(By.xpath("Your_Xpath_here")).click();

-1

votes

I Checked page load complete, work in Selenium 3.14.0

    public static void UntilPageLoadComplete(IWebDriver driver, long timeoutInSeconds)
    {
        Until(driver, (d) =>
        {
            Boolean isPageLoaded = (Boolean)((IJavaScriptExecutor)driver).ExecuteScript("return document.readyState").Equals("complete");
            if (!isPageLoaded) Console.WriteLine("Document is loading");
            return isPageLoaded;
        }, timeoutInSeconds);
    }

    public static void Until(IWebDriver driver, Func<IWebDriver, Boolean> waitCondition, long timeoutInSeconds)
    {
        WebDriverWait webDriverWait = new WebDriverWait(driver, TimeSpan.FromSeconds(timeoutInSeconds));
        webDriverWait.Timeout = TimeSpan.FromSeconds(timeoutInSeconds);
        try
        {
            webDriverWait.Until(waitCondition);
        }
        catch (Exception e)
        {
            Console.WriteLine(e);
        }
    }

-1

votes

For the people who need to wait for a specific element to show up. (used c#)

public static void WaitForElement(IWebDriver driver, By element)
{
    WebDriverWait wait = new WebDriverWait(driver, TimeSpan.FromSeconds(20));
    wait.Until(ExpectedConditions.ElementIsVisible(element));
}

Then if you want to wait for example if an class="error-message" exists in the DOM you simply do:

WaitForElement(driver, By.ClassName("error-message"));

For id, it will then be

WaitForElement(driver, By.Id("yourid"));

-3

votes

Are you using Angular? If you are it is possible that the webdriver doesn't recognize that the async calls have finished.

I recommend looking at Paul Hammants ngWebDriver. The method waitForAngularRequestsToFinish() could come in handy.

Selenium wait until document is ready

27 Answers