watir-webdriver is very slow locating an element by regexp on a heavy page

3
votes

I found watir-webdriver to be extremely slow locating an element by regexp on a very big page, this is at least true in FF 8.0.1 for me. Example on a flight search results page (containing about 50 search results, each of them is a good portion of html structure):

1.

irb(main):057:0>
t=Time.now;browser.div({:id=>”flightModule40”}).exists?;puts
”#{Time.now-t}” #=> 0.202

2.

irb(main):056:0>
t=Time.now;browser.div({:id=>/flightModule40/}).exists?;puts
”#{Time.now-t}” #=> 131.046

3.

irb(main):058:0>
t=Time.now;browser.div({:id=>/flightModule/, :index=>40}).exists?;puts
”#{Time.now-t}” #=>—is working for 30 minutes for me already…

All of the above works with acceptable speed in watir 1.6.5 and watir 2.0.4 in IE8.

For comparison, here are some times using watir-webdriver against IE8 and FF3.6

watir-webdriver on IE8:

  1. => 0.172017
  2. & 3. => Timeout::Error: execution expired from C:/ruby/1.8.7-p334/lib/ruby/1.8/timeout.rb:64:in `rbuf_fill'

watir-webdriver with FF3.6:

  1. => 0.055005,
  2. => 80.095009,
  3. => 101.734173.

Steps to get to the page under test - www.expedia.co.uk/Flights, do a roundtrip search for flights and switch to Return Flights tab on the results page.

Does any have any clue why this is happening?

1
webdriverwatirwatir-webdriver
You have several variables in play, both browser and the driver you are using.. Might I suggest trying it with watir-webdriver vs IE8 (and maybe also chrome) so we can see if this is equally slow there? If it's bad under watir-webdriver under either browser, vs if it's only firefox, could tell us a lot more about where the issue may lie. Might also want to try firefox 3.6 as a significant number of FF users are holding to that version and not upgrading due to plugin compatibility issues. – Chuck van der Linden
hi Chuck. watir-webdriver on IE8: 1 => 0.172017, 2 & 3 => Timeout::Error: execution expired from C:/ruby/1.8.7-p334/lib/ruby/1.8/timeout.rb:64:in `rbuf_fill' ... watir-webdriver with FF3.6: 1 =>0.055005, 2 => 80.095009, 3 => 101.734173. Page - do a roundtrip search from expedia.co.uk/Flights and switch to Return Flights tab. Timeout::Error - will raise as a separate question. Tests passing yesterday started to fail with timeout today (no updates to gems were made) - FF8.0.1. But clearly last 2 operations in IE8 took more than 1 minute before failing with exception. – alex.ikhelis
OK I edited the question to insert that data. Interesting that in IE it times out after a minute or two but you said for FF8 it just kept cranking for over 30 minutes. I'm not enough of an expert on the internals of webdriver, but hopefully this will give someone who is a bit more to work with. – Chuck van der Linden
You can report it as a bug (make sure you provide relevant HTML) github.com/jarib/watir-webdriver/issues – Željko Filipin
Thank you! I have raised it as github.com/jarib/watir-webdriver/issues/108 – alex.ikhelis

1 Answers

0
votes

One thing we have done at times when a query becomes oppressive is to get out of the selenium/watir loop altogether and ask the browser to execute some javascript to grab the elements we want. Obviously this has some limitations especially if you need to use watir on the object you get back (you can't). But if you are looking for something in particular, you can't get much faster. I am assuming you are using jQuery, but in reality the JavaScript can be arbitrary and whatever you need.

I think this would work...(but don't sue me if I have an error ;P )

browser.execute_script("jQuery(\"div[id*='flightModule40']"\")")