Do Watir-Webdriver and Capybara have performance issues when compared to Webdriver?

9
votes

I am about to change my test automation language from Java to Ruby (I have new job where Ruby fits better into the tech stack).

I have a lot of experience with Java and Webdriver but can see that wrappers such as Watir and Capybara appear to be used more in Ruby than directly accessing the Webdriver API.

My concern about using a such a library is performance. I normally try integrate 3rd party grids such as Saucelabs into my test frameworks but have learnt that caching of selenium web element objects is important as continually finding elements can have a performance impact.

If I use libraries such as Capybara, do I lose the ability to control caching strategies? I have previously investigated Geb and found that the framework continually re-created webelements rather than caching, and it appeared inflexible in changing that behaviour.

Is my concern that these libraries help you to avoid writing boiler plate code but at the cost of performance valid?

3
rubyselenium-webdrivercapybarawatir-webdriverselenium-grid
Can you give an example when you need to continually find an element? I also have a test suite but I think I don't have such an issue there. – Andrei Botalov
In my experience something as simple as el = find('#id'); el[:id].should == expected_id; el[:name].should == expected_name usually helps – Andrei Botalov
Most of the time I want to access the element only once so I don't feel need in caching by default. – Andrei Botalov
Do you use the page object pattern? – Robbie Wareham
Not in the form as it's written here - code.google.com/p/selenium/wiki/PageObjects. I use something similar. For example I don't follow rules "Generally don't make assertions", "Methods return other PageObjects", "Different results for the same action are modelled as different methods" – Andrei Botalov

3 Answers

3
votes

TL;DR

Convention over configuration; use page-object for caching.

Here are my thoughts on this matter. And please consider this less of an answer, and more of a response for discussion. I want feedback on this answer, feel free to give it.

One of the primary patterns in Ruby (and it actually comes from Rails), is Convention over Configuration. The basic idea is that when there is a convention, either dictated by the language or the community, you roll with it whenever possible. In your case, I would recommend using the community frameworks if at all possible. This will make it easier for other developers to use your code and easier to seek help if you need it.

As far as the actual caching goes, with this I am less familiar. I do know the page-object gem stores elements instead of recreating them during every usage*. This would appear to conform with your requirements of caching elements. In any case, I highly recommend this gem as it enforces the page-object model of testing.

Note: I am not sure if the page-factory mixin supports this caching of objects, or if it recreates the class on every usage.

*You can see how the Element is stored in page-object by viewing the source code to the Element class.

def initialize(element, platform)
  @element = element
3
votes

I won't comment on performance in general as I haven't done extensive comparisons (and you shouldn't really believe anyone who can't point to benchmarks). However, here's a performance tip (with or without wrappers):

Internally, WebDriver uses HTTP to send commands to the browser (or an intermediate server). Since the protocol is quite chatty, you can often gain a lot by switching out the default Net::HTTP-based client with one that supports Keep-Alive (assuming the server supports it). That way the Ruby client will open one socket per browser and reuse it for the duration of the session, instead of opening one per command.

Here's code that uses the net-http-persistent gem:

require "selenium/webdriver"
require "selenium/webdriver/remote/http/persistent"

client = Selenium::WebDriver::Remote::Http::Persistent.new
driver = Selenium::WebDriver.for :remote, url: 'http://...', http_client: client
2
votes

Unfortunately, I can't say anything specific about Capybara, but I can about watir-webdriver. By default, it always relocates element on the page, so you can't cache it. This behaviour can be turned off with

Watir.always_locate = false

though you can experience stale elements in this case (it, however, actually depends more on you just like with plain WebDriver). If you're planning you use watir-webdriver with page-object gem, its caching won't work as described in @screenmutt answer by default.

Also, there are two nice blog posts from Zejlko Fillipn about watir-webdriver performance when using SauceLabs and TestingBot.

Another issue of watir-webdriver is cross-browser performance. Because it uses XPath under the hood, you'll face significant performance problems when testing in IE.