How to approach "end-client" performance testing on single-page (web) applications?

10
votes

We are evaluating how to test performance on a single-page application (SPA) which relies heavily on JavaScript and dynamic content (updated via Ajax).

Popular load-testing tools like Apache JMeter or Gatling are able to generate huge loads with little hardware by sending HTTP requests. But they do not process any Javascript code.

We would like to measure performance as perceived by the client, that is, as perceived by end-user sitting in front of a browser (with all the rendering and JavaScript overhead included).

We have the following alternatives in mind:

  1. Use Selenium Grid to run a test script concurrently on several machines. Each instance submits latency data collected at runtime to a central repository.

  2. Similar to above, except that only one Selenium script instance is started. We would then JMeter/Gatling to generate the heavy load in the background.

  3. Do not use Selenium at all. While running a load test using JMeter/Gatling, manually inspect the behaviour of the page by using tools like Firebug, YSlow, etc.

Each approach has its strengths and drawbacks. On (1) I'm able to create the most "realistic" load, but it does not scale, (2) scales through JMeter/Gatling, but I have to make sure the load they create do not deviate too much from the actual application and (3) is the easiest to setup, but it's obviously time consuming since it's not automated.

So far I could not find any tool, framework, or even a set of guidelines on this topic, so I would like to know from experts here at SO: what kind of approach you they use for this type of test?

This entry on the Selenium Grid FAQ states that *"conducting performance/load testing with real browser is a pretty bad idea as it is hard/expensive to scale the load and the actual load is very inconsistent...". I can only think that they refer to "backend performance/load testing". Otherwise, if I'm interested in end-user performance, what's the alternative?

1
javascriptperformancetestingsingle-page-applicationperformance-testing
Well what you information will you get by testing your SPA. I don't think that it will have any valuable information; The performance is mainly dependent upon the JavaScript execution; So it will be different on each machine. You can test on yours, but that will not give you an insight of how it will perform on your clients machines. However, you still test the performance of your Apis since they partially affect the speed of the whole application. – ppoliani
I understand your point, but I disagree. I can have very fast API's, but the application will still be slow if the JavaScript code is poorly written. – E.Z.
I didn't say that the overall performance is dependent on how fast you API's are; As for your JavaScript performance, you can test it on your browser and see if there are any bottlenecks. You can search for common issues like, memory leaks, slow methods, repaints/reflows etc. I think that solving those issues is enough to assume that your app will run pretty fast on your clients' machines. – ppoliani
Of course, profiling javascript. How come I didn't think of that. Developer tools looks promising developers.google.com/chrome-developer-tools/docs/cpu-profiling – E.Z.
See also github.com/paulirish/automated-chrome-profiling – Ben Creasy

1 Answers

11
votes

Check out JMeter WebDriver http://jmeter-plugins.org/wiki/WebDriverTutorial/ . You can define one parallel to your load test in order to gauge performance perceived by the client while the server is under load.

From their site:

Web Driver Sampler automates the execution and collection of Performance metrics on the Browser (client-side). A large part of performance testing, up to this point, has been on the server side of things. However, with the advancement of technology, HTML5, JS and CSS improvements, more and more logic and behaviour have been pushed down to the client. This adds to the overall perceived performance of website/webapp, but this metric is not available in JMeter. Things that add to the overall browser execution time may include:

Client-side Javascript execution - eg. AJAX, JS templates CSS transforms - eg. 3D matrix transforms, animations 3rd party plugins - eg. Facebook like, Double click ads, site analytics, etc All these things add to the overall browser execution time, and this project aims to measure the time it takes to complete rendering all this content.