Web application testing using FitNesse and soapUI - any best practice on test management and maintainability?

Question

To all test automation experts :-) ! I'd like to hear your opinions on the following scenario:

There is a web application that I need to test. I have to run back-end tests on the server and front-end tests on the client. I also need to run end-to-end tests, involving both the back-end and the front-end.

The server exposes webservices (SOAP) and the front-end client consumes data from these services. There are also third-party clients consuming data from the web services. Sometimes, a test scenario requires that i do end-to-end tests, i.e. I make some changes in the front-end GUI and then use a web service on the back-end to find out whether the changes were successful or not.

I like FitNesse - in my opinion, the separation of WHAT and WHY from HOW is essential for designing good tests. There is the Selenesse module, which makes it possible to integrate Selenium tests with FitNesse wiki pages. This makes it easy to describe what and why I need to test something (wiki text) from how I want to test it (scenario tables and script tables) which is how I want things to be.

The problem with FitNesse is that it is somewhat cumbersome to test SOAP web services. Either, I need to develop a purpose-built SOAP client Java fixture, or I have to write Java fixtures extending the ServiceFixture class, written for FIT. Either way, the development effort is significantly larger than if I implement these tests in soapUI.

In my opinion, the drawback with soapUI, is that there is no easy way of explaining the WHAT and the WHY of a test (at least not in a way that is intuitive).

So, assuming I want a reasonable development effort for end-to-end testing, I have settled for the approach of writing GUI tests in FitNesse/Selenesse and back-end tests in soapUI. I now have the choice of trying to run soapUI tests from FitNesse, managing all tests there, or to run FitNesse tests from soapUI...

I have some concerns regarding test management (not so easy to see the test results in one view) and maintainablity (two tools with different laguages) of this approach. Do you have any ideas for best/good practice regarding this? Would you suggest a third tool for managing the other two?

Answer 1

Dou you use any continous integration tool like hudson, bamboo ?

I am asking this question because I suggest you to prefer continous integration approach so you can have the opportunity to test the applications automatically after each commit/build.

I mean if you use hudson or bamboo you can have the chance to run your tests after a developer commits anything. And additionaly you can run your test schedually.

One other advantage is, these tools (hudson/bamboo) can log the test scripts and can send an email in case of failure/sucess (your choice). So you can monitor your tests easily.

And also you have thr opportunity to run selenium and soapUI in parallel or consecutively.

---

I also have some suggestions about the soapUI tests.

The more test cases you have, the more time you need to develop, execute and maintain them. The important point is to consider maintainability while designing the tests.

If an application has multiple web services available, the WSDLs are bound to change and need to be updated in SoapUI. With everything in one soapUI project you only need to update the WSDLs in one place, not in multiple projects. So create just one soapUI project for one application.

Then you need to create test suite and test cases.

Include all the services’ main flows (success scenarios) in just one regression test suite. The requests of the web services should be ordered according to the logical business flow. For example if you test the web services of an online store you need to firstly search the item and then buy it. If you keep this logical business order within your soapUI tests, you can easily set a single global variable for each test step. I mean, at the first step you can search for the item X then buy the same item, this way allows to set a global variable for item X. It is easier to maintain or extend such a soapUI project. You have the opportunity to create a data source and collect variables (different items in our online store example) and extent the case for those items in a loop.

Answer 2

I recommend you create test suite with soapui and report test results with jenkins.

You can execute soapui tests and fitnesse tests with jenkins and genrate xml test result file. This setup is really useful when building end2end tests. We can tie any testset or test suite together with Jenkins really well and you have to great way of presenting and preserving tests results.

When focusing on a working component, getting things done in the sprint or the complete app is not stable enough to invest huge amount of effort in working end2end test already I think you should focus on working soapui test separately instead.