Test Automation framework - Tools important?

0
votes

I have been working on Test Automation from last few months and have been using the tool named "Testcomplete". But I have noticed that the tools do not matter a lot in the field of automation. Only thing you expect from an automation tool is the ability of the tool to spit out the recognition strings for the different controls used in the test application. Apart from this, you will always have to build a automation framework which will serve your needs writing code.

So my question is, Is my thinking that automation tools do not matter a lot in the field of automation correct? In the sense, you can use any tools to get your automation running. Or Do the tools really matter? (Please ignore the costs factor of the tools). Also if I need to learn a new automation tool, then what do I concentrate on? Or how do I go about learing the tool? In short, what exactly does "learning a tool" mean?

4
automation

4 Answers

4
votes

My 3 best reasons for choosing which tool to use:

  • it works. This is important, not all tools work in all scenarios i.e. flash, silver light, adobe air, legacy apps with no automation support, etc.
  • whole team skills. This includes not only testers, but also developers. Test automation shouldn't be an isolated effort, developers should also collaborate on it. This is far easier when dev and test are using the same language/platform.
  • price. Doesn't have to be free (but it could), but of course its an important factor.

Personally we use the same test runner as the one for the unit tests. That along with extra third party automation pieces that do the plumbing for you.

3
votes

Some additional thoughts on why the tool is important:

  • Community - What's the user community like? Are there a lot of user-generated resources out there to help?
  • Support - (if vendor) What's customer support like? Do they fix problems quickly? Is it easy to find solutions to common problems?
  • Extensibility - Often in test automation, you'll need to roll your own or code work-arounds, if the tool does not support a particular type of object in your application. How easy is it to extend the product? What programming language does the tool use? What kind of support do you get from the IDE?
0
votes

An other piece of advice: sometimes you'll need wrapper classes around certain frameworks. We were using WatiN, which was really good at its time, but it lacked Chrome (it had a small percentage that time) support. The thing that killed WatiN for us was the lack of coping with new FireFox releases: FireFox 8 was out, and we had to run on our tests on FireFox 3.6... Selenium was the solution, but it had a totally different logic and we already had more than a 1000 tests.

So we had to create a wrapper class around Selenium to "fake" it was WatiN. We had some issues, but we had to rewrite only some special cases... And not all tests.

The point is, sometimes, support for frameworks just cease to exist. But with an own framework focusing on what the test actually does instead of how it works would save you in this situation.

0
votes

Variety of Test Automation Frameworks and Test Automation Tools are already available in the market. Thus, I would not recommend the built your own Test Automation Framework at all.

As far as selection of automation tools is a concern, I would say it does matter on the following basis:

  1. Support: How much level of support you have when you are choosing an automation tool for your project.
  2. Community: How big community is using that tool and how responsive that community is about sharing knowledge?
  3. Pricing: (Proprietary or Open Source) Last but not the least is the pricing of the automation tool that you are planning to introduce in your project.
  4. QA teams' expertise also matters sometimes. For example, in case your QA team does not have a developer or semi-developer skills vs Not-Technical QA Team, etc.

Regarding the Automation framework, there are many automation frameworks also available in the market already, therefore no need to reinvent the wheel. and selection of automation framework mostly depends on your selection of scripting language.

For example, if you choose python as your scripting language, then you have option to choose UnitTest, PyTest etc. as an automation framework.

In case of Java, you have option of JUnit and TestNG as an automation framework.

and so on, the base on your selection of scripting language.

Finally, when it comes to structuring your automation framework, it solely depends on many things as the following:

  1. Your nature of the project
  2. Single product vs multiple products
  3. and many more... Check an example of multiple product automation project directory structure. https://github.com/pancht/python-selenium-framework

I hope, in some way, I would have helped you out in giving an answer to your question.

Thanks,

Panchdev Singh Chauhan