QA automation flow

1
votes

Right now I'm doing automation testing for a small company. I have been doing this for about half a year. But there is a problem: I have absolutely no experience creating such a system. From sticks and guana I created a working system and even implemented it, but I feel that many things can be improved.

I am using TestRail and Java.

At the moment the picture looks like this:

  1. Create cases in the TestRail (test management system).
  2. Unite cases in to suites.
  3. Write code for every test.
  4. Mark every test with annotation for matching test code and test case from TestRail.
  5. When tests is launched they will create test run for every suite.
  6. Then run tests with run id as argument.
  7. Tests will ask TestRail for list of test cases from this run.
  8. Using reflection they will find all required classes.
  9. Run them via JInit 4.
  10. Send report into TestRail.

I am sure that such a mechanism of test runs with test case control already exists, but Google only tries to sell me qa automation lectures.

Do you have any suggestions about:

  • tools, that I should use;
  • flow moments, that can be improved.
1
testingautomationautomated-testsqaflow

1 Answers

2
votes

Maybe use Jenkins for test execution? There is a Jenkins TestRail plugin and for TestNG reporting you can use Jenkins TestNG plugin.

Let me share how I do it, maybe it will help you and will give you some ideas about the automation process and test case execution.

I have set of test cases for manual execution that are stored in JIRA using Zephyr JIRA plugin. Test cases are categorized and tagged.

I identify test cases that are good candidates for automation. Using Java, Maven, Selenium and TestNG I automate those test cases.

I group automated test cases in test suites if needed. TestNG allows to do it.

I execute automated tests in various ways. Sometimes I execute them on local virtual box running Linux Mint with MATE desktop. In my case virtual box with 2 CPU cores and 8 GBs of ram handles 5 parallel threads. Sometimes I execute automated tests on AWS instance where I use docker selenium containers. I have used c4.4xlarge AWS instance and ran ~30 selenium docker nodes which gave me ability to run 30 tests in parallel. You can write a script that will set AWS instance, install docker, run docker selenium hub and nodes, check out your automated tests from your repo to that AWS instance, execute your tests, gather the results and kill the instance.

You can set Jenkins job to execute your automated tests and have testng jenkins plugin to show reports in Jenkins.

You can also integrate automated tests with Zephyr JIRA plugin API, so when automated test case pass/fail - it will mark Zephyr test case in JIRA passed or failed.