Automated tests and Multiple Configurations

4
votes

I am using Microsoft Test Manager 2012 to create and run both automated and manual tests. I have defined 2 configurations: one for my tests that should be run on SQL Server and one for tests that should run on Oracle.

When I create a test case MSTM automatically create two tests: one for Sql Server and one for Oracle. They have the same ID wich means that they are the same test. So far so good. That is what should happen.

But I want to automate both tests. When I create my code and associate it with the test case both tests gets the automated code. Thats bad because I want test 1 to run on SQL and test 2 to run on Oracle and they are sharing the same test code.

Does It mean that I have to put some logic in my tests so they know in which Database they should run on? Is there a better approach to avoid that?

1
tfscoded-ui-testsmicrosoft-test-manager
Do you have different environments for running automated tests: one for SQLServer and one for Oracle? Do you want to use the same test code for both (just using different DB connections)? (I'm not sure what do you mean saying you have "test 1 and test 2 are sharing the same code". To what extend are these tests identical?) – Elena
Another question is how are you going to start the automated tests? Using DefaultLabTemplate11 build definition template or from command line (batch script) calling tcm.exe? – Elena
@Elena I only have 1 environments for running my tests. I want to use the same test code for both, using different DB connections ou another technique. I am using DefaultLabTemplate11 but I can change it. – Rafael Colucci

1 Answers

1
votes

Option 1
Create two different Test Cases, one for SQLServer and one for Oracle and then to automate first by Test1 and the second by Test2. So you can run both of them by a single build definition which is using DefaultLabTemplate11.

Using suggested approach the test code will not have to implement additional logic in order to recognize which database they should run on.

You said that you want to use the Test Case for both manual and automated testing, so I suppose the Test Case you have today consists of several Test Steps describing how to run tests manually.
In this case you can clone existing Test Case and use shared Test Steps but you will still need to update both Test Cases when you add/remove some Test Steps... this will be the only disadvantage of suggested approach.

In order to get rid of this disadvantage you could create "Manual database tests" Test Case, assign two Configurations you already have to it and use it for manual tests only. This Test Case will contains all Test Steps for the tester.
Then create two Test Cases I've described above, automate them and use them for automated tests only.

Option 2
If your test environment is a virtual machine you could avoid creating two Test Cases:

  1. Store the connection string in a file on your test environment and let the test read this file.
  2. Create a two snapshots: the first with the file containing SQLServer connection string, the second containing Oracle connection string.
  3. Create two build definitions one reverting the environment to the first snapshot and the second reverting it to the second snapshot.

This way you can keep your single Test Case but on the other hand having two build definitions can become unhandy if the build definition is supposed to build the source code before deploying and running tests.

Well... the third option will be to implement additional logic in the test in order to recognize which database it runs on.
But in this case you will also have to create two build definitions since you have two Configurations and you can only select one per build definition.

(I hope I've managed to describe it well, don't hesitate to ask it something is still not clear).