How to run MTM tests on multiple product builds?

1
votes

We have MTM tests running on Release build of our product (Desktop Application). Now we want the same tests to run on two product builds: Beta and Release.

When a test run is initiated from MTM (or tcm), we need a way to pass a 'value' to the test run telling it which version/build of the product it needs to test. This 'value' will then be read in the test method and correct decision will be taken while the tests are executing (like installation path, test results file updates etc).

Is there any way to achieve this? in TFS or MTM?

4
tfscoded-ui-testsmicrosoft-test-managermtm
Not sure I understood, what do you want to do, correctly. You are currently starting your test from MTM, right? You want to deploy/install both versions of your product (Beta and Release) on the same environment using different installation path, right? What exactly do you mean with "parallelly"? You want to start your tests from MTM and what exactly do you want to happen then? – Elena
Yes we are starting tests from MTM. At times we need to test Beta build and other time we want to test Release build on the same environment. So while starting test execution from MTM we want to indicate that we want to run tests on Beta build (or Release build) and based on that our CodedUI test will install correct product build. Hope I am clear now. – userhpkj

4 Answers

1
votes

Consider using Test Settings.

If you start an automated tests from MTM you can specify Test Settings to use when running this tests.

In "Advanced" part of Test Settings you can specify scripts to run on your environment before running the tests.

  1. Create two scripts, one for Release and one for Beta version. These scripts could create a file with particular content, set an environment variable or do something else that can then be checked by your test, when it’s running.
  2. Create two Test Settings, one for Release and one for Beta version and set up appropriate script to run for each Test Settings.
  3. Use one of these Test Settings when starting tests.

This way you could pass information to your test.

0
votes

Pls take a look at the below link, if it can be used to suit your needs.

http://blogs.infosupport.com/switching-browser-in-codedui-or-selenium-tests-based-on-mtm-configuration/

one question: Are you using Build-Deploy-Test flow to install the product on the environment or doing it any other way?

0
votes

So, when you select to run a set of automated tests and pick the build from the drop down list this tells MTM which drop folder to go look in. So if your configuration is code, as it should be, then you can set this up to be automatic.

It is not possible to pass additional variables when you start a test run in MTM.

You could setup your tests to run from the Release Management tool instead. You would then be able to configure the environment however you like based on passed in veriables.

http://nakedalm.com/execute-tests-release-management-visual-studio-2013/

0
votes

We also faced similar problem in our project. We decided to modify the build definition template to take product build type (Beta or RTM or Release) as an input parameter. Using this value during TFS build, we can either update the TFS build name to reflect the product build type or create a file (xml) as part of TFS build process to contain this type detail.

See here for more detail on how to add Arguments and Parameters to build definition: http://www.ewaldhofman.nl/post/2010/04/27/Customize-Team-Build-2010-e28093-Part-2-Add-arguments-and-variables.aspx