TFS Visual Studio 2012 - Build and send remote automated tests to pool of VMs

Question

I am having difficultly trying to come up with a working automated build and automated testing solution for testing several applications remotely.

I have several different applications which install onto Windows test machine VMs (Win 7) and an automated test solution which is built using a test settings file and test case filter and run automated tests against this application.

I have two groups of VMs, one group to build the automated tests solution which then sends the tests remotely to the second group which has the application installed on.

Currently this is done by a default template build definition (VS 2012) which builds the test solution on a build VM and sends the tests remotely to a test VM (has application needed manually installed prior to starting build), defining a test controller in the test settings file (there is a test settings file per test VM / build definition) the tests are sent to that test controller. Each test VM has a test controller and test agent on the same VM to prevent the tests going to multiple machines, the tests need to be sent to one machine with that specific application installed.

I am wanting to scale this process and allow for complete automation, so I can just kick off a specific application build which will install the required application on a free test VM, build the test solution code on a build VM and send the tests remotely to the test VM which now has the application needed installed on it, run the tests and send the results back.

I am having issues with doing this between the build and test VMs and having the test settings file on the build machine updated with the test controller name of the free test VM.

Is something like this possible to do and if so what would be the best way to go about this?

Any help or feedback would be greatly appreciated.

Setup:- TFS / Visual Studio 2012 and Windows 7 build / test machines

Reference links:- How can TFS build process be configured to execute tests on Test Agents through a Test Controller?

Answer 1

Ok, so you need to look at Lab Management for configuring environments. TFS has a built in method of saying that a set of machines go together and that you want to run tests against it. You can then use MTM to push groups of automated tests to environments and record the data.

In addition you should look at Release Management for VS 2013 (works with 2012) for pushing out the bits to your environments.

With those two tools you can maintain release pipelines where not only can you push tests but they pull tests as part of the release process.

http://nakedalm.com/execute-tests-release-management-visual-studio-2013/

You can have a build kick off the process of build->deploy->test and repeat the process easily...