TFS Test Agent Configuration Tool for Selenium UI tests

Question

My team has several Selenium Unit Tests that we run locally. Selenium opens a browser and runs through our application. This works fine locally. Now, management wants to run the tests, on demand, via TFS. Should be easy, right? WRONG!

I am having trouble setting up our TFS 2015 server to run these Selenium UI tests.

Everything I read tells me to run the "Test Agent for Visual Studio 2015" tool. Problem is, the tool does not exist on my machine and I cannot understand why. I have tried many things, like installing the MS Visual Studio "Agents", with no luck.

When I try to open the freshly installed "Agents" I get a message saying: "Test Agent for Visual Studio 2015 has no configuration tool".

So the VS.NET "Agents" panel never opens. Seems bizarre to me that I installed something that needs further configuration, with no way to perform that configuration.

From further reading, I think I need to provision a group machine in the TFS web administration page, but am totally unsure if that's required for my situation, as I cannot even get past the basics here.

I see other people have had this problem, with very little response or help from Microsoft. I am a little baffled at why this seems so very difficult to do on a TFS server? It runs locally just fine, like a breeze. But TFS? It's like a giant puzzle.

Once this is working, I need to configure the test agent to run in "interactive mode" so it can run the browser but I cannot even begin to figure that out yet. Where do you set it to "run interactive" because I dont see any of those options. Am I missing TFS installation components? Do I have the wrong Visual Studio? Do I need the Ultimate edition of Visual Studio to be able to perform UI tests with the browser? We have the Professional edition.

Here is someone with the same problem: https://connect.microsoft.com/VisualStudio/feedback/details/1712725/test-agent-for-visual-studio-2015-has-no-configuration-tool

Does anyone have any ideas or instructions on how to setup the "Test Agent" I need to run my team's existing Selenium UI tests in TFS2015? Seems very difficult when it should not be.

Answer 1

In the link you provided, Allen has explained that "There is no configuration UI with the test agent anymore. This is because we have simplified the existing remote testing scenario by doing the install and configuration for you when running via the build pipeline."

TFS 2015 Update2 now have a "Run Functional Test" task that you can use to run tests (included Coded UI Tests) against machine groups. So, first you have to upgrade your TFS 2015 to TFS 2015 Update2 if you haven't.

The tasks you need for test scenarios using Visual Studio 2015 and Visual Studio Team Services (VSTS) or Team Foundation Server (TFS) 2015:

1. Create environments from physical or virtual machines that you've already set up.
2. Set up your build to run your app and tests in the environments that you created.
3. After your build finishes, review your test results to start resolving problems that you found.

So, your build process template should look like the screenshot below. You need to specify every task to meet the requirement of your project. All TFS tasks can be found at this website, you can get more information for each task from it. Coded UI or Selenium tests that are running on full fidelity browsers would need Interactive Process checked.: