Run Functional Tests step in Vnext TFS 15 is awfully slow

0
votes
  • Running functional tests in TFS 15 with Vnext in comparison to the old system with MTM and Tests Environemnts, it is awefully slow. it takes like 10 minutes after initial test start, before the first tests are started. And while running the tests, they take longer as normal.
  • Distribution of tests is slightly "unhappy", tests get distributed at the beginning of the test run, but if 1 maschine is finished, while the other one has still 5 long test runs doens't make sense. Bucket size was a way more intelligent system

is there are a way to improve this? We have updated to RC2 and we are not happy with the test outcome. Feeling like test tast is a bottleneck

1
unit-testingtfsfunctional-testing
Depending on how fast you get responses here, it may be helpful to submit an issue on GitHub. Microsoft folks usually respond to me within a few hours after I submit one. github.com/Microsoft/vsts-tasks/issues – m00nbeam360.0
Hey Sachi, good idea. Submited my concerns. Thanks for the link – Ingo B
You can post back the corresponding solutions here once you figured out the issue there. And did this performance issue only occurs with functional test? Or also with some other tests even build ? Moreover, when you say "old system" did you also use TFS with MTM and Tests Environemnts in it? – PatrickLu-MSFT
Hey Patrick, haven't got a response so far. We have tested it against TFS 2013 MTM. TFS 2015 MTM and TFS 2015 Vnext. Problem only occurs at functional test step. – Ingo B

1 Answers

0
votes

Ok so case is that the tests get distributed at the start of the test run and the test runner itself works different now. Unlike bucket system that distributes the tests one after another.. Also the tests only shows if failed or finished AFTER they are done, so if 200 tests get distributed it will only show the outcome when all 200 are done. kinda awkward..