Profiling tests in Visual Studio Community 2015

Question

Posts on the web suggest that you can profile tests in Visual Studio by right-clicking and selecting "Profile Test": http://adamprescott.net/2012/12/12/performance-profiling-for-unit-tests/ But I only see "Run Test" and "Debug Test" for my NUnit tests, and the same for xUnit.NET. What am I missing in order to profile tests? Is this just not supported in Community edition, or I am missing some configuration or component?

(It would seem odd if it's not supported in Community, given I can profile executables in Community, and thus could painfully work around this issue by creating an executable that runs the test, and profile that. Why support profiling executables but not profiling tests?)

Steps to reproduce for NUnit: created new C# library project in Visual Studio Community 2015, pasted content of http://nunit.org/index.php?p=quickStartSource&r=2.6.4 into new file, installed NuGet packages as follows:

<packages>
  <package id="NUnit" version="2.6.4" targetFramework="net452" />
  <package id="NUnit.Runners" version="2.6.4" targetFramework="net452" />
  <package id="NUnitTestAdapter" version="2.0.0" targetFramework="net452" />
</packages>

Even restarted Visual Studio. Tests show up in Test Explorer and can be run, but no "Profile Test" option available on right-click menu. Also tried equivalent steps for xUnit.net, but no joy.

Answer 1

This ishow I was able to profile an NUnit test in VS Community 2015 earlier today.

Profiling the NUnit Test Runner as an executable

1. Be sure you're running VS2015 as Administrator.
2. Click Analyze > Performance Profiler... from the Toolbar.
3. Choose Performance Wizard and click Start.
4. Page 1: In my case, I wanted to see allocations so I clicked .NET memory allocation.
5. Page 2: Leave the option An executable (.EXE file) checked and continue.

6. Page 3: On this page you have to define the executable to run.

   This will be the test runner nunit3-console.exe for NUnit, or whatever the equivalent is for your test framework.

   • What is the full path to the executable? C:\Program Files (x86)\NUnit.org\nunit-console\nunit3-console.exe
   • Command-line arguments: bin\Debug\Test.dll --inprocess --test TestNamespace.TestClassName.Test_Method_Name
   • Working directory: \\MAC\Home\Documents\GitHub\ApplicationName\Test

You will need to substitute these paths with ones that make sense for your system. The --inprocess switch causes the tests to be run inline from the NUnit process. Without that switch, a child process is spawned and even though the profiler will appear to work, you'll just be profiling nunit3-console.exe, not your own code.

7. Page 4: Click Finish.

   Keep in mind that the profiler will generate report files and save them to your working directory. In my case, since my working directory was a UNC share, it required me to pick a local folder path to save the reports to before the profiler would start.

   A terminal window should appear briefly with the NUnit runner output in it. The window auto-closes, so if you see a flash of red text, you won't have time to read the error before it is gone. You can copy the command from Page 3 into a command prompt for more leisurely reading.

8. After the command runs (whether it succeeded or not), you should get a report where you can track how many allocations your test caused.

Unfortunately, allocations in a small test will probably get overshadowed by the allocations caused by the NUnit.Framework itself. I clicked around to see if there was a way to exclude them from the results, but didn't find a way to do it, so I just ignored them.

If you want to profile a different test, you can open the Performance Explorer and right-click nunit3-console.exe > Properties to change the command line arguments and then click Actions > Start Profiling to refresh the report.

Conclusion

This solution succeeds at profiling the results of a single NUnit test, but that statement comes with some caveats.

It was only marginally less obnoxious than creating a separate executable to profile, and that fact that the NUnit allocations appear in the report could make it a non-starter if you need to do some really sensitive profiling.

Maybe someone with more VS 2015 experience can help me improve this answer with some tips on how to exclude the NUnit.Framework dll from the report?

Answer 2

It's possible to profile MSTest tests (can also profile nunit/xunit with test adapters) using vstest.console.exe

• Select executable program as the target from Analyze -> Performance Profiler
• Provide the path of the vstest.console.exe (usually lives in C:\Program Files (x86)\Microsoft Visual Studio 12.0\Common7\IDE\CommonExtensions\Microsoft\TestWindow)
• Provide parameters as <full path to test dll file>.dll /TestCaseFilter:"FullyQualifiedName~<namespace>.<test class>" as the parameters
• If you are using nUnit or xUnit, then provide nunit test adapter,or xunit test adapter path using /TestAdapterPath argument to vstest.console.exe

More info on how to provide parameters to vstest.exe

Answer 3

The answer appears to be in a comment on the Adam Prescott page referenced in the question:

August 16, 2013 at 4:57 pm

Unfortunately, per MSDN, this feature is only available in the Premium and Ultimate editions.

http://msdn.microsoft.com/en-us/library/ms182372.aspx

The link in the comment currently refers to the 2015 edition of Visual Studio. The 2010 edition of the page clearly shows which versions of Visual Studio 2010 support profiling. The pages for the 2012 and later version omit a clear statement of which versions support profiling.