I have a C# project in Visual Studio 2017 Professional. It has some unit tests: methods declared with [Test] which display a little green tick icon in the editor, where I can right-click and 'Run' or 'Debug' the test. When I do so, a window opens with all tests in the solution and I can click a green 'play' triangle to run them all.
I can also run my program under the profiler starting from Debug -> Performance Profiler.
So far so good. Now, how do I run a unit test under the profiler?
When I first asked this question I looked for profiling unit tests and saw it was a feature in Visual Studio Professional. But in fact, I'm not using VS's own unit testing support, but rather NUnit, driven by Resharper. Sorry to throw everyone off the scent.
Using VS2017 Professional and the Diagnostic Tools you can profile code between two break points!
Thus, you can debug your unit test (xunit, nunit, mstest, whatever!) with any tool (VS Test Explorer, ReSharper, etc) like this:
enable diagnostic tools while debugging (see answer from tm1)
Set a break point at the start of your unit test, another one at the end (or wherever you like)
Debug the unit test with your favorite tool, you hit the first break point.
Open the Diagnostic Tools window (Debug > Windows > Show Diagnostic Tools)
Start Recording a CPU profile (small record button below all those graphs)
Continue (F5) to the next break point
VS will now create a snapshot for this period and you can dig into the relevant methods. Voila!
Just for completeness, an alternative, commercial solution is Jetbrain's DotTrace, which is part of ReSharper Ultimate at the point of writing. Here you could directly profile your unit test without manual break points.
If you can debug and stop at a break-point, you should also be able to get CPU and memory usage according to this page, given you enable diagnostics tools while debugging.
Edit
According to #patrick-stalph the 3-step proposed solution does not work in Visual Studio 2017 Professional.
Disclaimer
I'm using Visual Studio Enterprise 2017, so I cannot check, whether this works in Professional, or not.
Open Test Explorer e.g. Ctrl + Q, then type Test Explorer and hit Enter
You should see your tests in Test Explorer. If you cannot, Build.
Choose a test, right click it, and choose Profile Test
Workaround
I must note, that I'm using xunit and autofixture, and I haven't seen this feature working. See this issue, why I haven't.
Hence, based on the answer of Jeremy Liberman, I was able to work around using the steps below:
Install xunit.runner.console NuGet package to the test project
Click Analyze > Performance Profiler... from the Toolbar.
Change Target to Executable
Click Start
Follow the Wizard, till you reach Choose the application that you want to profile (.EXE, .DLL, Website)
Choose An executable (.EXE file)
Based on Run tests with the xUnit.net console runner, I've specified the following values:
Full path: c:\<...>\src\packages\xunit.runner.console.2.4.0\tools\net472\xunit.console.exe (fit this to your environment)
Command-line arguments: <Product>.Tests\bin\Debug\<Product>.Tests.dll
Working directory: c:\<...>\src\ (the root of the Solution)
Next, Finish
Note, that this profiles all of the tests in the test assembly. You can choose, which tests you want to run by providing more Command-line arguments to xunit.runner.console. To learn more about the console runner options, run the console runner with no command line options.
My problem is very similar to NUnit Unit tests not showing in Test Explorer with Test Adapter installed, in that I couldn't get the NUnit tests to appear in the Test Explorer (despite multiple rebuilds, setting the Test architecture to x64, etc). I rebuilt and rebuilt, and despite having NUnit 3 and the NUnit 3 Test Runner installed into my Solution (via Tools -> NuGet Package Manager -> Manage NuGet Packages For Solution...) I couldn't get the tests to appear.
On the advice of another response to the above question, I installed the NUnit 3 Test Runner as an Extension (which modified all of Visual Studio, not just my project).
After the extension is installed (and I do Yet Another Rebuild :) ) all my tests are listed in the Test Explorer.
Question: Why do are the tests shown in the Test Explorer for the Visual Studio-wide NUnit 3 Test Runner Extension, but NOT in the Solution-specific NUnit 3 Test Runner NuGet package?
What's even weirder is that the exact same project, with the per-Solution NuGet packages for NUnit, work just fine on my personal computer (WITHOUT the NUnit Extensions installed), using Visual Studio 2017 Community edition. It's only when I .ZIP the project and email it to another computer (which runs Visual Studio 2017 Enterprise) where I have this problem.
I'd prefer to install NUnit into the solution so that I can hand out copies to my students and have the project be completely self-contained.
Does anyone have any advice on how to further dig into the problem with the per-Solution NuGet package? I'm kinda stymied since the only obvious thing to try is "rebuild it again and see if that helps" :), so any thoughts on stuff to try / strategies to use would be great.
At this point it looks like the Virtual Machine that I'm running the tests in thinks that the project is being run from a network drive. .Net has different rules for running code remotely and by default doesn't want to execute remote code (which seems reasonable :) ).
It's worth pointing out that NUnit should (and does!) work just fine in a VM where the project is saved onto a virtual disk - it's just the 'network drive'/file share that's causing the problem.
I'm using Selenium with C# to create my automated tests. Now i have to use some sort of reporting tools to save the test cases whether they passed,failed...etc.
I've seen a lot of tools like Allure, and Jenkins. But they require an XML file. and i can't find it when i run my tests using Visual 2013. Why is that ?
how to obtain these XML files in details if i'm missing something ?
Is there any way i could achieve this with minimum effort ?
EDIT:
How can i run my tests using Nunit console runner ? where can i find it? i downloaded the .zip for nunit but i couldn't find the runner?
The NUnit Visual Studio Adapter does not currently produce XML results, although there is an issue on GitHub to add the ability. Your best bet is to run your tests using the NUnit Console runner when you want to create reports. It always produces an XML result file.
You can install the NUnit Console by adding the NUnit.Console NuGet package to your test project. The console will be in a directory like packages\NUnit.ConsoleRunner.3.4.0\tools in your solution root.
Another option is to install the MSI for the console runner. It will then be under C:\Program Files (x86)\NUnit.org\nunit-console
For creating reports, ReportUnit creates excellent HTML reports from your test results.
To run the tests, I would recommend setting up a simple command line build that builds your solution, runs the tests and then produces the report. For day to day development, just running your tests in Visual Studio or on the command line will likely be enough. Most developers find the Visual Studio Test Explorer Window to be a poor UI, but usable for seeing passed and failed tests and running/debugging them.
If you want to setup a command line build, one good tool is Cake Build. It will take a bit of time to setup, but it is an excellent way to run your build tasks as your project gets larger.
NUnit testing can be run with the nunit-console.exe application which is installed with nunit under {Project_root}/lib/nunit/nunit-console.exe. It downloads with NuGet when NUnit installs.
It can be passed a list of testing binaries, or testing project files, or an nunit project (listing multiple if needed).
{PathToProject}\lib\nunit\nunit-console.exe {PathToTestDll}\Project1.Tests.dll {PathToTestDll}\Project2.Tests.dll /xml=nunit-result.xml
or create an NUnit Project with the NUnit Project Editor if you want to group all your test projects into a single config file.
{PathToProject}\lib\nunit\nunit-console.exe {PathToNUnitProject}\Project.Tests.nunit /xml=nunit-result.xml
Allure doesn't require xml. At lease now :)
https://github.com/unickq/allure-nunit
You just use [AllureNUnit] attribure
I am trying to use OpenCover for a calculator application, written in C#.
I have opened it in Visual Studio 2013 and installed the OpenCover and Report Generation packages using the package manager console. I cannot understand the other commands to continue. Specifically what does it mean by -target<target application>?
I cannot understand a single step to continue.
When you install OpenCover it would/should have presented you with a readme.txt that provides a link to the wiki http://www.github.com/OpenCover/OpenCover/wiki/usage - there is also a usage.pdf installed with the tool.
-target means the target application, this can be the app you have just written or a coverage tool such as nunit that would run parts of your application code whilst it is running the supplied tests.
If you're happy with any code coverage without it specifically needing to be OpenCover, then see my answer here.
Using the AxoCover plugin makes it very quick and simple to get your NUnit tests displaying their code coverage.
I have a simple solution in visual studio 2013 that is composed by one web project, one library project and one unit test project. When I open the solution and try to run the unit tests they are not discover by visual studio. To run the tests I try to go to the menu and choose Test -> Run -> Run all tests or by opening the test explorer window. By those to methods visual studio doesn’t discover any tests in the solution.
Creating first a simple unit tests project and try to run the test, visual studio know discover the test and I can run it. Then, if I open my previous solution visual studio now discovers all the tests. I try to save my solution but closing it and reopening, without creating a unit test project first, the visual studio doesn’t find the tests again. This is a very strange behave that I don’t know why this is happening.
I used to working alone in this project that was using the source control git integrated with the visual studio team foundation. The problem of visual studio not discover the unit tests start when a new element came to the project and when I need to recreate the solution through the source control online. Before this, all tests always been discovered by visual studio.
For creation the unit tests I use the dll Microsoft.VisualStudio.QualityTools.UnitTestFramework. My version of visual studio is: Microsoft Visual Studio Express 2013 for Web Version 12.0.30723.00 Update 3. My version of .net framework is 4.5.50938.
All of my tests are like this:
[TestClass]
public class Service1Test
{
[TestMethod]
public void Test1()
{
Assert.IsTrue(True);
}
}
Some things I've noticed I have to do from time to time to get tests to show up properly.
If your solution is in a protected drive that you need administrator access to read/write, sometimes only a portion of the tests come up. Definitely run VS as administrator in that case.
If your solution is 64 bit, make sure that Test > Test Settings > Default Processor Architecture is set to x64. Sometimes it gets set to x86. Set it to x64, then rebuild.
Sometimes just restarting Visual Studio does the trick because the test explorer will start up again.
Don't forget to actually build the test project/solution. (If you want it to get built with the rest of the projects, right-click on your solution > Properties > Configuration Properties > Configuration > check the "Build" box for your test project)
Ensure the tests are in a public section of your test class
If you using NUnit, make sure to download NUnit Adapter first.
Go to Tools → Extensions and Updates… → Online → search for "NUnit Test Adapter".
Make sure your test class is public so it can be found. And if you're referencing another class, make sure of the same.
Also, sometimes if you have no Asserts or you're not decorating the test with a [TestMethod], a test might not be recognized.
2 more things:
1) Async unit tests act funny at best, and none at all at worst. Have a look at this article by Stephen Cleary and keep from there if it interests you.
2) If you use NUnit and you run into the same issues, keep in mind it's [TestCase] for Nunit, instead of [TestMethod]
Having said the above, here's an article I've posted on the code project, with both MSTest & NUnit, in case you want to give it a spin and make sure you're not missing anything.
I had the same issue but none of the other solutions worked. Turns out that I was using the NUnit 3 framework with the 2 adapter.
If you're using NUnit 3, go to Extensions and Updates and install the NUnit3 Test Adapter.
I'm having this issue from time to time. What works for me is to shutdown Visual Studio and go to folder:
%LocalAppData%\Microsoft\VisualStudio\12.0\ComponentModelCache
and delete it content.
Once you open Visual Studio and load your project again Test Explorer should contain back your tests
XUnit users may notice Test Explorer window no longer lists any tests. To make tests discoverable again try this important tip, highlighted below.
If you're having problems discovering or running tests, you may be a victim of a corrupted runner cache inside Visual Studio. To clear this cache, shut down all instances of Visual Studio, then delete the folder %TEMP%\VisualStudioTestExplorerExtensions. Also make sure your project is only linked against a single version of the Visual Studio runner NuGet package (xunit.runner.visualstudio).
For future googlers I had a rare scenario that caused this.
On my base test class I had a property named TestContext. This interfered with MSTest's reserved TestContext property causing all my tests to be hidden from VS/Resharper except one (which did not inherit from the base).
for me it was changing 'solution configurations' to Debug (instead of Release).
My problem was because my unit test method wasn't void, and it was receiving parameters.
I have found that unit test methods marked as async void aren't discovered by the VS Test Explorer. This seems to be because VS would not have any way to wait for a test to finish and decide if it succeeded or not. If you absolutely need to have a test method to run asynchronously then get it to return a Task instead like async Task.
I found that this fixed the issue for me.
Try building all projects as MSIL (Any CPU) instead of x86/x64. Worked for me strangely
While AndyG's solution works, a more lasting solution could be to set the PreferredToolArchitecture environment variable to "x64", either by:
How to make Visual Studio use the native amd64 toolchain
or by:
Control Panel | System and Security | System | Advanced System Settings|Environment Variables
PreferredToolArchitecture = x64
DefaultToolArchitecture = Native64Bit
PROCESSOR_ARCHITECTURE = x64
ProcessorArchitecture = x64
I was facing the same problem and I've remembered, again (this situation happened before), that selecting "Mixed Platform" on the solutions platform menu works, as well as the other answers.
I'd managed to add mine as
public static void TestMethod1(){}
started working once I removed static....
Go to Nuget package manager and download Nunit Adapter as follow.
go to the project menu > Configuration Manager check your test project platform matches the rest of the project and is check to build then rebuild.
Just ran into this as well as I didn't see a similar case that was similar to mine.
In the .csproj file of my test project, NUnit reference privacy was set to False:
<Reference Include="nunit.framework, Version=2.6.4.14350, Culture=neutral, PublicKeyToken=96d09a1eb7f44a77, processorArchitecture=MSIL">
<HintPath>..\packages\NUnit.2.6.4\lib\nunit.framework.dll</HintPath>
<Private>False</Private>
</Reference>
After I set <Private> to True it worked.
You just need to install this package only:
NUnit TestAdapter
To get tests to show in the Test Explorer Window I had to install NUnit3 Test Adapter 3.0 which was not available in Package Manager.
Downloaded from https://visualstudiogallery.msdn.microsoft.com/0da0f6bd-9bb6-4ae3-87a8-537788622f2d
I had the exact same problem.
It was due to incompatible version of NUnit I'd added to my project (3.2.0) and the Test Adapter I had installed (2.0.0).
To fix, use "Tools > Extensions and Updates" and search for NUnit3 Test Adapter, it discovered my tests after that.
Cheers
Let's just say for argument's sake that you need to use X64 architecture on your test project in order for the dependencies to build properly (as in my case). You may need to modify your Default Processor Architecture under the Test--Test Settings menu. Setting this to X64 allowed my test explorer to find my tests using Microsoft.VisualStudio.TestTools.UnitTesting.
Sorry for adding to the long list, but I had a completely a different problem.
First, I would like to mention I discovered my issue when clicking 'Run All' in the Test Explorer and then watching the build output window in Visual Studio. You have to actively watch it, as afterwards the message disappears.
As for the issue, it looks like during the scanning of the tests, the DLL gets loaded and its test types are enumerated. This causes the references to be loaded and if any failure occurs during this process, the tests will not be shown in the explorer. I had two issues preventing the test DLL to be successfully loaded:
There was still a binding redirect left in the config file (redirecting to a version lower version NHiberate than what was referenced in the test project).
A conflicting assembly reference (2nd level references not being able to load). AsmSpy is btw a great tool to hunt for these.
If you load a Visual Studio (VS 2015 Community in my case) solution from a network share or My Documents directory that is part of a share, you will get into this trouble. I solved it by moving the solution and its underlying projects to a local folder.
After spending 2 days... none of the above worked for me. The only "solution" was: Go to project properties -> Build Tab. Then click Advanced button on the right bottom corner of the pane. Change "Debug Info:" to "full" and click OK.
Here are the screen shots:
I came across the same issue.
And investigated and found that the dll's were not build,
and put in the right folder. as soon as I changed my configuration
they appeared.
- the Projects build options, what folder should be used?
- the build menu entry build configuration, they should be checked.
that fixed it for me.
For the Visual Studio 2013.5, clearing the \TestResults directory in the solution helped. Visual Studio corrupted the mdf file in which it stores the discovered tests, thus preventing the discovery of unit tests.
Be sure your all projects are runing with the same configuration.
Under your project's Properties => Debug => Platform in drop down list choose the appropriate platform (for me it was "Any CPU") as determined at your other projects.
I know that the unit tests are not found if the solution is not built, so that is something to try (Build the solution), but that solution is like the help desk asking if your computer is plugged in...
After a clean rebuild did not fix the problem for me, running a full batch build did fix it.
Had the same issue; tests suddenly stopped being discovered.
Nunit Test Adapter had become disabled somehow. Clicking enable in the extension manager fixed it for me.
I had the same problem until I realized I made a cut/paste error and left off [Test Method] before the test.