DotTrace can be used to profile tests on TeamCity. But this example works with NUnit, while my tests are written in Visual Studio Tests. When running my tests I use the build-in test engine VSTest, no such option is available for the DotTrace profiler plugin. Instead I need to specify the path for the test engine executable.
I did find an executable VSTest.exe but it's path includes "Microsoft SDKs\Windows Phone\v8.0\ExtensionSDKs", so unsurprisingly it doesn't work. Saying it can't find a start.bat file, that does indeed not exist in that .exe file.
What is the executable path that TeamCity uses for VSTest?
Then I can configure the performance test to use that test engine. I know the tests can be executed using DotTrace, having run it in Visual Studio, just don't want to install Visual Studio and the DotTrace plugin there when there's a TeamCity plugin.
According to this, you can install Visual Studio Test Agent on your TeamCity Agent and then specify vstest.console.exe as a profiling target in the XML config.
Nevertheless, I suggest you to try a new dotTrace plugin version (if it asks you for credentials, specify guest with no password). It's not yet officially announced but fully functional.
The main difference is that the plugin is now integrated into 'unit testing' build steps, e.g. Visual Stiduo Tests, NUnit, MSpec, MSTest. So, it doesn't require a configuration file. Thus, after you install the plugin, all unit testing steps will get the additional option Run build step under dotTrace profiler. All you need to do is add the Visual Studio Tests step, check this option there and specify performance thresholds as usual.
I have a simple solution in visual studio 2013 that is composed by one web project, one library project and one unit test project. When I open the solution and try to run the unit tests they are not discover by visual studio. To run the tests I try to go to the menu and choose Test -> Run -> Run all tests or by opening the test explorer window. By those to methods visual studio doesn’t discover any tests in the solution.
Creating first a simple unit tests project and try to run the test, visual studio know discover the test and I can run it. Then, if I open my previous solution visual studio now discovers all the tests. I try to save my solution but closing it and reopening, without creating a unit test project first, the visual studio doesn’t find the tests again. This is a very strange behave that I don’t know why this is happening.
I used to working alone in this project that was using the source control git integrated with the visual studio team foundation. The problem of visual studio not discover the unit tests start when a new element came to the project and when I need to recreate the solution through the source control online. Before this, all tests always been discovered by visual studio.
For creation the unit tests I use the dll Microsoft.VisualStudio.QualityTools.UnitTestFramework. My version of visual studio is: Microsoft Visual Studio Express 2013 for Web Version 12.0.30723.00 Update 3. My version of .net framework is 4.5.50938.
All of my tests are like this:
[TestClass]
public class Service1Test
{
[TestMethod]
public void Test1()
{
Assert.IsTrue(True);
}
}
Some things I've noticed I have to do from time to time to get tests to show up properly.
If your solution is in a protected drive that you need administrator access to read/write, sometimes only a portion of the tests come up. Definitely run VS as administrator in that case.
If your solution is 64 bit, make sure that Test > Test Settings > Default Processor Architecture is set to x64. Sometimes it gets set to x86. Set it to x64, then rebuild.
Sometimes just restarting Visual Studio does the trick because the test explorer will start up again.
Don't forget to actually build the test project/solution. (If you want it to get built with the rest of the projects, right-click on your solution > Properties > Configuration Properties > Configuration > check the "Build" box for your test project)
Ensure the tests are in a public section of your test class
If you using NUnit, make sure to download NUnit Adapter first.
Go to Tools → Extensions and Updates… → Online → search for "NUnit Test Adapter".
Make sure your test class is public so it can be found. And if you're referencing another class, make sure of the same.
Also, sometimes if you have no Asserts or you're not decorating the test with a [TestMethod], a test might not be recognized.
2 more things:
1) Async unit tests act funny at best, and none at all at worst. Have a look at this article by Stephen Cleary and keep from there if it interests you.
2) If you use NUnit and you run into the same issues, keep in mind it's [TestCase] for Nunit, instead of [TestMethod]
Having said the above, here's an article I've posted on the code project, with both MSTest & NUnit, in case you want to give it a spin and make sure you're not missing anything.
I had the same issue but none of the other solutions worked. Turns out that I was using the NUnit 3 framework with the 2 adapter.
If you're using NUnit 3, go to Extensions and Updates and install the NUnit3 Test Adapter.
I'm having this issue from time to time. What works for me is to shutdown Visual Studio and go to folder:
%LocalAppData%\Microsoft\VisualStudio\12.0\ComponentModelCache
and delete it content.
Once you open Visual Studio and load your project again Test Explorer should contain back your tests
XUnit users may notice Test Explorer window no longer lists any tests. To make tests discoverable again try this important tip, highlighted below.
If you're having problems discovering or running tests, you may be a victim of a corrupted runner cache inside Visual Studio. To clear this cache, shut down all instances of Visual Studio, then delete the folder %TEMP%\VisualStudioTestExplorerExtensions. Also make sure your project is only linked against a single version of the Visual Studio runner NuGet package (xunit.runner.visualstudio).
For future googlers I had a rare scenario that caused this.
On my base test class I had a property named TestContext. This interfered with MSTest's reserved TestContext property causing all my tests to be hidden from VS/Resharper except one (which did not inherit from the base).
for me it was changing 'solution configurations' to Debug (instead of Release).
My problem was because my unit test method wasn't void, and it was receiving parameters.
I have found that unit test methods marked as async void aren't discovered by the VS Test Explorer. This seems to be because VS would not have any way to wait for a test to finish and decide if it succeeded or not. If you absolutely need to have a test method to run asynchronously then get it to return a Task instead like async Task.
I found that this fixed the issue for me.
Try building all projects as MSIL (Any CPU) instead of x86/x64. Worked for me strangely
While AndyG's solution works, a more lasting solution could be to set the PreferredToolArchitecture environment variable to "x64", either by:
How to make Visual Studio use the native amd64 toolchain
or by:
Control Panel | System and Security | System | Advanced System Settings|Environment Variables
PreferredToolArchitecture = x64
DefaultToolArchitecture = Native64Bit
PROCESSOR_ARCHITECTURE = x64
ProcessorArchitecture = x64
I was facing the same problem and I've remembered, again (this situation happened before), that selecting "Mixed Platform" on the solutions platform menu works, as well as the other answers.
I'd managed to add mine as
public static void TestMethod1(){}
started working once I removed static....
Go to Nuget package manager and download Nunit Adapter as follow.
go to the project menu > Configuration Manager check your test project platform matches the rest of the project and is check to build then rebuild.
Just ran into this as well as I didn't see a similar case that was similar to mine.
In the .csproj file of my test project, NUnit reference privacy was set to False:
<Reference Include="nunit.framework, Version=2.6.4.14350, Culture=neutral, PublicKeyToken=96d09a1eb7f44a77, processorArchitecture=MSIL">
<HintPath>..\packages\NUnit.2.6.4\lib\nunit.framework.dll</HintPath>
<Private>False</Private>
</Reference>
After I set <Private> to True it worked.
You just need to install this package only:
NUnit TestAdapter
To get tests to show in the Test Explorer Window I had to install NUnit3 Test Adapter 3.0 which was not available in Package Manager.
Downloaded from https://visualstudiogallery.msdn.microsoft.com/0da0f6bd-9bb6-4ae3-87a8-537788622f2d
I had the exact same problem.
It was due to incompatible version of NUnit I'd added to my project (3.2.0) and the Test Adapter I had installed (2.0.0).
To fix, use "Tools > Extensions and Updates" and search for NUnit3 Test Adapter, it discovered my tests after that.
Cheers
Let's just say for argument's sake that you need to use X64 architecture on your test project in order for the dependencies to build properly (as in my case). You may need to modify your Default Processor Architecture under the Test--Test Settings menu. Setting this to X64 allowed my test explorer to find my tests using Microsoft.VisualStudio.TestTools.UnitTesting.
Sorry for adding to the long list, but I had a completely a different problem.
First, I would like to mention I discovered my issue when clicking 'Run All' in the Test Explorer and then watching the build output window in Visual Studio. You have to actively watch it, as afterwards the message disappears.
As for the issue, it looks like during the scanning of the tests, the DLL gets loaded and its test types are enumerated. This causes the references to be loaded and if any failure occurs during this process, the tests will not be shown in the explorer. I had two issues preventing the test DLL to be successfully loaded:
There was still a binding redirect left in the config file (redirecting to a version lower version NHiberate than what was referenced in the test project).
A conflicting assembly reference (2nd level references not being able to load). AsmSpy is btw a great tool to hunt for these.
If you load a Visual Studio (VS 2015 Community in my case) solution from a network share or My Documents directory that is part of a share, you will get into this trouble. I solved it by moving the solution and its underlying projects to a local folder.
After spending 2 days... none of the above worked for me. The only "solution" was: Go to project properties -> Build Tab. Then click Advanced button on the right bottom corner of the pane. Change "Debug Info:" to "full" and click OK.
Here are the screen shots:
I came across the same issue.
And investigated and found that the dll's were not build,
and put in the right folder. as soon as I changed my configuration
they appeared.
- the Projects build options, what folder should be used?
- the build menu entry build configuration, they should be checked.
that fixed it for me.
For the Visual Studio 2013.5, clearing the \TestResults directory in the solution helped. Visual Studio corrupted the mdf file in which it stores the discovered tests, thus preventing the discovery of unit tests.
Be sure your all projects are runing with the same configuration.
Under your project's Properties => Debug => Platform in drop down list choose the appropriate platform (for me it was "Any CPU") as determined at your other projects.
I know that the unit tests are not found if the solution is not built, so that is something to try (Build the solution), but that solution is like the help desk asking if your computer is plugged in...
After a clean rebuild did not fix the problem for me, running a full batch build did fix it.
Had the same issue; tests suddenly stopped being discovered.
Nunit Test Adapter had become disabled somehow. Clicking enable in the extension manager fixed it for me.
I had the same problem until I realized I made a cut/paste error and left off [Test Method] before the test.
I've got integration tests which I launch from Visual Studio 2008 Development Edition (they are executed as MSTest unit-tests, even though they are actually integration tests). I would like to do some profiling while the tests are being carried out.
In my case, I have an MSTest "ordered" test I'm running. I'd like to collect sampling data while each of the ordered tests is executed.
How do I do this?
p.s. Perhaps I can accomplish this from the command line, if I manually launch mstest from a command prompt, while trying to follow command line directions. Unfortunately, when I try to launch VsPerfCLREnv from the Visual Studio command prompt, it is "file not found." What the heck?
I just got profiling working in VS2008. In case it helps anyone, here is how I got it to work:
Ensure you are in Debug configuration mode
Click Analyse > Launch Performance Wizard
Select Profile a dynamic link library
Enter the following:
Dll path: The dll you want to instrument, located in the bin\Debug of your unit test project
Executable path: C:\Program Files (x86)\Microsoft Visual Studio 9.0\Common7\IDE\MSTest.exe
Start Option Arguments: /testmetadata:TestList.vsmdi /runconfig:TestRunConfig1.testrunconfig
Adjust this based on the name of your vsmdi and testrunconfig files.
Working directory: Your solution root path
Choose Instrumentatation. I haven't tried Sampling, so I cannot comment on that.
Run the new performance project
Using this approach I identified the bottleneck in my code, and the unit tests are now 15x faster, and the users are seeing a 10x speed improvement.
Unfortunately Ordered Tests don't have the 'Create Performance Session' option available:
If you had a simple unit test you would be able to create a performance session.
To use command-line profiling you can navigate to \Program Files\Microsoft Visual Studio 9.0\Team Tools\Performance Tools where you will see vsperfclrenv.cmd and other tools mentioned on the profiler blog.
I have Gallio/MbUnit installed and am using VS 2010 RC and I want to be able to run a single unit test or just all unit tests inside of a TestFixture and not all the tests in the entire project everytime I debug. How do you do this in VS 2010?
Install testdriven.net, there is a free personal version
http://www.testdriven.net/
As Simon said, TD.Net is just working well.
If you work with VS2010 Premium or Ultimate, you may also use the built-in test run feature of VS2010. It should find and run your MbUnit tests as well (if the Gallio add-in was installed)
You may also run your tests by using Resharper 5, but it seems that the latest beta of Resharper 5 has broken the Gallio test driver. It should be fixed soon however.
Apart from the VS2010 IDE, you may also run your tests with Echo (a powerful console application located in the bin folder of the Gallio installation directory), Icarus (Winform application with a nice UI; same location), or from PowerShell (by using the Run-Gallio snap-in). You can also invoke the test runner from one of the numerous extensions of Gallio for MSBuild, NAnt, TeamCity, etc.
I didn't see it earlier but VS 2010 has a Test List Editor which allows me to add the specific test to a a list and run it from there so that only the one test is run. I know that TD.Net probably has a right click/run test option for a test method but I didn't want to buy it and install yet another tool.
Another alternative, how we do it is select the specification inside visual studio, press F5, and it runs the specification via a Rake task.
Example:
class DiffuserObserverSpecification
: AutoMockSpecification<DiffuserObserver, IDiffuserObserver>
class When_diffuser_observer_is_created : DiffuserObserverSpecification
class When_diffuser_observer_has_injected_diffuser_change
: DiffuserObserverSpecification
So to execute the behaviour expectations of all DiffuserObserverSpecification, we select the DiffuserObserverSpecification and press F5.
And to execute just the behaviour for When_diffuser_observer_is_created, select that and press F5.
After one month you forget what I debugger looks like.