Develop Reference

Some Unit Tests are not running in ADO Pipeline (tests from other test projects successfully run). Dotnet test using moq

I have a solution, it has 4 projects, each project has a corresponding test project in a "tests" solution
I've noticed that sometimes a test fails locally but passes the CI build, and depending where the test is located, sometimes it does successfully fail on CI build.
Our ADO pipeline steps looks like:
Here is a csproj from a test project that is running successfully:
Here is a csproj from a test project that isnt running:
Any ideas? I have narrowed down some test classes that I know are running, but I cant tell what is different about those class' csproj versus the ones that are not running.

Turns out the test project that was not running was created outside of other projects:
when i manually move it in finder, and try to replace the reference paths in the csproj to reflect how the paths in the functioning test projects are laid out, the project refuses to load.
anyone know how to easily and safely refactor/move this project??
the best solution here is to create a new project, name it similarly (or name it whatever) then delete the old one once its running properly, and finally rename the new one to the old name

VS Tests run locally but aren't found on the build server

I've got a project that I'm trying to create a build definition for in VSTS. The goal is to build the project, run all the unit tests, then publish if they pass. Tests are via Microsoft.VisualStudio.TestTools.UnitTesting, and locally, everything works fine - VS 2015 discovers and runs all the tests without a hitch.
However, something's going wrong in the build definition. Steps:
Run dotnet restore.
Build Solution **\*.sln.
Run dotnet publish.
Test Assemblies **\$(BuildConfiguration)\**\*Test*.dll.
Remaining publish logic.
Everything seems to run fine on the VSTS build, until step 4:
Warning: No test is available in C:\agent\_work\19\s\test\Foo.Test\bin\Debug\netcoreapp1.0\Foo.Test.dll. Make sure that installed test discoverers & executors, platform & framework version settings are appropriate and try again.
Test files look something like this:
namespace Foo.Test
{
[TestClass]
public class ServiceTests
{
public ServiceTests()
{
}
[TestMethod]
public void RequestSuccessfullyAddsFilter()
{
...
So, what could be stopping the build definition from finding the tests? It's finding the dll, I don't have any unusual parameters set in the test assemblies step. This seems like a pretty cut-and-dry setup, but there must be some step I'm overlooking in here.

Got a solution from one of my teammates - it's a .NET Core project, so switching the test task over to the dotnet test task resolved the issue.

NUnit failed to load DLL

I am getting the following error message when trying to run unit tests in Visual Studio:
NUnit failed to load w:\Repos\trading.tools\Trading.Tools.Test\bin\x64\Debug\Trading.Tools.Test.dll
I am using
Visual Studio Community 2013
NUnit Adapter 3.4.0.0
NUnit 3.4.1
The weird thing is, that I have another project which is set up the same way as this one and it works just fine.
I also downloaded NUnit 3.4.1 and installed it. When I run
nunit3-console.exe Trading.Tools.Test.dll
everything works just fine.
Any ideas what I can do?
Many thanks
Konstantin
Edit #1
Here is the full console output from Visual Studio when trying to run all test.
Test run will use DLL(s) built for framework Framework45 and platform X86. Following DLL(s) will not be part of run:
Trading.Tools.Test.dll, Trading.Tools.dll are built for Framework Framework45 and Platform X64.
Go to http://go.microsoft.com/fwlink/?LinkID=236877&clcid=0x409 for more details on managing these settings.
NUnit Adapter 3.4.0.0: Test discovery starting
NUnit failed to load w:\Repos\trading.tools\Trading.Tools.Test\bin\x64\Debug\Trading.Tools.Test.dll
Assembly contains no NUnit 3.0 tests: w:\Repos\trading.tools\Trading.Tools\bin\x64\Debug\Trading.Tools.dll
NUnit Adapter 3.4.0.0: Test discovery complete
As you can see it is very obvious that NUnit expects a x86 build, but I build for a x64 platform. And again, my x64 build works just fine if I execute it using nunit3-console.exe.
What I see in the csproj file is this:
<Reference Include="nunit.framework, Version=2.6.4.14350, Culture=neutral, PublicKeyToken=96d09a1eb7f44a77, processorArchitecture=MSIL">
<SpecificVersion>False</SpecificVersion>
<HintPath>..\packages\NUnit.3.4.1\lib\net45\nunit.framework.dll</HintPath>
</Reference>
The weird thing here is that it specifies using Version=2.6.4.14350 but referencing a 3.4.1 dll.
So the next question from this point is how can I make NUnit execute my x64 build? Any ideas?

I had a similar issue, the key is the fact that it is the Test Runner in Visual Studio that is stating that only x86 assemblies will be tested. I am assuming from this that it then forces the use of the x86 NUnit runner. To change this (in VS2015 and VS2017 at least), go to Test > Test Settings > Default Processor Architecture > X64.

You can also set the execution target in the runsettings file. You then have to select that file. This should make the solution more stable.
A runsettings file which only set this can look like:
To enable it, do as shown in the figure below:
When you select it from the test menu (1), it will be added as the selected one in the menu (2), and a Rebuild will then make the test appear in the Test Explorer (3)
There is an extra bonus by using a runsettings file, and that is that it will then run properly on the TFS Build system, if you use that. I have written a blog post on that issue, see http://hermit.no/how-to-control-the-selection-of-test-runner-in-tfsvsts-making-it-work-with-x86x64-selected-targets/

I couldn't execute my tests and found that to be one of the issues. It turns out that my TestFixture was internal. Just switching it to public solved my case.

After unsuccessfully trying all other responses above, the following worked for me:
In my case the .NET project and solution is on a mounted drive (I use a MacBook and Parallels for .NET development). The mount also contains the /bin/debug and /bin/release locations where NUnit was attempting to read the "test" DLL from.
The fix was to move the solution/project files to the C: drive of my Windows image. The tests were discovered immediately.
Apparently the shared/mounted location was not to its liking. I don't know why, since the mount is permanent and read/writable to all users on the Windows image. I suspect file permissions problems or maybe somehow the entire mount is not accessible to the user/process running the NUnit discovery logic.

I happened to get this error when writing the unit test method. And noticed the root cause that one of the dependent dll was missing to load. This error("NUnit failed to load the .dll") was shown in the Output("Test") window after modifying the test method code and trying to run it. After updating the nuget package for the dependent dll, nunit started picking the test project dll and test cases got executed.

Today I was running into this issue as well (on a .NET Framework 4.8 based NUnit project). The solution for me was to also install the Microsoft.NET.Test.Sdk package.

To get to the root cause, it may help to attempt to run your tests using the NUnit CLI.
In my case, the CLI logged a bind failure I didn't see in Visual Studio. After I had fixed that, my tests ran correctly via CLI and VS.

I got this error in a .Net 6.0 Asp.Net solution and here's how I solved it.
It was only occurring in one Test project whose tests wouldn't run, the other Test projects were running fine in Test Explorer and in Debug.
The tests that were failing to be detected had "Test Not Run":
In the Output is the error:
NUnit Adapter 4.3.1.0: Test discovery starting NUnit failed to load [dll path]
No test is available in [dll path]
What I did was comment out each class and bring them back one by one until the Tests would STOP to RUN. Then with the failing class put a break point on a [Test] method.
If you can't hit the break point then its failing in this classes [SetUp]. Put a BreakPoint in the [SetUp] and use F11 to step off the edge of the method, ie F11 off the final curly brace..
AND THEN you get a prompt saying which DLL it can't load.
In my case it was “couldn’t load a DLL Microsoft.AspNetCore.Mvc.Core”
Referencing the DLL via Package Manager resolved the problem.
Edit: if this happens in a Unit Test you may want to reference the Microsoft.AspNetCore.App FrameworkReference rather than the Microsoft.AspNetCore.Mvc.Core package reference:
<ItemGroup>
<FrameworkReference Include="Microsoft.AspNetCore.App" />
</ItemGroup>

Tests succeed when run from Test View but fail when run from test list editor or command line

I have a test suite, comprised of both unit tests and integration tests, in a project using C# on .NET 4.0 with Visual Studio 2010. The test suite uses MSTest. When I run all tests in solution (either by hitting the button in the testing toolbar or using the Ctrl-R A shortcut chord) all of the tests, integration and unit, pass successfully.
When I either attempt to run the same tests from the command line with mstest (explicitly using the only .testsettings file present) or attempt to run them from the Test List Editor or using the .vsmdi file the integration tests fail.
The integration tests test the UI and so have dependencies on deployment items and such, whereas the unit tests do not. However, I cannot seem to pin down what is actually different between these two methods of running the tests.
When I inspect the appropriate Out directories from the test run, not all of the files are present.
What would cause some of the files that deploy correctly in one situation from Visual Studio to not deploy correctly in another?

The static content started being copied shortly after I wrote the comments above. The other major issue I ran into was that the integration test project referenced libraries that were dependencies of the system-under-test (with copy-local set to true) in order to ensure that the DLLs would be present when they were needed. For some reason, these stubbornly refused to copy when the tests were run through Test List or mstest.
What I eventually did to work around it was include [DeploymentItem] attributes for the DLLs that I needed. This got things working no matter how the tests were run. What I am still unclear on, that may have answered the underlying solution, or provided a better solution, is how Test View/mstest differ from the regular test runner (assuming that the correct .settings file was passed to mstest.). I'm putting these notes/workarounds in an answer, but I'll leave the question open in case anyone can address the underlying cause for how the different test execution paths differ.

How to Automate Silverlight Unit Tests in Hudson?

I would like to run automated Silverlight unit tests from a Hudson build server. It seems there are two options:
Use Statlight, although it seems to be designed for TeamCity rather than Hudson, so it would involve a bit of hacking to get it to work.
Use NUnit Silverlight tests.
Can anyone recomend either of these options? Or is there a better alternative?

You can try using Lighthouse Silverlight Unit Test Runner, it works with every Build Server including Hudson, TeamCity and CCNet because it by default produces NUnit compatible xml results file:
http://lighthouse.codeplex.com/

In our company we are using NUnit with Hudson for automatized unit testing. It is simple to setup and execute.
Just download and unzip latest nunit somewhere on Hudson host.
Add Windows batch command as last buildstep with content like:
C:\NUnit\bin\net-2.0\nunit-console.exe "%WORKSPACE%\src\Test\AllTests.nunit" /config=Release /xml="%WORKSPACE%\src\Test\TestResults.xml"
This will execute tests as defined in "AllTests.nunit" file. It is possible tu point just to one assembly (.dll).
To populate test results within Hudson Job page, you would need to install Hudson NUnit plugin. Its possible directly from Hudson plugin management.
After instalation there will be new Post build action: Publish NUnit test result report.
If you check it, you've got line to enter path to test result report. Corresponding path for example above is:
src/Test/TestResults.xml
Hope it helps you to decide ;-)