I've got a project that I'm trying to create a build definition for in VSTS. The goal is to build the project, run all the unit tests, then publish if they pass. Tests are via Microsoft.VisualStudio.TestTools.UnitTesting, and locally, everything works fine - VS 2015 discovers and runs all the tests without a hitch.
However, something's going wrong in the build definition. Steps:
Run dotnet restore.
Build Solution **\*.sln.
Run dotnet publish.
Test Assemblies **\$(BuildConfiguration)\**\*Test*.dll.
Remaining publish logic.
Everything seems to run fine on the VSTS build, until step 4:
Warning: No test is available in C:\agent\_work\19\s\test\Foo.Test\bin\Debug\netcoreapp1.0\Foo.Test.dll. Make sure that installed test discoverers & executors, platform & framework version settings are appropriate and try again.
Test files look something like this:
namespace Foo.Test
{
[TestClass]
public class ServiceTests
{
public ServiceTests()
{
}
[TestMethod]
public void RequestSuccessfullyAddsFilter()
{
...
So, what could be stopping the build definition from finding the tests? It's finding the dll, I don't have any unusual parameters set in the test assemblies step. This seems like a pretty cut-and-dry setup, but there must be some step I'm overlooking in here.
Got a solution from one of my teammates - it's a .NET Core project, so switching the test task over to the dotnet test task resolved the issue.
Related
I have a solution, it has 4 projects, each project has a corresponding test project in a "tests" solution
I've noticed that sometimes a test fails locally but passes the CI build, and depending where the test is located, sometimes it does successfully fail on CI build.
Our ADO pipeline steps looks like:
Here is a csproj from a test project that is running successfully:
Here is a csproj from a test project that isnt running:
Any ideas? I have narrowed down some test classes that I know are running, but I cant tell what is different about those class' csproj versus the ones that are not running.
Turns out the test project that was not running was created outside of other projects:
when i manually move it in finder, and try to replace the reference paths in the csproj to reflect how the paths in the functioning test projects are laid out, the project refuses to load.
anyone know how to easily and safely refactor/move this project??
the best solution here is to create a new project, name it similarly (or name it whatever) then delete the old one once its running properly, and finally rename the new one to the old name
I have a solution written on c#, .netcore 2.2 and the testing framework is xunit that looks pretty much like this:
-src
--controllers
--services
-test
--controllers.integrationtest
--services.integratiotest
we are performing some integration test with two distinct dlls (controllers.integrationtest.dll & services.integrationtest) that use the same database (solutionname.test)
now, when I run "dotnet test" on the solution, it seems that the two dlls are trying to access the same resource (database) and one dll gets a deadlock.
The thing is that I want to disable the parallel execution of the dlls when I run "dotnet test" to avoid that deadlock, so I search that up and the documentation says that to disable parallel execution of dlls you have to:
Add a xunit.runner.json in the root of the csproject which I did and works fine. (this is working on visual studio because I tested it with another feature that deletes the "_" character of test names: [methodDisplay])
Configure xunit.runner.json file to copy always or preserve newest in visual studio so that gets copied on bin folder (as any appsetings.json file)
I've read that you can place an assembly attribute in the assemblyInfo.cs file that by the way it seems that was replaced by the plain csproj so I'm a bit confused.
The ultimate goal that I want to achieve is that when devops use dotnet test the integration test doesn't blow up on concurrency problems.
In xUnit the default value for ParallelizeAssemblies is false so I can only assume that a behavior of dotnet test is causing both to be executed together.
Try and run the tests using dotnet vstest. This requires the DLLS to have been built.
dotnet vstest **/*test.dll
This is a common question, it's really quite astounding to me how difficult such a simple task is turning out to be ...
I have a Visual Studio (formerly Xamarin) C# cross-platform library. It has some code in the shared "abstract" library, and a few machine-specific implementation classes for Android, iOS, and UWP libraries. It also has a few automated (non-UI) unit tests that use the Microsoft.VisualStudio.TestTools.UnitTesting package. This is a great start, when Visual Studio 2017 is feeling generous it runs my unit tests and life is good.
Now I want to run the unit tests on simulators of the actual devices, from a command line (so I can run them from Jenkins and capture the output). So, for example, I want to run my unit tests on a simulator of an iPad. From various web sites I get the feeling that I can't use the Microsoft package for this, so I need to switch to either Xunit or NUnit. I gave up on Xunit earlier today, here's how NUnit went.
Go into the Unit Test project and remove the NuGet packages for Microsoft.NET.Test.Sdk, MSTest.TestFramework, and MSTest.TestAdapter.
Add NUnit instead (version 3.7.0).
Go into each CS file, change to "using NUnit.Framework", change [TestMethod] to [Test], and [TestClass] to [TestFixture].
Project:Manage NuGet Packages, add the NUnit3TestAdapter.
Test Explorer: Run All
At this point I get this weird error for the UWP machine-specific library:
The "XamlCTask" task could not be initialized with its input parameters
The "DebugType" parameter is not supported by the "XamlCTask" task. Verify the parameter exists on the task, and it is a settable public instance property.
That's a really strange message, and if I Rebuild the UWP library, it rebuilds without errors. Not sure why it's building the UWP library as part of the Unit Tests, the only thing listed under "Projects" is the abstract (non-machine-specific) library.
Also strange that building the Unit Test library works, it's only when I do the Test Explorer: Run All that I get these errors.
OK fine, it's a known error.
Close VS2017, delete bin and obj folders, reopen, no better. Close VS2017, hack the Xamarin.form.targets as listed. Same error. Realize error is in the global Xamarin.Forms.targets, not the one in my solution. Hack it. Close & reopen solution, rebuild. No error this time!
Test Explorer: Run all. Nothing. Output:Build shows no failures, Output:Tests is completely empty. Close & reopen solution. Get new errors:
[Failure] Could not find file 'C:\Workspace\PopComm.iOS\obj\Debug\TemporaryGeneratedFile_5937a670-0e60-4077-877b-f7221da3dda1.cs'.
[Failure] Could not find file 'C:\Workspace\PopComm.iOS\obj\Debug\TemporaryGeneratedFile_E7A71F73-0F8D-4B9B-B56E-8E70B10BC5D3.cs'.
[Failure] Could not find file 'C:\Workspace\PopComm.iOS\obj\Debug\TemporaryGeneratedFile_036C0B5B-1481-4323-8D20-8F5ADCB23D92.cs'.
Test Explorer: Run all. This time get more interesting errors:
------ Discover test started ------
NUnit Adapter 3.7.0.0: Test discovery starting
Assembly contains no NUnit 3.0 tests:
C:\Workspace\PopComm.iOS\bin\iPhone\Debug\PopComm.dll
NUnit Adapter 3.7.0.0: Test discovery complete
========== Discover test finished: 0 found (0:00:00.0530024) ==========
Of course that assembly doesn't contain any unit tests! It's not my unit test project!
Sigh.
OK, create a new project of type "NUnit 3 Unit Test Project". Move my test classes into this new project and add a dependency to my non-machine-specific library.
Test Explorer: Run all - Tests! There are tests! And they even run successfully.
Close VS2017, open again, Test Explorer: Run all. No tests. Output:Tests is empty.
Close project, delete all bin and obj folders, Rebuild Solution, Test Explorer: Run All. No tests.
Push code to git, pull on the Mac.
Open solution in Visual Studio for the Mac.
Use the mostly-hidden View:Test feature to bring up the Unit Tests window, run the tests. Tests found and run. Yippee!
Now I want to run my unit tests on an iPad simulator, not inside VSMac.
Add a new project to my solution, type iOS:Tests:Unified API:Unit Test App.
Wizard asks what is the minimum iOS version I want to support. I figure I should match the iOS version that my machine-specific library targets, so I go looking in the VSMac options for the project. The iOS target version isn't shown anywhere. Use the default (10.3) for the Unit Test App.
Do I have to copy all of my unit test CS files into this new app now? Or can I somehow reference my cross-platform unit test project in this Unit Test App?
Pushing everything and pulling on the Windows machine, I try adding a reference to my unit test library but get an error:
The project 'NUnit.Tests' cannot be referenced. The referenced project is targeted to a different framework family (.NETFramwork)
Hum. I wonder what framework the new NUnit.Tests.iOS application is targeting. Check its options - it doesn't say anywhere. Remove the reference from the Unit Test App back to the unit test files, and just copy the unit test source files over (non-optimal).
Run the application in the iPad simulator, it finds & runs the tests, but I don't know how to (1) run it from the command line, and (2) capture the unit test statuses in a file.
So after all this, I still don't know how to run my unit tests (1) from the command line, (2) on an iPad simulator, and (3) capturing the output in a text file.
I am getting the following error message when trying to run unit tests in Visual Studio:
NUnit failed to load w:\Repos\trading.tools\Trading.Tools.Test\bin\x64\Debug\Trading.Tools.Test.dll
I am using
Visual Studio Community 2013
NUnit Adapter 3.4.0.0
NUnit 3.4.1
The weird thing is, that I have another project which is set up the same way as this one and it works just fine.
I also downloaded NUnit 3.4.1 and installed it. When I run
nunit3-console.exe Trading.Tools.Test.dll
everything works just fine.
Any ideas what I can do?
Many thanks
Konstantin
Edit #1
Here is the full console output from Visual Studio when trying to run all test.
Test run will use DLL(s) built for framework Framework45 and platform X86. Following DLL(s) will not be part of run:
Trading.Tools.Test.dll, Trading.Tools.dll are built for Framework Framework45 and Platform X64.
Go to http://go.microsoft.com/fwlink/?LinkID=236877&clcid=0x409 for more details on managing these settings.
NUnit Adapter 3.4.0.0: Test discovery starting
NUnit failed to load w:\Repos\trading.tools\Trading.Tools.Test\bin\x64\Debug\Trading.Tools.Test.dll
Assembly contains no NUnit 3.0 tests: w:\Repos\trading.tools\Trading.Tools\bin\x64\Debug\Trading.Tools.dll
NUnit Adapter 3.4.0.0: Test discovery complete
As you can see it is very obvious that NUnit expects a x86 build, but I build for a x64 platform. And again, my x64 build works just fine if I execute it using nunit3-console.exe.
What I see in the csproj file is this:
<Reference Include="nunit.framework, Version=2.6.4.14350, Culture=neutral, PublicKeyToken=96d09a1eb7f44a77, processorArchitecture=MSIL">
<SpecificVersion>False</SpecificVersion>
<HintPath>..\packages\NUnit.3.4.1\lib\net45\nunit.framework.dll</HintPath>
</Reference>
The weird thing here is that it specifies using Version=2.6.4.14350 but referencing a 3.4.1 dll.
So the next question from this point is how can I make NUnit execute my x64 build? Any ideas?
I had a similar issue, the key is the fact that it is the Test Runner in Visual Studio that is stating that only x86 assemblies will be tested. I am assuming from this that it then forces the use of the x86 NUnit runner. To change this (in VS2015 and VS2017 at least), go to Test > Test Settings > Default Processor Architecture > X64.
You can also set the execution target in the runsettings file. You then have to select that file. This should make the solution more stable.
A runsettings file which only set this can look like:
To enable it, do as shown in the figure below:
When you select it from the test menu (1), it will be added as the selected one in the menu (2), and a Rebuild will then make the test appear in the Test Explorer (3)
There is an extra bonus by using a runsettings file, and that is that it will then run properly on the TFS Build system, if you use that. I have written a blog post on that issue, see http://hermit.no/how-to-control-the-selection-of-test-runner-in-tfsvsts-making-it-work-with-x86x64-selected-targets/
I couldn't execute my tests and found that to be one of the issues. It turns out that my TestFixture was internal. Just switching it to public solved my case.
After unsuccessfully trying all other responses above, the following worked for me:
In my case the .NET project and solution is on a mounted drive (I use a MacBook and Parallels for .NET development). The mount also contains the /bin/debug and /bin/release locations where NUnit was attempting to read the "test" DLL from.
The fix was to move the solution/project files to the C: drive of my Windows image. The tests were discovered immediately.
Apparently the shared/mounted location was not to its liking. I don't know why, since the mount is permanent and read/writable to all users on the Windows image. I suspect file permissions problems or maybe somehow the entire mount is not accessible to the user/process running the NUnit discovery logic.
I happened to get this error when writing the unit test method. And noticed the root cause that one of the dependent dll was missing to load. This error("NUnit failed to load the .dll") was shown in the Output("Test") window after modifying the test method code and trying to run it. After updating the nuget package for the dependent dll, nunit started picking the test project dll and test cases got executed.
Today I was running into this issue as well (on a .NET Framework 4.8 based NUnit project). The solution for me was to also install the Microsoft.NET.Test.Sdk package.
To get to the root cause, it may help to attempt to run your tests using the NUnit CLI.
In my case, the CLI logged a bind failure I didn't see in Visual Studio. After I had fixed that, my tests ran correctly via CLI and VS.
I got this error in a .Net 6.0 Asp.Net solution and here's how I solved it.
It was only occurring in one Test project whose tests wouldn't run, the other Test projects were running fine in Test Explorer and in Debug.
The tests that were failing to be detected had "Test Not Run":
In the Output is the error:
NUnit Adapter 4.3.1.0: Test discovery starting NUnit failed to load [dll path]
No test is available in [dll path]
What I did was comment out each class and bring them back one by one until the Tests would STOP to RUN. Then with the failing class put a break point on a [Test] method.
If you can't hit the break point then its failing in this classes [SetUp]. Put a BreakPoint in the [SetUp] and use F11 to step off the edge of the method, ie F11 off the final curly brace..
AND THEN you get a prompt saying which DLL it can't load.
In my case it was “couldn’t load a DLL Microsoft.AspNetCore.Mvc.Core”
Referencing the DLL via Package Manager resolved the problem.
Edit: if this happens in a Unit Test you may want to reference the Microsoft.AspNetCore.App FrameworkReference rather than the Microsoft.AspNetCore.Mvc.Core package reference:
<ItemGroup>
<FrameworkReference Include="Microsoft.AspNetCore.App" />
</ItemGroup>
Using TFS 2010 and vs 2010:
When kicking off a build on the build machine, Unit Tests failing are with a System.IO.FileNotFoundException when trying to do a System.Reflection.Assembly.Load.
If i run these tests locally they work fine. Also, if i remote into the build machine and open the solution manually the unit tests will work fine.
The assembly that is being loaded is a project that is contained in the solution, and is referenced by the unit test project. It is used for localization, that is why we are doing an assembly.load.
Is there some temp location on the build machine that the Build Service uses that is causing this issue? is there something i need to configure so that the Assembly can be found by the Build Service?
thanks
Sorry about the misleading comment it's your unit test that's failing, not your build
In the test that's trying to do the load have you marked it up with a deployment attribute
e.g.
[TestMethod]
[DeploymentItem("SomeLibrary.dll")]
public void TestSomeStuff()
{
}
I had the same issue in that my tests ran okay on my machine, but without the attribute they failed on our buold server, they weren't getting moved in to the folder where all the test thingies go.