We have a VS2013 .net 5.0 Solution (VS2013 Premium) with all Unit tests passing fine locally but failing several tests when running in VS Test Loader by TFS Build with this or similar Exception: System.TypeLoadException: Could not load type 'System.Diagnostics.Fakes.ShimEventLog' from assembly 'System.4.0.0.0.Fakes, Version=4.0.0.0, Culture=neutral, PublicKeyToken=0ae41878053f6703'.
This is an example of a failing test:
[TestMethod]
public void WriteToEventLogTest_HappyPath()
{
EventLogEntryType eTypeInfo = EventLogEntryType.Information;
bool sourceExistCalled = false;
bool writeEntrycalled = false;
using (ShimsContext.Create())
{
ShimEventLog.SourceExistsString = s =>
{
sourceExistCalled = true;
return true;
};
ShimEventLog.AllInstances.WriteEntryStringEventLogEntryType = (#this, str, et) =>
{
writeEntrycalled = true;
};
Logging.WriteToEventLog(IpAddress, eTypeInfo);
Assert.IsTrue(sourceExistCalled, "SourceExist() not called");
Assert.IsTrue(writeEntrycalled, "WriteEntry() not called");
}
}`
We using TFS 2013 update 5 running on Windows Server 2012 R2. Is there anything that can likely cause this problem? Should we update TFS to the latest which is Update 5 at the moment?
Problem was resolved by sharing fakes configuration files between the Test projects on a Solution level
In our situation, we were running several unit test DLLs through VSTest in Jenkins.
Example call:
"c:\Program Files (x86)\Microsoft Visual Studio 14.0\Common7\IDE\CommonExtensions\Microsoft\TestWindow\vstest.console.exe"
"./TestDLL1.UnitTests/bin/Debug/TestDLL1.UnitTests.dll"
"./TestDLL2.UnitTests/bin/Debug/TestDLL2.UnitTests.dll"
"./TestDLL3.UnitTests/bin/Debug/TestDLL3.UnitTests.dll"
/Enablecodecoverage /UseVsixExtensions:false /Logger:trx
Some of the test projects had Fakes for the same DLL, with most setup to Fake everything. One test project was only faking one class, as the XML shows below, in order to prevent the warning about "Some fakes could not be generated."
<Fakes xmlns="http://schemas.microsoft.com/fakes/2011/" Diagnostic="true">
<Assembly Name="Utilities"/>
<StubGeneration>
<Clear />
</StubGeneration>
<ShimGeneration>
<Clear />
<Add FullName="UIUtils.ExceptionDisplay"/>
</ShimGeneration>
</Fakes>
For some reason, VSTest was using the above version for the other test projects. Modifying the one test project to generate all the fakes fixed the issue with System.TypeLoadException. We may consolidate our Fakes in the future, to be able to more easily limit which classes are faked, but for now this was a much quicker solution to the issue.
If only there was an easier way to suppress the "Some fakes could not be generated" warning...
just has a similar issue when using Shims in multiple separate Unit test projects. Apparently the generated shims can overwrite each over or something like that.
Here's the instructions I followed to fix it: Scroll to the middle of this page: https://msdn.microsoft.com/en-us/library/hh708916.aspx
It's under the heading of optimise build times, but use it to fix your problem too.
Because such assemblies rarely change on your machine, you can reuse the generated Fakes assemblies in other projects. From your unit test projects, you can simply take a reference to the
compiled Fakes assemblies that are placed under the FakesAssemblies
in the project folder.
Create a new Class Library with the .NET runtime version matching
your test projects. Let’s call it Fakes.Prebuild. Remove the
class1.cs file from the project, not needed.
Add reference to all the System and third-party assemblies you need
Fakes for. and generate the fakes, edit the .fakes file if you like,
and build to generate the fake assemblies.
From your Unit Test projects Just make sure that you have a reference
to the Fakes runtime DLL: C:\Program Files\Microsoft Visual Studio
12.0\Common7\IDE\PublicAssemblies\Microsoft.QualityTools.Testing.Fakes.dll
Then for each assembly that you have created Fakes for, add a
reference to the corresponding DLL file in the
Fakes.Prebuild\FakesAssemblies folder of your project. (This folder
get created when you compile)
To confirm You will need to add the reference by Browsing to the
generated fake assembly..
Related
I'm working on a tool that needs to extract the NUnit tests names that are contained in an assembly.
As I need the exact name of the tests when they are run, and NUnit's way of naming the tests is not straightforward, I use the NUnit TestEngine to explore the assembly.
Something like that :
Assembly testAssembly; //loaded somewhere else
using var testEngine = new TestEngine();
testEngine.WorkDirectory = Path.GetDirectoryName(testAssembly.Location);
var package = new TestPackage(testAssembly.Location);
using var runner = testEngine.GetRunner(package);
var testsFound = runner.Explore(new TestFilter(string.Empty));
I build the whole tool with one of our projects as test subject, and it works well.
But when I try it on other similar projects (.Net 6, NUnit test projects for a web API), I have the following error :
Unable to load one or more of the requested types.
Could not load file or assembly 'Microsoft.AspNetCore.Hosting.Abstractions, Version=6.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60'. The system cannot find the file specified.
The tests run properly inside the Visual Studio, with VS test runner or Resharper.
The current directory is set to the bin folder.
Looking at the project, it seems that this is a implicit dependency of Microsoft.AspnetCore.Hosting (referenced by AppInsight)
But :
I don't see any version 6.0.0.
I have another similar project that loads properly and have the exact same dependencies
I can't find any Nuget package that has this version (highest is 2.2.0), so I can't force the reference.
The Assembly loads properly (with Assembly.LoadFrom()), it fails in NUnit, when it tries to call LoadTests
I can't find where this reference is needed and why one solution is working and others are not, while they are very similar !
I'm out of ideas to make it work, so I'm looking for suggestions.
Have you considered simply telling NUnit to produce the list of tests for you?
The --explore option followed by a file name creates XML output but without a following name it produces a readable list of names on the console. You can redirect the output to a file if that's what you want.
nunit3-console yourtests.dll --explore
UPDATE
Make sure you use the standard .NET Framework build of NUnit3-console with the above option, not the .NET 6.0 build. The standard runner creates whatever process is needed to properly explore the test assembly.
I have a solution, which contains numerous projects. Each of these projects has an associated test project which uses MSTest V2. Some of these test projects take advantage of Microsoft Fakes.
When I run the tests in the Visual Studio IDE, everything works fine. However, my build pipeline fails. I attempted to emulate what my build pipeline is doing and it also fails on my machine.
The error message I get is the following:
System.TypeLoadException: Could not load type
'System.Fakes.ShimDateTime' from assembly 'mscorlib.4.0.0.0.Fakes,
Version=4.0.0.0, Culture=neutral, PublicKeyToken=0ae41878053f6703'.
After playing around with the vstest.console.exe application, it seems that if I run each test assembly separately all of the tests pass, but if I attempt to pass in multiple assemblies which utilize Microsoft Fakes they fail.
vstest.console.exe "Path/To/First.dll" Passes
vstest.console.exe "Path/To/Second.dll" Passes
vstest.console.exe "Path/To/First.dll" "Path/To/Second.dll" Fails
vstest.console.exe "Path/To/First.dll" "Path/To/Second.dll" /InIsolation Fails
I'm not sure what is going on here or how to resolve this issue.
I eventually figured out what the problem was:
Test Project A had not defined System.DateTime in a Microsoft Fakes config file, so an assembly was generated named mscorlib.4.0.0.0.Fakes.dll which was missing the System.Fakes.ShimDateTime.
Test Project B had defined System.DateTime in a Microsoft Fakes config file, so an assembly was generated named mscorlib.4.0.0.0.Fakes.dll which contained the type System.Fakes.ShimDateTime.
This lead the following structure:
ProjectA.Tests/bin/Debug/
ProjectA.Tests.dll -- Contains the unit tests for Project A
mscorlib.4.0.0.0.Fakes.dll -- Does not contain System.Fakes.ShimDateTime
ProjectB.Tests/bin/Debug/
ProjectB.Tests.dll -- Contains the unit tests for Project B
mscorlib.4.0.0.0.Fakes.dll -- Does contain System.Fakes.ShimDateTime
I'm not sure what the Visual Studio IDE is doing differently, but the vstest.console.exe seems to load all of the assemblies, ignoring duplicates based on fully qualified names. That is, it loads the following files:
ProjectA.Tests/bin/Debug/ProjectA.Tests.dll
ProjectA.Tests/bin/Debug/mcsorlib.4.0.0.0.Fakes.dll
ProjectB.Tests/bin/Debug/ProjectB.Tests.dll
It ignored the file ProjectB.Tests/bin/Debug/mcsorlib.4.0.0.0.Fakes.dll, which had the type System.Fakes.ShimDateTime defined within it.
The only solution to this is to run a separate vstest.console.exe per test project (instead of all at once), or to ensure that the generated Microsoft Fakes assemblies (*.Fakes.dll) have the same types defined across the board, so it doesn't matter which version of mscorlib.4.0.0.0.Fakes.dll is loaded. And, since the Azure Pipelines VsTest task uses a single run of vstest.console.exe this is important to know.
I Have lots of unit tests where I shim the Microsoft.Win32.RegistryKey using Microsoft Fakes. When I run the build on my local machine the unit test passed. But if I run it on build machine the unit test fails with the following error for these tests.
System.TypeLoadException: Could not load type
'Microsoft.Win32.Fakes.ShimRegistryKey' from assembly
'mscorlib.4.0.0.0.Fakes, Version=4.0.0.0, Culture=neutral,
PublicKeyToken=0ae41878053f6703'.
The curious thing is that both my local machine and build machine are 64-bit and the unit test project are targeting AnyCPU. Is there some configuration could be missing on the build machine?
I just has a similar issue when using Shims in multiple separate Unit test projects. Apparently the generated shims can overwrite each over or something like that.
Here's the instructions I followed to fix it:
Scroll to the middle of this page: https://msdn.microsoft.com/en-us/library/hh708916.aspx
It's under the heading of optimise build times, but use it to fix your problem too.
Because such assemblies rarely change on your machine, you can reuse the generated Fakes assemblies in other projects.
From your unit test projects, you can simply take a reference to the compiled Fakes assemblies that are placed under the FakesAssemblies in the project folder.
Create a new Class Library with the .NET runtime version matching
your test projects. Let’s call it Fakes.Prebuild. Remove the
class1.cs file from the project, not needed.
Add reference to all the System and third-party assemblies you need
Fakes for. and generate the fakes, edit the .fakes file if you like,
and build to generate the fake assemblies.
From your Unit Test projects Just make sure that you have a reference
to the Fakes runtime DLL: C:\Program Files\Microsoft Visual Studio
12.0\Common7\IDE\PublicAssemblies\Microsoft.QualityTools.Testing.Fakes.dll
Then for each assembly that you have created Fakes for, add a
reference to the corresponding DLL file in the
Fakes.Prebuild\FakesAssemblies folder of your project. (This folder
get created when you compile)
To confirm You will need to add the reference by Browsing to the
generated fake assembly..
I am trying to setup some unit testing for a connector assembly that communicates with a web service. I was following along with an example provided:
http://blogs.msdn.com/b/atverma/archive/2012/05/03/unit-test-a-project-having-external-dependency-wcf-proxy-using-fakes-amp-visual-studio-11-beta.aspx
However, when I add fakes for my connector assembly, the project fails to build and no errors are reported. So, I modified the fakes file to turn on diagnostic and use noisy verbosity... and still received no error messages. Then I disabled BOTH stub and shim generation in the fakes file, and it STILL failed to build.
If i remove the fakes for my connector assembly, it builds. I add fakes for it, the project does not build. Without getting any error messages, I have no idea how to make this work.
Any help would be appreciated.
Put the Diagnostic to false for the mean time.
<Fakes xmlns="http://schemas.microsoft.com/fakes/2011/">
<Assembly Name="Namespace here" Version="1.4.4731.18265"/>
</Fakes>
For last couple hours i have been trying to generate a Unit Test for a Silverlight application.
A number of posts refer to a "Silverlight Unit Test Project" which is part of the Silverlight Toolkit. But I downloaded the toolkit and still do not have the Test Project, it seems to be only available in VS 2010?
I have added a "Silverlight Class Library" project and added references to:
Microsoft.Silverlight.Testing
Microsoft.VisualStudio.QualityTools.UnitTesting.Silverlight
and the following TestClass:
using Microsoft.Silverlight.Testing;
using Microsoft.VisualStudio.TestTools.UnitTesting;
namespace UnitTesting
{
[TestClass]
public class Class
{
[TestMethod]
public void TestMethod()
{
.....
}
}
}
But there are no tests being discovered by Visual Studio 2012 Test Explorer. Even after re-build of solution and restart of application.
Anyone have any ideas? Is this even possible?
This link has the answer that worked for me:
http://social.msdn.microsoft.com/Forums/vstudio/en-US/5e991b0d-8061-4c4e-a17d-82b4abd58d6c/vs-2012-silverlight-unittest
I recommend starting a new Silverlight project and installing the
SilverlightToolkit-Testing NuGet package. In your test files, put in
usings for Microsoft.Silverlight.Testing and
Microsoft.VisualStudio.TestTools.UnitTesting and use regular
[TestClass] and [TestMethod] attributes. To run them, you can use the
Toolkit test runner by putting RootVisual =
UnitTestSystem.CreateTestPage(); in your App.Application_Startup(),
use Silverlight Unit Test Adapter (which currently is at v0.0.1 and
doesn't really work), or (the best approach by far) install ReSharper
and the AgUnit plugin.
To complete this thread,
The Silverlight DLLS are located in C:\Program Files (x86)\Microsoft SDKs\Silverlight\v5.0\Toolkit\dec11\Testing
I could not get Resharper 7.1 to run the tests but this library helped. You will need to extract using 7-zip so that the DLLS are not blocked. Then restart Visual Studio 2012 and Resharper will run your unit tests.
I believe that you need to install the Silverlight Unit Test Adapter to get the tests to show up in Test Explorer
http://visualstudiogallery.msdn.microsoft.com/caca1e81-becb-41e4-9110-bc247f3f400b?SRC=VSIDE
I was able to run some tests:
Given Visual Studio 2012 Professional(with test runner).
Create class library targeting .NET 4.5 with name like MyProject.Tests.
Reference C:\Program Files (x86)\Microsoft Visual Studio 11.0\Common7\IDE\PublicAssemblies\Microsoft.VisualStudio.QualityTools.UnitTestFramework.dll or from your location.
Add test as usual test for .NET 4.5.
Add project reference to MyProject - project targeting Silverlight 5.
Add some tests. Build. May get error of missing reference:
Error 12 The type 'System.Xml.Serialization.IXmlSerializable' is defined in an assembly that is not referenced. You must add a reference to assembly 'System.Xml, Version=5.0.5.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e'
Reference C:\Program Files (x86)\Reference Assemblies\Microsoft\Framework\Silverlight\v5.0\System.Xml.dll
Build and get same error. Open *.csproj and ensure hint path:
xml
<Reference Include="System.Xml">
<HintPath>C:\Program Files (x86)\Reference Assemblies\Microsoft\Framework\Silverlight\v5.0\System.Xml.dll</HintPath>
</Reference>
Run test, e.g. via right click on TestMethod -> Run Tests. May get error:
System.IO.FileNotFoundException: Could not load file or assembly 'System.Windows, Version=5.0.5.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e' or one of its dependencies. The system cannot find the file specified.WRN: Assembly binding logging is turned OFF.
To enable assembly bind failure logging, set the registry value [HKLM\Software\Microsoft\Fusion!EnableLog] (DWORD) to 1.
Note: There is some performance penalty associated with assembly bind failure logging.
To turn this feature off, remove the registry value [HKLM\Software\Microsoft\Fusion!EnableLog].
Fix is:
<Reference Include="System.Windows">
<HintPath>C:\Program Files (x86)\Reference Assemblies\Microsoft\Framework\Silverlight\v5.0\System.Windows.dll</HintPath>
</Reference>
Notes:
Recall that Silverlight 5 assemblies are in the same format as .NET 4.5.
Test fail because .NET 4.5 assemblies are default for project, we need to override via HintPath. I think there may be other way via MSBuild scripts modification and/or assembly binding redirection.
.NET core assemblies are loaded from 4.5, if these differ from Silverlight things may fail. I hope not.
Features depending on Silverlight hosting runtime may fail. Like showing Silverlight window or access HTML DOM. Which is good indicators to refactor code to be Silverlight agnostic. Possible error:
Test Outcome: Failed
Result Message:
System.DllNotFoundException: Unable to load DLL 'agcore': The specified module could not be found. (Exception from HRESULT: 0x8007007E)
Result StackTrace:
at MS.Internal.XcpImports.Application_GetCurrentNative(IntPtr context, IntPtr& obj)
at MS.Internal.XcpImports.Application_GetCurrent(IntPtr& pApp)
at System.Windows.Application.get_Current()
indicates need to load ActiveX runtime for SL into process.
Referencing Silverlight Toolkit versions of testing assemblies(with [TestMethod] attribute inside) instead of .NET one leads to issue that tests are visible, but not run.