I found that one of my tests that passes in VS2013 is failing in VS2015, the test calls a service that includes among other things a call to Console.Clear();
to find out whats going on I made a simple unit test
[TestMethod]
public void ExampleTest()
{
Console.Clear();
}
This test passes in visual studio 2013 but in 2015 I get the following error:
Test Name: ExampleTest
Test FullName: solution.Common.Test.CacheManagerTest.ExampleTest
Test Source: C:\solution.Common.Test\CacheManagerTest.cs : line 34
Test Outcome: Failed
Test Duration: 0:00:00.3015003
Result StackTrace: at System.IO.__Error.WinIOError(Int32 errorCode,
String maybeFullPath) at System.Console.GetBufferInfo(Boolean
throwOnNoConsole, Boolean& succeeded) at System.Console.Clear()
at sol.Common.Test.CacheManagerTest.ExampleTest() in
C:\solution.Common.Test\CacheManagerTest.cs:line 35 Result Message:
Test method Alexandria.Common.Test.CacheManagerTest.ExampleTest threw
exception: System.IO.IOException: The handle is invalid.
I do understand that it is bad design for my service to fail if it is not called by a console. The reason I am asking this question is to understand why this is failing in the new version of Visual Studio. Is this the intended behavior? What changed?
I did not see anything obvious in the change log that would seem related to this.
Edit: I am calling the Console.clear from the following dll
Microsoft\Framework.NETFramework\v4.5.1\mscorlib.dll
Edit 2:
picture of testproject properties in both visual studios
The changes in VS2015 are pretty visible, use the Test > Debug > All Tests to get insight. You can see that it now has a new test host process, its name is TE.ProcessHost.Managed.exe, stored in the C:\Program Files (x86)\Microsoft Visual Studio 14.0\Common7\IDE\CommonExtensions\Microsoft\TestWindow directory.
Previous versions of VS used a different host, vstest.executionengine.exe. One notable change in the new test host is that it is no longer a console mode program. Something you can see by running Dumpbin.exe /headers on the exe.
Another way to see the underlying problem is with Task Manager. Note how running a test in an older VS version causes a conhost.exe process to get added. This is the process that owns the console window for a console mode app. A problem I've seen before is that this process tends to leak, not terminating when the test completes. Adding ever more conhost.exe instances, at one point researching this problem I had 12 of them running. Presumably the changes in VS2015 were meant to address that problem.
Technically you can configure the unit test with a .runsettings file and use the <ForcedLegacyMode> element to force the old test host process to be used. This however has no effect on the outcome of this test, looks like they addressed this in multiple ways.
That's a fair amount of guessing, I recommend you use connect.microsoft.com to file a feedback report. You can quote this Q+A for reference.
Meanwhile, you can consider a workaround. Do note that Console.Clear() is in general a trouble-maker, it will also fail in normal usage when the output of a console mode app is redirected. Very easy to do from a command line prompt with the > operator. Which is the ultimate reason it fails in a unit test. You'll want to make the code resilient so it can work properly both in production and in a unit test. Like this:
if (!Console.IsOutputRedirected) Console.Clear();
Which requires targeting .NET 4.5 or higher. You can use the code in this SO post if you need to target earlier versions.
Related
Can we publish the details of failed test cases statistics in HTML report of SpecRun?
Example: Analytics of failed test cases in previous executions.
If one test scenario is failing from the past 5 runs, In Error Details section of specrun report, 5 needs to be displayed against that test case. So that we can analyse whether this test case failure is new in current run or its there from previous builds.
Can we make any settings to run only failed test cases in the previous run with SpecFlow+SpenRun ?
To publish historical failed tests and displayed them against that test case and run only failed test cases might not be achieveable in azure devops.
However you can achieve rerun failed test cases via visual studio test task. You can configure the threshold to rerun failed tests and the maximum attempts under Execution options in visual studio test task.
"Run All" from "Test Explorer" does not complete (VS2017 Enterprise) anymore. It stalls with Passed (411), Not Run (309). The counts vary a little, usually roughly half and half.
The output window (Visual Studio | Output tab | Show output from: Tests) contains the following error message:
"The active test run was aborted. Reason: Unhandled Exception: System.AppDomainUnloadedException: Attempted to access an unloaded AppDomain."
The tests continue to run fine in ReSharper (720 of 720 pass). R# is where I usually run my tests. I jump over to Microsoft's "Test Explorer" when I am trying to Analyze Code Coverage (though the tests stall with or without code coverage). It (Analyze Code Coverage) worked as recently as 5/15/2018 (and at least a half dozen to a dozen times before that).
Check the tests output pane for more details - open the output window and select "Tests" option from the "Show output from" combo box.
In my case, my tests project was targeting a version of .NET core I didn't have installed. I just had to change the project target framework to the correct version and it solved the problem.
The Microsoft test runner was being tripped up by a single unit test class that happened to have Task.Run() calls such as the following:
var task = Task.Run(async () =>
{
<various code>
});
These tests were missing calls to task.Wait() to wait for each task to finish before exiting the test.
(This appears to trip up the Microsoft test runner but not the ReSharper test runner. Specifically, the Microsoft Test Runner aborts the sln test run and skips 300+ tests. ReSharper was able to run all tests without incident.)
Aside: The reason for the varied behavior on Windows 7 versus Windows 10 is because the test class was for a Windows 10 sensitive 3rd party control/library.
Just simple
Build -> Clear Solution
Help for me.
I used the CodedUI to record the installation of a product, and tried to replay the recording in visual studio. I'm getting some curious issues and I'm not sure how to work around these.
Message: Test method CodedUITestProject5.CodedUITest1.CodedUITestMethod1 threw exception:
Microsoft.VisualStudio.TestTools.UITest.Extension.FailedToLaunchApplicationException: "The application cannot be started because it cannot be found in the specified location, or the user does not have sufficient permissions to start the application." File: C:\Users\Sakamoto\AppData\Local\Temp\{CF4F5CDB-4597-4C70-BBFD-2687BB031067}\.cr\Setup.exe. Alternative file name: %TMP%\{CF4F5CDB-4597-4C70-BBFD-2687BB031067}\.cr\Setup.exe
The funny thing is the setup file is located on my desktop, and that's where the recording launched it from. Looking in UIMap.Designer.cs, it has to correct path listed in the recorded method right under the Variable Declarations section.
I am also running Visual Studio 2017 as Administrator.
Changing the UIMap.Designer.cs file so that all paths point to the file on the desktop, I get the following error:
Message: Test method CodedUITestProject5.CodedUITest1.CodedUITestMethod1 threw exception:
Microsoft.VisualStudio.TestTools.UITest.Extension.FailedToLaunchApplicationException: "The application cannot be started. This could be due to one of the following reasons:
1) Another instance of the application is already running and only one instance can be running at a time.
2) The application started another process and has now stopped. You may need to launch the process directly.
3) You do not have sufficient privileges for this application." File: C:\Users\Sakamoto\Desktop\Setup.exe.
Can someone help me get an automated run of the setup process going?
In the Test Explorer window of Visual Studio 2015 for C#, I select all the 360 test methods in my solution, and choose "Run Selected Tests". But not all of them are executed, as indicated by the Output window:
------ Run test started ------
The active Test Run was aborted because the execution process exited unexpectedly. To investigate further,
enable local crash dumps either at the machine level or for process
te.processhost.managed.exe. Go to more details:
http://go.microsoft.com/fwlink/?linkid=232477
========== Run test finished: 188 run (0:00:12.7254721) ==========
How can I find out which test method fails to run, narrow the culprit test method down? Thanks.
Probably not the most subtle way to do it, but you could comment your tests one by one, and see at which point your run stops failing.
My problem is very similar to one reported here: UnitTestIsolationException when debugging tests using Fakes
However, I am not even able to run the tests. Using the sample code provided here: Isolating Code Under Test with Microsoft Fakes (Getting Started with Shims), I get following exception on running the test below :
the offending line is :
using (ShimsContext.Create())
Exception on running the test:
Test Name: TestMethod1
Test FullName: TestingShimsAndStubs.UnitTest1.TestMethod1
Test Source: c:\poc\TestingShimsAndStubs\TestingShimsAndStubs\UnitTest1.cs : line 12
Test Outcome: Failed
Test Duration: 0:00:00.0182403
Result Message:
Test method TestingShimsAndStubs.UnitTest1.TestMethod1 threw
exception:
Microsoft.QualityTools.Testing.Fakes.UnitTestIsolation.UnitTestIsolationException:
Failed to get profiler module handle 'C:\Program Files (x86)\Microsoft
Visual Studio
12.0\Common7\IDE\CommonExtensions\Microsoft\IntelliTrace\12.0.0\Microsoft.IntelliTrace.Profiler.12.0.0.dll'.
The specified module could not be found --->
System.ComponentModel.Win32Exception: The specified module could not
be found Result StackTrace:
at Microsoft.QualityTools.Testing.Fakes.UnitTestIsolation.LibraryMethods.GetModuleHandle(String
fileName)
at Microsoft.QualityTools.Testing.Fakes.UnitTestIsolation.IntelliTraceInstrumentationProvider.LoadProfilerModule(String
profilerPath)
--- End of inner exception stack trace ---
at Microsoft.QualityTools.Testing.Fakes.UnitTestIsolation.IntelliTraceInstrumentationProvider.LoadProfilerModule(String
profilerPath)
at Microsoft.QualityTools.Testing.Fakes.UnitTestIsolation.IntelliTraceInstrumentationProvider.Initialize()
at Microsoft.QualityTools.Testing.Fakes.UnitTestIsolation.UnitTestIsolationRuntime.InitializeUnitTestIsolationInstrumentationProvider()
at Microsoft.QualityTools.Testing.Fakes.Shims.ShimRuntime.CreateContext()
at Microsoft.QualityTools.Testing.Fakes.ShimsContext.Create()
at TestingShimsAndStubs.UnitTest1.TestMethod1() in c:\poc\TestingShimsAndStubs\TestingShimsAndStubs\UnitTest1.cs:line 16
I am using Microsoft Visual Studio Ultimate 2013 Edition (Version 12.0.40629.00 Update 5) with .Net Framework v 4.6.01055 on my machine. Also tried running the tests on Visual Studio 2015 Enterprise Edition as well
Am not sure what VS2013/VS2015 was happening but opening it as an Administrator (Start -> Right Click Visual Studio 2013 -> Run As Administrator ) and then opening the Test project seems to have fixed the issue!!!
Maybe there is some Corporate Policy that is preventing my regular user account to not access the folder. Impersonating an admin account seems to have fixed the issue. Thanks a lot for fixing it!
The solution came after opening a post here Unable to run Unit Tests that uses Microsoft Fakes where one of the admins fixed the issue