Unfortunately I have a Specflow test passing locally, but it fails on the VSO Build vNext server, and I really need to see verbose information during the test run so I can figure out what is going on.
But I'm struggling to try to inject ITestOutputHelper into a Specflow binding like so
public SomeSteps(ITestOutputHelper outputHelper)
but Specflow complains with the message
BoDi.ObjectContainerException Interface cannot be resolved: Xunit.Abstractions.ITestOutputHelper (resolution path: ...)
How on earth can view log and view output during a Specflow test?
not sure if I'm using a newer version and it's easier now, but this seems to work for me:
ScenarioContext.Current.ScenarioContainer.Resolve<ITestOutputHelper>().WriteLine("Hello");
This is the best I could come up with, it's not ideal but it does accomplish what you want.
You create a new class that implements your generated xunit class. In my example, the generated class is called YourNormalFeatureClass
public class SpecialTest : YourNormalFeatureClass
{
private Xunit.Abstractions.ITestOutputHelper helper;
public SpecialTest(ITestOutputHelper helper) : base()
{
this.helper = helper;
}
public override void ScenarioSetup(ScenarioInfo scenarioInfo)
{
base.ScenarioSetup(scenarioInfo);
// you'd want a better way to keep track of this string
TechTalk.SpecFlow.TestRunnerManager.GetTestRunner().ScenarioContext.Set(this.helper, "helper");
}
}
Now, you're able to access your XUnit ITestOutputHelper from within your steps file via
var helper = this._scenarioContext.Get<Xunit.Abstractions.ITestOutputHelper>("helper");
helper.WriteLine("output from within the steps file that will be written to xunit!");
You'd need to be defensive with that helper variable to make sure that you don't get any NullReferenceException's
The downside to this is that you now have 2 copies of the same test because you inherited the old test. So in this case you have the tests from SpecialTest and YourNormalFeatureClass. This means that you'd need to not run YourNormalFeatureClass tests and only run the SpecialTest tests.
All of this would be easily solved if SpecFlow allowed you to customize the code generation process. That way you could expose the ITestOutputHelper via the generated code. The consumption of it from within the steps would be the same.
This may be a new addition to SpecFlow since this question was asked (6 years ago), but TechTalk.SpecFlow.Infrastructure.ISpecFlowOutputHelper should solve your problem. Inject it and use it in much the same way you would with xUnit's ITestOutputHelper, e.g.
[Binding]
public class SomeSteps
{
private readonly ISpecFlowOutputHelper output;
public SomeSteps(ISpecFlowOutputHelper output)
{
this.output = output;
}
[When(#"I write some debug info")]
public void WhenIWriteSomeDebugInfo()
{
this.output.WriteLine("Hello world!");
}
}
Related
I want to add a custom test reporter to NUnit. I already did it with NUnit2, but I now need to use NUnit3.
To implement the reporter, I need to get various events from the framework, like start, end and failure of tests.
In NUnit2 I used NUnitHook to register my EventListener and it worked pretty good.
In NUnit3 I need to use the extension point mechanism, but when I add the extension point to the project, VisualStudio (2012 ultimate) immediately fails to discover the NUnit tests.
[TypeExtensionPoint(Description = "Test Reporter Extension")]
public class MyTestEventListener : ITestEventListener
{
public void OnTestEvent(string report)
{
Console.WriteLine(report);
}
}
If I remove the ITestEventListener implementation declaration from the class, it rediscovers the tests perfectly.
[TypeExtensionPoint(Description = "Test Reporter Extension")]
public class MyTestEventListener //: ITestEventListener
{
public void OnTestEvent(string report)
{
Console.WriteLine(report);
}
}
Am I doing something wrong? is there a better way to achieve it?
You don't say where you are putting this code, but I am suspecting it's in your test assembly. If so, that's not where it belongs. NUnit engine extensions get installed into the NUnit engine, so they need to be in a separate assembly. Once you have a separate assembly, you need to tell the engine where it is. Currently, you do this by creating a file of type .addins in the same directory as the engine. (You could modify the existing file, but that introduces maintenance problems in the future)
A future release will have an easier way to install addins, but they will continue to be entirely separate from your tests.
A further problem is that you are using TypeExtensionPointAttribute. I didn't notice this originally in your code and it's probably the biggest error so I'm adding this info now.
An "ExtensionPoint" is the thing you are extending. NUnit defines ExtensionPoints, while you create Extenisons to extend them. TypeExtensionPointAttribute is used inside NUnit to define extension points. It's not used by you. You use the ExtensionAttribute to define your extension.
Your extension should be defined something like this:
[Extension(Description = "Test Reporter Extension", EngineVersion="3.4")]
public class MyTestEventListener : ITestEventListener
{
public void OnTestEvent(string report)
{
Console.WriteLine(report);
}
}
You don't say what version of NUnit you are running. Test Listeners are only supported beginning with version 3.4. The EngineVersion property above is purely documentary at this point, because 3.4 is also the first version to recognize it.
There is a new writeup in the NUnit docs that may be helpful: https://github.com/nunit/docs/wiki/Writing-Engine-Extensions
I am working on a Visual Studio (Community 2015) project in C# that reads metadata from video files. Therefore I want to write a Unit Test that tests if the program handles an inaccessible file the right way. So I want to add a test file to my test data for which the Unit Test does not have read permissions.
But what is the best way to do this? I didn't find any option to set permissions in Visual Studio. The only idea I have at the moment is to revoke reading permissions directly in the properties of the file. Through the Windows File Explorer, thus outside of Visual Studio. But then I wouldn't be able to read the file myself. Also, it doesn't seem to be the right way and I think there should be a more clean way to solve this problem directly in Visual Studio.
Any ideas?
If you are writing a unit test then it should just check if your code is written correcly and not about the external issues which are difficult to replicate or may not be available on other machines. So I think you should read about Dependency Injection and how to do mock dependencies in your test. I can give you an easy to understand example but please do write it in your words.
public interface IFileReader
{
string ReadFile( string filePath );
}
public class FileReader : IFileReader
{
public string ReadFile( string filePath )
{
return System.IO.File.ReadAllText( filePath );
}
}
So Suppose you have a class VideoMetaDataReader then in that class you will inject the interface as a dependency and use the ReadFile method to read. And then in your test
[TestFixture]
public class FileReaderTests
{
[Test]
public void ShouldDisplayANiceMessage_WhenFileIsInaccessible()
{
var moq = new Moq.Mock<IFileReader>();
moq
.Setup( x => x.ReadFile( Moq.It.IsAny<string>() ) )
.Throws<Exception>();
var metaDataReader = new MetaDataReader( moq.Object );
metaDataReader.ReadVideoFile( "video.mp4" );
Assert.AreEqual( 1, meaDataReader.Errors.Count );
}
}
I am assuming there will be a errors property which will return encountered errors but it depends how you want to do it (I didn't think too much on this).
Anyways, the point is don't test what is not covered by your code. What your code needs to do is to display a nice message if there is an exception and that you can do if you mock the interface (which means don't call the actual implementation and instead do something which I want in this test) and then check if your code can handle that. btw i used nunit and moq for test and mocking.
Ok, so here's the deal: I have a complex, heavily dependent class LegacyClass that I'd like to shim so that I get rid of all its dependencies while unit testing other parts of the code base. That class creates dependencies already inside its default constructor, so I need to override it with something with no external dependencies, say, with an empty default constructor. And this is what I'm trying to do (using the Visual Studio 2013 Professional Test Framework):
using System;
using Microsoft.QualityTools.Testing.Fakes;
using Microsoft.VisualStudio.TestTools.UnitTesting;
namespace MyApp_Unit_Tests {
[TestClass]
public class UnitTest1 {
[TestMethod]
public void TestInstantiation1() {
using (ShimsContext.Create()) {
MyNamespace.Fakes.ShimLegacyClass.Constructor = x => { };
var legacyClassInstance = new MyNamespace.Fakes.ShimLegacyClass();
var sut = new MyNamespace.Gui.ViewModels.MainWindowViewModel(legacyClassInstance);
}
}
}
}
However, this does not work. When MainWindowViewModel is instantiated, for some reason all the same external dependencies are still required as with using the original class! Why?
The exception I'm getting, though, is System.BadImageFormatException, so I probably have some confusion about the target CPU settings, too, but anyway the root cause is that it's attempting to load the external DLL referred to only in the original (non-shimmed) legacy class in its default constructor, while I think it no longer should.
Obviously I've been misunderstood, but where's the mistake? Can I not override default constructors, after all, even with using Shims, or is my approach just wrong? What am I missing?
Thanks a million in advance for any advice!
-Seppo
I had same problem and I solved it maybe this approach going to help you
using (ShimsContext.Create())
{
LegacyClass obj=new LegacyClass();
ShimLegacyClass shimobj=new ShimLegacyClass(obj);
//
// modify every thing you want on shimobj
//
shimobj.InstanceBehavior = ShimBehaviors.Fallthrough;
//rest of test
}
This approach helps you to break dependencies in every part you want and keep the rest same as main class
I'm currently trying to learn proper unit-test. So now I'm trying to write unit-tests for a class that should map data from an XML-File to the proper objects. Of course all functionality of the class is dependent on the existence of the corresponding XML-file. The XML-file is loaded in the constructor of the class.
I'm using C# with NUnit. So far I've got two tests:
[Test]
public void ShouldAllowInstanceToBeCreatedWhenXMLFileIsPresent()
{
if (File.Exists(SettingsReader.XML_SETTINGS_PATH))
{
SettingsReader settingsReader = new SettingsReader();
Assert.AreNotEqual(null, settingsReader);
}
}
[Test]
[ExpectedException("Telekanzlei.Clientmanager.XMLDataLayer.XMLFileNotFoundException")]
public void ShouldThrowExceptionWhenXMLFileIsNotPresent()
{
if (!File.Exists(SettingsReader.XML_SETTINGS_PATH))
{
SettingsReader settingsReader = new SettingsReader();
}
else
throw new XMLFileNotFoundException();
}
I'm not sure if checking the existence of the file in the test is a proper way to go, so any suggestions on those test are welcome too. But my question is, how to proceed with the following tests. Obviously all following tests are going to fail, if the XML-file is not present.
So do I assume that the XML-file is present, while keeping in mind, that a failing test could just mean that it's not? That wouldn't seem right to me.
Is there a general pattern, to handle a problem like this?
Thx for any help
edit: rewrote the second test, as it was failing if the file was actually present...
edit2: May it is helping to tell you, what the SettingsReader actually does. So far it looks like this:
public class SettingsReader
{
public static readonly string XML_SETTINGS_PATH = "C:\\Telekanzlei\\Clientmanager_2.0\\Settings.xml";
public XElement RootXElement { get; private set; }
public SettingsReader()
{
if (!File.Exists(XML_SETTINGS_PATH))
throw new XMLFileNotFoundException();
using (var fs = File.OpenRead(XML_SETTINGS_PATH))
{
RootXElement = XElement.Load(fs);
}
}
}
I'm not sure, but I guess a StreamReader wouldn't be the way to go here, would it?
The problem is not with your unit tests but with the design of the class. I'd suggest refactoring the class so it doesn't open the file but instead operates on a stream. Then your unit tests could simply replace a file stream for a memory stream - simples! :)
public class SettingsReader()
{
public SettingsReader(System.IO.StreamReader reader)
{
// read contents of stream...
}
}
// In production code:
new SettingsReader(new StreamReader(File.Open("settings.xml")));
// In unit test:
new SettingsReader(new StringReader("<settings>dummy settings</settings>"));
Remember, opening a file and parsing settings data are two very different concerns.
If you must I suggest you use SetUp method to copy or verify that the file exist.
I suggest making sure the file is present by adding it to the test project and marking it as "copy always" once you get that working there is no need to re-check it.
If you have a lot of tests that require external files perhaps you should use MsTest - it has an attribute called DeploymentItem that makes sure that the file is copied to the same location as the test.
Consider rewriting code so dependencies can be passed in or somehow else stubbed for the code you want to unit-test.
I.e. pass something like "IMySettingsFileProvider" instance to SettingsReader constructor where IMySettingsFileProvider.SettingsXml returns some setting stream. This way you can mock IMySettingsFileProvider interface for the test instead of requiring file to be present on disk.
One option is to put this at the top of the test fixture. Then the tests will only be valid when the file exists.
[SetUp]
public void Setup()
{
Assume.That(File.Exists(SettingsReader.XML_SETTINGS_PATH));
}
I am using nunit 2.5.9.10348 and trying to extract the current test name in the TearDown event so I can assign a screengrab filename the test name however it is always null (see the attached image). The private _context variable does have the TestName however this is no use to me!
Has anyone had success using this new TestContext functionality (from 2.5.7).
From your screenshot I see that _context has keys "TestName" and "Properties". But TestAdapter looks for keys "Test.Name" for Name and "Test.Properties" for Properties. So, there is something wrong with TestContext initialization (I think wrong data was put to Remoting.Messaging.CallContext).
After a little investigation (see comments):
NUnit tests should run by NUnit testig environment for Context to be available.
I had the same issue. It occurred when in a TearDown method I executed method, which actually was to make the teardown
[TearDown]
public void CleanUp()
{
TestContext.CurrentContext.Test.FullName; //!=null
someClassInstance.DoTearDown();
}
class SomeClass
{
public void DoTearDown()
{
TestContext.CurrentContext.Test.FullName; //==null
}
}
I have no idea why, but it seemed so. Is it your case?
UPDATE: Now I looked at the screenshot, so it's not your case :)
Same issue with R# test runner. Just downloaded NUnit sources and added a workaround in TestAdapter to make it work with r#
public string Name
{
get
{
return (_context["Test.Name"] ?? _context["TestName"]) as string;
}
}