I am working on a Visual Studio (Community 2015) project in C# that reads metadata from video files. Therefore I want to write a Unit Test that tests if the program handles an inaccessible file the right way. So I want to add a test file to my test data for which the Unit Test does not have read permissions.
But what is the best way to do this? I didn't find any option to set permissions in Visual Studio. The only idea I have at the moment is to revoke reading permissions directly in the properties of the file. Through the Windows File Explorer, thus outside of Visual Studio. But then I wouldn't be able to read the file myself. Also, it doesn't seem to be the right way and I think there should be a more clean way to solve this problem directly in Visual Studio.
Any ideas?
If you are writing a unit test then it should just check if your code is written correcly and not about the external issues which are difficult to replicate or may not be available on other machines. So I think you should read about Dependency Injection and how to do mock dependencies in your test. I can give you an easy to understand example but please do write it in your words.
public interface IFileReader
{
string ReadFile( string filePath );
}
public class FileReader : IFileReader
{
public string ReadFile( string filePath )
{
return System.IO.File.ReadAllText( filePath );
}
}
So Suppose you have a class VideoMetaDataReader then in that class you will inject the interface as a dependency and use the ReadFile method to read. And then in your test
[TestFixture]
public class FileReaderTests
{
[Test]
public void ShouldDisplayANiceMessage_WhenFileIsInaccessible()
{
var moq = new Moq.Mock<IFileReader>();
moq
.Setup( x => x.ReadFile( Moq.It.IsAny<string>() ) )
.Throws<Exception>();
var metaDataReader = new MetaDataReader( moq.Object );
metaDataReader.ReadVideoFile( "video.mp4" );
Assert.AreEqual( 1, meaDataReader.Errors.Count );
}
}
I am assuming there will be a errors property which will return encountered errors but it depends how you want to do it (I didn't think too much on this).
Anyways, the point is don't test what is not covered by your code. What your code needs to do is to display a nice message if there is an exception and that you can do if you mock the interface (which means don't call the actual implementation and instead do something which I want in this test) and then check if your code can handle that. btw i used nunit and moq for test and mocking.
Related
Description
I am writing unit tests for a method, which copies a file from a source to a destination. Basically it includes this code:
public void MyMethod()
{
// ...
File.Copy(source, destination, true);
// ...
}
In my unit test project, I have a test file: (test.png), which is located in the Resources folder of my unit test project. And I've set the Copy to Output property to Always.
I have 3 unit tests which are testing this method.
When they hit the line of the code which copies the file: source = "Resources\\test.png".
Issue
When I run the unit test individually, they all pass and everything is fine.
However, when I run All Tests in Visual Studio, I get this run time error and unit tests fail:
System.IO.DirectoryNotFoundException
Could not find a part of the path 'Resources\test.png'.
My Thoughts...(UPDATED)
Probably because Visual Studio runs each unit tests simultaneously in a separate thread and they all accessing the same file at the same time?
I think for every unit test, Visual Studio is cleaning bin/Debug and bin/Release folders. Then it copies all the required project files in that folder. This causes sometimes the file actually does not exist?
Question
How can I fix this problem?
Is there any settings of configurations to resolve this?
How can I run all unit tests in Visual Studio (and Team City) when multiple unit tests are accessing the same file?
You could try to rule out the multi-threading issue by following the instructions from MSDN: Executing Unit Tests in parallel on a multi-CPU/core machine, setting parallelTestCount to 1. If the tests now pass, you've narrowed down the problem.
However, if your tests are still failing when you run them in a group - and I think this is the more likely scenario -, then my advice would be to check for any state those tests are sharing. The pattern you describe (i.e. passes in isolation; fails when not in isolation) is a symptom typically exhibited by tests that are (incorrectly) sharing state, and that state is being modified by those tests, causing one or more tests to fail.
Accessing the same file should not be a problem. Make sure you don't have a cleanUp Fixture(TestSuite level) to delete the file. Because from exception it looks like the file is being deleted after running a test.
Also concurrent read operation is fine and perfectly legal. If your unit tests are overwriting the file then it's a problem.
What happened was, since I was using relative path to the testing files, for some reason when running the unit tests in batch, test runner working directory is different than when running individual tests, hence it couldn't find the directory.
So I used this function to build the absolute path to the testing files:
private string GetFilePath([CallerFilePath] string path = "")
{
return path;
}
Then:
string projectDir = Path.GetDirectoryName(GetFilePath());
string testFile = Path.Combine(projectDir, #"Resources\test.png";
I'd speculate that your problem is that one of the tested methods changes directory, given the explicit "Directory Not Found" exception. It's improbable that file locking or any concurrency problems would cause the behaviour described.
If unit testing you shouldn't really be testing whether File.Copy (or any of the File class methods) worked since you didn't write that code. Instead you should test whether your code interacts correctly with the File type (i.e. did it pass the correct source file name, desination file name and overwrite value when you called "Copy"). First create an interface for the File class and a wrapper for it that implements the interface;
public interface IFileWrapper
{
void Copy(string sourceFileName,string destFileName,bool overwrite);
//Other required file system methods and properties here...
}
public class FileWrapper : IFileWrapper
{
public void Copy(string sourceFileName, string destFileName, bool overwrite)
{
File.Copy(sourceFileName, destFileName, overwrite);
}
}
You should then make the class you are testing include an IFileWrapper parameter (dependency injection). In your unit tests you can then use a mocking framework such as Moq or you could write your own mock;
public class MockFileWrapper : IFileWrapper
{
public string SoureFileName { get; set; }
public string DestFileName { get; set; }
public bool Overwrite { get; set; }
public void Copy(string sourceFileName, string destFileName, bool overwrite)
{
SoureFileName = sourceFileName;
DestFileName = destFileName;
Overwrite = overwrite;
}
}
In real implementations pass in FileWrapper as the IFileWrapper parameter but in your unit tests pass in MockFileWrapper. By checking the properties of mockFileWrapper in your unit tests you can now determine whether you class calls Copy and how it is called. Since you are no longer sharing a real file between your unit tests you will avoid the chance of the tests sharing state or potentially locking the file.
As you mentioned in your answer, the test framework does not always run tests with the working directory set to your build output folder.
To instruct the test framework to place build artifacts or other files from your build output into the test directory, you need to use DeploymentItemAttribute. For your case, you would do something like:
const string destination = "Destination.txt";
const string source = "MyData.txt";
[DeploymentItem(source)]
[TestMethod]
public void MyMethod()
{
// …
File.Copy(source, destination, true);
// …
}
[TestCleanup]
public void Cleanup()
{
// Clean up the destination so that subsequent tests using
// the same deploy don’t collide.
File.Delete(destination);
}
Also ensure that your files are marked with a Build Action of Contents and Always Copy. Otherwise, they won’t be in the build output directory and won’t be able to be copied to the test directory.
Unfortunately I have a Specflow test passing locally, but it fails on the VSO Build vNext server, and I really need to see verbose information during the test run so I can figure out what is going on.
But I'm struggling to try to inject ITestOutputHelper into a Specflow binding like so
public SomeSteps(ITestOutputHelper outputHelper)
but Specflow complains with the message
BoDi.ObjectContainerException Interface cannot be resolved: Xunit.Abstractions.ITestOutputHelper (resolution path: ...)
How on earth can view log and view output during a Specflow test?
not sure if I'm using a newer version and it's easier now, but this seems to work for me:
ScenarioContext.Current.ScenarioContainer.Resolve<ITestOutputHelper>().WriteLine("Hello");
This is the best I could come up with, it's not ideal but it does accomplish what you want.
You create a new class that implements your generated xunit class. In my example, the generated class is called YourNormalFeatureClass
public class SpecialTest : YourNormalFeatureClass
{
private Xunit.Abstractions.ITestOutputHelper helper;
public SpecialTest(ITestOutputHelper helper) : base()
{
this.helper = helper;
}
public override void ScenarioSetup(ScenarioInfo scenarioInfo)
{
base.ScenarioSetup(scenarioInfo);
// you'd want a better way to keep track of this string
TechTalk.SpecFlow.TestRunnerManager.GetTestRunner().ScenarioContext.Set(this.helper, "helper");
}
}
Now, you're able to access your XUnit ITestOutputHelper from within your steps file via
var helper = this._scenarioContext.Get<Xunit.Abstractions.ITestOutputHelper>("helper");
helper.WriteLine("output from within the steps file that will be written to xunit!");
You'd need to be defensive with that helper variable to make sure that you don't get any NullReferenceException's
The downside to this is that you now have 2 copies of the same test because you inherited the old test. So in this case you have the tests from SpecialTest and YourNormalFeatureClass. This means that you'd need to not run YourNormalFeatureClass tests and only run the SpecialTest tests.
All of this would be easily solved if SpecFlow allowed you to customize the code generation process. That way you could expose the ITestOutputHelper via the generated code. The consumption of it from within the steps would be the same.
This may be a new addition to SpecFlow since this question was asked (6 years ago), but TechTalk.SpecFlow.Infrastructure.ISpecFlowOutputHelper should solve your problem. Inject it and use it in much the same way you would with xUnit's ITestOutputHelper, e.g.
[Binding]
public class SomeSteps
{
private readonly ISpecFlowOutputHelper output;
public SomeSteps(ISpecFlowOutputHelper output)
{
this.output = output;
}
[When(#"I write some debug info")]
public void WhenIWriteSomeDebugInfo()
{
this.output.WriteLine("Hello world!");
}
}
Reading through https://msdn.microsoft.com/en-us/library/jj635153.aspx I have created a .RunSettings files with a few parameters similar to the example:
<TestRunParameters>
<Parameter name="webAppUrl" value="http://localhost" />
<Parameter name="webAppUserName" value="Admin" />
<Parameter name="webAppPassword" value="Password" />
</TestRunParameters>
I plan on having a .RunSettings file for each of our environments with appropriate URLs and credentials for running a CodedUI test on the specified RunSettings file's environment.
I can see that from command line to reference the Settings file I can run:
vstest.console myTestDll.dll /Settings:Local.RunSettings /Logger:trx
vstest.console myTestDll.dll /Settings:QA.RunSettings /Logger:trx
etc...
But I don't see any way that calls out how to actually utilize the TestRunParameters from within the codedUI test.
What I would like to do is set up test initializers that use the TestRunParameters to determine where to log in, and what credentials to use. Something like this:
[TestInitialize()]
public void MyTestInitialize()
{
// I'm unsure how to grab the RunSettings.TestRunParameters below
string entryUrl = ""; // TestRunParameters.webAppUrl
string userName = ""; // TestRunParameters.webAppUserName
string password = ""; // TestRunParameters.webAppPassword
LoginToPage(entryUrl, userName, password);
}
public void LoginToPage(string entryUrl, string userName, string password)
{
// Implementation
}
Information on how to reference the TestRunParameters is greatly appreciated!
EDIT
/// <summary>
/// Summary description for CodedUITest1
/// </summary>
[CodedUITest]
public class CodedUITest1
{
public static string UserName = string.Empty;
[ClassInitialize]
public static void TestClassInitialize(TestContext context)
{
UserName = context.Properties["webAppUserName"].ToString();
Console.WriteLine(UserName);
}
[TestMethod]
public void CodedUITestMethod1()
{
this.UIMap.RecordedMethod1();
// To generate code for this test, select "Generate Code for Coded UI Test" from the shortcut menu and select one of the menu items.
}
// Rest of the default class - TestContext instantiation, UI map instantiation, etc
}
The exception I'm getting when running:
NullReference Exception
#williamfalconeruk I have updated my test class as above, but I am still getting the same error, any idea what I'm doing wrong?
For those that use Resharper with this issue, I discovered the fix (no need to disable Resharper):
Go to Visual Studio top menu -> Resharper -> Options
Find the Tools section, expand "Unit Testing"
Click on "MsTest". The checkbox should be on enabled, but the Test Settings file path below that may be blank. If it is, click on browse and select the runsettings file you want to use.
Click save, rebuild and try to run the tests, the parameters should now work.
Not sure why but simply selecting the Test Settings file from the Tests menu -> Test Settings does not actually work when using Resharper, so this file needs to be explicitly pointed to directly in the Resharper options.
I also came across this recently, as we wanted to move away from legacy environment variable usage. The solution provided below was suitable for our needs, but there may be a better one...
It turns out you can access these in the TestContext of a ClassInitialize method of your Test fixture. There is a Properties dictionary which has these parameters, and you could access the values here:
[ClassInitialize]
public static void TestClassinitialize(TestContext context)
{
var webAppUrl = context.Properties["webAppUrl"].ToString();
//other settings etc..then use your test settings parameters here...
}
Note: these are static so if you need access to this you may need to set up static properties to access within your Test code.
An alternative suggested is using a Data Driven Test approach.
Here's some basic info on data driven tests here which may also help:
https://msdn.microsoft.com/en-us/library/ms182527.aspx
As I said, the above solution suited our needs at a basic level.
UPDATE: see image below in response to test settings returning null...
This works for me (VS2017-pro):
namespace TestApp.Test
{
[TestClass]
public class UnitTest1
{
// This enables the runner to set the TestContext. It gets updated for each test.
public TestContext TestContext { get; set; }
[TestMethod]
public void TestMethod1()
{
// Arrange
String expectedName = "TestMethod1";
String expectedUrl = "http://localhost";
// Act
String actualName = TestContext.TestName;
// The properties are read from the .runsettings file
String actualUrl = TestContext.Properties["webAppUrl"].ToString();
// Assert
Assert.AreEqual(expectedName, actualName);
Assert.AreEqual(expectedUrl, actualUrl);
}
[TestMethod]
public void TestMethod2()
{
// Arrange
String expectedName = "TestMethod2";
// Act
String actualName = TestContext.TestName;
// Assert
Assert.AreEqual(expectedName, actualName);
}
}
}
Make sure to select the runsettings file you wish to use, here: Test -> Test Settings.
An alternative to disable Resharper is to enable MSTest support and select the test setting file on Resharper Options dialog (->Tools->Unit Testing->MsTest).
I was trying to do this exact thing as well. As many of you may know, running tests through MTM exposes some additional properties to the TestContext, including the name of the Run Settings used. I used this property as a "Foreign Key" of sorts for our test data, allowing us to specify environment URLs etc. without hardcoding them or using the incredibly lackluster "Data Driving" tools that come with out of the box testing.
Of course, there's no way to expose any run-time properties when executing tests as part of a BDT or release workflow besides what #kritner is attempting which microsoft describes HERE. However if you read the comments of that link you'll discover what you may be able to infer here:
You need to use VS 2013 R5 or VS 2015 to use this solution
It will only work for Unit Tests!
Those of us who are trying to execute UI or Load tests as part of a CI or CD workflow are completely screwed. You don't get any additional properties in testContext, even when executing a Plan/Suite with certain test configurations (not settings) created in MTM. #Adam may have been able to get this to work when running vs debugging, but that may have only worked with unit tests. Through CodedUI I've been unable to retrieve the properties without getting a NullReferenceException. Here's an example of the janky code I was using to investigate:
if (testContextInstance.Properties["__Tfs_TestConfigurationName__"] != null) //Exposed when run through MTM
{
TFSTestConfigurationName = testContextInstance.Properties["__Tfs_TestConfigurationName__"].ToString();
}
else TFSTestConfigurationName = "Local"; //Local
var configName = testContextInstance.Properties["configurationName"] ?? "No Config Found";
Trace.WriteLine("Property: " + configName);
And the XML of my .runsettings file:
<?xml version="1.0" encoding="utf-8"?>
<RunSettings>
<!-- Parameters used by tests at runtime. These are required as a substitute for TFS/MTM test settings.-->
<!-- File instructions: https://msdn.microsoft.com/en-us/library/jj635153.aspx#example -->
<!-- TFS instructions: https://blogs.msdn.microsoft.com/visualstudioalm/2015/09/04/supplying-run-time-parameters-to-tests/ -->
<TestRunParameters>
<Parameter name="configurationName" value="Local" />
</TestRunParameters>
</RunSettings>
And an excerpt from the .trx produced by the BDT workflow:
Property: No Config Found
For the NullReferenceException issue:
I was also facing the same issue recently and the solution to it is to have the latest update of Visual Studio 2013. Right now the latest update is Update 5. I am not sure which particular update fixes this issue. I applied Update 5 and was able to successfully access the TestRunParameters in the ClassInitialize method.
You can find the updates # https://support.microsoft.com/en-us/kb/2829760
So I had two machines, on one it was all working fine and on the other I was getting the exception. I investigated that the only difference is the Update of VS; applied it and that solved the problem. :)
With NUnit 3, I was able to find the properties in the runsettings file, using
TestContext.Parameters
So in this case it would be:
string entryUrl = TestContext.Parameters["webAppUrl"];
string username = TestContext.Parameters["webAppUserName"];
string password = TestContext.Parameters["webAppPassword"];
I was able to resolve this for Unit tests by disabling Resharper. Wish I could say the same for Coded UI tests.
Why dont you use Application settings?
you can then read them anywhere like
var yesICan= Properties.Settings.Default.IToldYou;
you can create Properties out from them, pretty much you can do a lot.
public string URL_WEBOFFICE
{
get
{
return Properties.Settings.Default.WEBOFFICE_URL.Replace("***", Properties.Settings.Default.SiteName);
}
}
I'm trying to implement some Unit Testing in my MVC Web API ASP.NET project.
I am using VS 2013 (Ultimate, Update 2), and want to use Moq.
I have created a Test project within the solution.
In following along with this blog post, I'm thinking that I should be able to right-click on "GetCountOfPlatypusItemRecords" here:
public class PlatypusItemsController : ApiController
{
private readonly IPlatypusItemRepository _PlatypusItemRepository;
public PlatypusItemsController(IPlatypusItemRepository PlatypusItemRepository)
{
if (PlatypusItemRepository == null)
{
throw new ArgumentNullException("PlatypusItemRepository");
}
_PlatypusItemRepository = PlatypusItemRepository;
}
[Route("api/PlatypusItems/Count")]
public int GetCountOfPlatypusItemRecords()
{
return _PlatypusItemRepository.GetCount();
}
. . .
...and see a "Create Unit Tests..." menu item; but I don't (after "Organize Usings," I see "Generate Sequence Diagram," not "Create Unit Tests..."). Why is "Create Unit Tests..." not available for me?
Note: I am referencing Moq and nunit.framework in both the main project and the test project, and added the corresponding "using NUnit.Framework" and "using Moq"
From the looks of it, the Unit Test Generator is no longer a part of VS 2013. Check out this extension though, which apparently replicates some of those features.
Having said that, I'd advise against generating your unit tests. Most of the time, meaningful tests are the kind where you really have to think through what you are testing, and IMO the generator guides you in the wrong direction. Also, I should point out that Moq is independent of both the testing framework and the way you've written your tests, so you don't really need to generate the tests in order to try out Moq.
I'm currently trying to learn proper unit-test. So now I'm trying to write unit-tests for a class that should map data from an XML-File to the proper objects. Of course all functionality of the class is dependent on the existence of the corresponding XML-file. The XML-file is loaded in the constructor of the class.
I'm using C# with NUnit. So far I've got two tests:
[Test]
public void ShouldAllowInstanceToBeCreatedWhenXMLFileIsPresent()
{
if (File.Exists(SettingsReader.XML_SETTINGS_PATH))
{
SettingsReader settingsReader = new SettingsReader();
Assert.AreNotEqual(null, settingsReader);
}
}
[Test]
[ExpectedException("Telekanzlei.Clientmanager.XMLDataLayer.XMLFileNotFoundException")]
public void ShouldThrowExceptionWhenXMLFileIsNotPresent()
{
if (!File.Exists(SettingsReader.XML_SETTINGS_PATH))
{
SettingsReader settingsReader = new SettingsReader();
}
else
throw new XMLFileNotFoundException();
}
I'm not sure if checking the existence of the file in the test is a proper way to go, so any suggestions on those test are welcome too. But my question is, how to proceed with the following tests. Obviously all following tests are going to fail, if the XML-file is not present.
So do I assume that the XML-file is present, while keeping in mind, that a failing test could just mean that it's not? That wouldn't seem right to me.
Is there a general pattern, to handle a problem like this?
Thx for any help
edit: rewrote the second test, as it was failing if the file was actually present...
edit2: May it is helping to tell you, what the SettingsReader actually does. So far it looks like this:
public class SettingsReader
{
public static readonly string XML_SETTINGS_PATH = "C:\\Telekanzlei\\Clientmanager_2.0\\Settings.xml";
public XElement RootXElement { get; private set; }
public SettingsReader()
{
if (!File.Exists(XML_SETTINGS_PATH))
throw new XMLFileNotFoundException();
using (var fs = File.OpenRead(XML_SETTINGS_PATH))
{
RootXElement = XElement.Load(fs);
}
}
}
I'm not sure, but I guess a StreamReader wouldn't be the way to go here, would it?
The problem is not with your unit tests but with the design of the class. I'd suggest refactoring the class so it doesn't open the file but instead operates on a stream. Then your unit tests could simply replace a file stream for a memory stream - simples! :)
public class SettingsReader()
{
public SettingsReader(System.IO.StreamReader reader)
{
// read contents of stream...
}
}
// In production code:
new SettingsReader(new StreamReader(File.Open("settings.xml")));
// In unit test:
new SettingsReader(new StringReader("<settings>dummy settings</settings>"));
Remember, opening a file and parsing settings data are two very different concerns.
If you must I suggest you use SetUp method to copy or verify that the file exist.
I suggest making sure the file is present by adding it to the test project and marking it as "copy always" once you get that working there is no need to re-check it.
If you have a lot of tests that require external files perhaps you should use MsTest - it has an attribute called DeploymentItem that makes sure that the file is copied to the same location as the test.
Consider rewriting code so dependencies can be passed in or somehow else stubbed for the code you want to unit-test.
I.e. pass something like "IMySettingsFileProvider" instance to SettingsReader constructor where IMySettingsFileProvider.SettingsXml returns some setting stream. This way you can mock IMySettingsFileProvider interface for the test instead of requiring file to be present on disk.
One option is to put this at the top of the test fixture. Then the tests will only be valid when the file exists.
[SetUp]
public void Setup()
{
Assume.That(File.Exists(SettingsReader.XML_SETTINGS_PATH));
}