Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 3 years ago.
Improve this question
I am writing unit tests but the part that confuses me most is whether it should test the functionality or not?
For example, if there is a method which does two things
Deletes files from a folder
Returns whether the folder is empty or not
public bool DeleteFTPFiles(string xyz)
{
...
path = GetFTPPath(xyz);
DeleteFolderFiles(path);
return IsFtpFolderEmpty(path);
}
DeleteFolderFiles - deletes files based on some logic.
Now, if I have to do unit testing for this method(DeleteFTPFiles).
Do I have to create folder structure and add some files through my unit tests as an Arrange test?
Assert whether files are deleted based on the condition?
Also, test if IsFtpFolderEmpty returns true or false based on whether it is empty or not?
If so, how is this different from Integration tests?
For example, if there is a method which does two things
The method you've chosen to write DeleteFTPFiles() is a poor choice because the result does not match the name. If the file is not deleted, the method may still return true? That's faulty logic. If I used that code, I would assume the result is if the file was or wasn't deleted, not if the directory was empty.
If I were to write it, it would just be DeleteAllFiles(), because it doesn't need to know where it happening, just that it is. I would then pass in another class that is has the method necessary to do the work.
public class MySpaceManager()
{
private readonly IFileManager _fileManager;
public MySpaceManager(IFileManager fileManager)
{
_fileManager = fileManager;
}
public bool TryDeleteAllFiles1(logicalDirectory)
{
var files = _fileManager.GetFiles(logicalDirectory);
var result = true;
foreach(var file in files)
result = result && _fileManager.Delete(file);
return result;
}
// or maybe
public bool TryDeleteAllFiles2(logicalDirectory)
{
var files = _fileManager.GetFiles(logicalDirectory);
foreach(var file in files)
_fileManager.Delete(file);
var result = _fileManager.GetFiles(logicalDirectory).Count() == 0;
return result;
}
}
Should unit tests tests the functionality of a method?
Here is my explanation:
A unit-test should only test what it's meant to encapsulate. This may include one or more of the following (not necessarily an exhaustive list):
Runs to Completion
Throws an Exception
Some type of logic (eg. AddTwoNumber() does indeed do that logic)
Executes some external dependency
Does not execute some external dependency
Lets take this hypothetical class and break down what and why each is tested:
public class MySpaceManagerTests
{
// First simple, best good path for code
public void TryDeleteAllFiles2_WithEmptyPath_ThrowsNoException()
{
/// ** ASSIGN **
// I'm using NSubstitute here just for an example
// could use Moq or RhinoMocks, whatever doesn't
// really matter in this instance
// the important part is that we do NOT test dependencies
// the class relies on.
var fileManager = Substitute.For<IFileManager>();
fileManager
.GetFiles(Args.Any<string>())
.Returns(new List<IFile>());
var mySpaceManager = new MySpaceManager(fileManager);
// ** ACT && ASSERT**
// we know that the argument doesn't matter so we don't need it to be
// anything at all, we just want to make sure that it runs to completion
Asser.DoesNotThrow(() => mySpaceManager.TryDeleteAllFiles2(string.Empty);
}
// This looks VERY similar to the first test but
// because the assert is different we need to write a different
// test. Each test should only really assert the name of the test
// as it makes it easier to debug and fix it when it only tests
// one thing.
public void TryDeleteAllFiles2_WithEmptyPath_CallsFileManagerGetFiles()
{
/// ** ASSIGN **
var fileManager = Substitute.For<IFileManager>();
fileManager
.GetFiles(Args.Any<string>())
.Returns(new List<IFile>());
var mySpaceManager = new MySpaceManager(fileManager);
// ** ACT **
mySpaceManager.TryDeleteAllFiles2(string.Empty)
// ** ASSERT **
Assert.DoesNotThrow(fileManager.Received().GetFiles());
}
public void TryDeleteAllFiles2_With0Files_DoesNotCallDeleteFile
{
/// ** ASSIGN **
var fileManager = Substitute.For<IFileManager>();
fileManager
.GetFiles(Args.Any<string>())
.Returns(new List<IFile> { Substitute.For<IFile>(); });
var mySpaceManager = new MySpaceManager(fileManager);
// ** ACT **
mySpaceManager.TryDeleteAllFiles2(string.Empty)
// ** ASSERT **
Assert.DoesNotThrow(fileManager.DidNotReceive().GetFiles());
}
public void TryDeleteAllFiles2_With1File_CallsFileManagerDeleteFile
{
// etc
}
public void TryDeleteAllFiles2_With1FileDeleted_ReturnsTrue()
{
/// ** ASSIGN **
var fileManager = Substitute.For<IFileManager>();
fileManager
.GetFiles(Args.Any<string>())
.Returns(new List<IFile> { Substitute.For<IFile>(); },
new list<IFile>());
var mySpaceManager = new MySpaceManager(fileManager);
// ** ACT **
var actual = mySpaceManager.TryDeleteAllFiles2(string.Empty)
// ** ASSERT **
Assert.That(actual, Is.True);
}
public void TryDeleteAllFiles2_With1FileNotDeleted_ReturnsFalse()
{
/// ** ASSIGN **
var fileManager = Substitute.For<IFileManager>();
fileManager
.GetFiles(Args.Any<string>())
.Returns(new List<IFile> { Substitute.For<IFile>(); },
new List<IFile> { Substitute.For<IFile>(); });
var mySpaceManager = new MySpaceManager(fileManager);
// ** ACT **
var actual = mySpaceManager.TryDeleteAllFiles2(string.Empty)
// ** ASSERT **
Assert.That(actual, Is.False);
}
}
Unit-test may test this code but it should be written a little another way.
Looking at this code rather make sense to talk about integration tests not unit-tests.
To have ability to write unit-test need to decouple your code from concrete implementations. You want test your code not FTP-service, is'n it?
To make code testable need to refactor your code following these steps:
Introduce the IFileStorage-abstraction:
public interface IFileStorage
{
string GetPath(string smth);
void DeleteFolder(string name);
bool IsFolderEmpty(string path);
}
public sealed class FtpFileStorage : IFileStorage
{
public string GetPath(string smth) { throw new NotImplementedException(); }
public void DeleteFolder(string name) { throw new NotImplementedException(); }
public bool IsFolderEmpty(string path) { throw new NotImplementedException(); }
}
Code should depend from abstractions not concrete implementations:
public class SmthLikeServiceOrManager
{
private readonly IFileStorage _fileStorage;
public SmthLikeServiceOrManager(IFileStorage fileStorage)
{
_fileStorage = fileStorage;
}
public bool DeleteFiles(string xyz)
{
// ...
var path = _fileStorage.GetPath(xyz);
_fileStorage.DeleteFolder(path);
return _fileStorage.IsFolderEmpty(path);
}
}
Now you can write real unit-test using one of the mocking libraries such as
Moq
NSubstitute
Related articles on StackOverflow:
Mocking using Moq in c#
..
Related
I am not sure about how such a pattern named or even if it exists, but I named it 'container pattern'.
What I am trying to accomplish: to have an abstraction to hold a list of entities, being able only to add entities, and remove them only when entities saved to the database. I must say it works quite well and I like it much more than passing around List<> like I did earlier.
I just learned that testing private fields is big no-no, but I don't know how I can test Add method alone. Or how to test SaveAndClean without invoking Add. So far testing private field using additional constructor seem clean, but probably there are better solutions.
namespace test
{
class Container
{
private readonly List<Entity> _results;
private readonly IRepostory _repo;
// used for prod
public Container(IRepostory repo)
: this(new List<Entity>(500000), repo)
{
}
// used for tests
internal Container(List<Entity> results, IRepostory repo)
{
_results = results;
_repo = repo;
}
public void Add(Entity entity)
{
_results.Add(entity);
}
public async Task<bool> SaveAndClearAsync()
{
if (!_results.Any())
{
return true;
}
try
{
await _repo.SaveAsync(_results);
_results.Clear();
return true;
}
catch (Exception ex)
{
// logging
}
return false;
}
}
}
[Fact]
public void Add_AddToExisting_EntityAdded()
{
// Arrange
var results = new List<Entity>();
results.Add(new Entity { Name = "md51" });
var repo = new Mock<IRepository>(MockBehavior.Strict);
var service = new Container(results, repo.Object);
var newEntity = new Entity { Name "md52" };
// Act
service.Add(newEntity);
// Assert
Assert.Equal("md51", results[0].Name);
Assert.Equal("md52", results[1].Name);
Assert.Equal(2, results.Count);
}
In your case I would test the behavior as a black box. And from a black box perspective only calling Add doesn't produce any behavior so I'd leave it at that. But calling Add() 2 times and SaveAndClearAsync does, so just test that.
You shouldn't change your code interface for the sole purpose of testing. That's an anti-pattern as well.
I recommend this Dave Farley video on test mistakes.
I have a large set of integration tests that test a website server. Most of these tests are fine to run in parallel. However, I have a few that change settings and can cause each other to fail when run in parallel.
As a simplified example, let's say I had these tests:
TestPrice_5PercentTax
TestPrice_10PercentTax
TestPrice_NoTax
TestInventory_Add10Items
TestInventory_Remove10Items
The inventory tests will not get in the way of each other, and are not affected by the price tests. But the price tests will change the Tax setting, so that if both 5 and 10 run in parallel, 10 could end up changing the setting before 5 is done, and 5 would fail because it saw 10% tax instead of the 5% it expected.
I want to define a category for the three price tests, and say that they may not run at the same time as one another. They can run at the same time as any other tests, just not the other price tests. Is there a way to do this in MSTest?
MsTest v2 has functionality as following
[assembly: Parallelize(Workers = 0, Scope = ExecutionScope.MethodLevel)]
// Notice the assembly bracket, this can be compatible or incompatible with how your code is built
namespace UnitTestProject1
{
[TestClass]
public class TestClass1
{
[TestMethod]
[DoNotParallelize] // This test will not be run in parallel
public void TestPrice_5PercentTax() => //YourTestHere?;
[TestMethod]
[DoNotParallelize] // This test will not be run in parallel
public void TestPrice_10PercentTax() => //YourTestHere?;
[TestMethod]
[DoNotParallelize] // This test will not be run in parallel
public void TestPrice_NoTax() => //YourTestHere?;
[TestMethod]
public void TestInventory_Add10Items() => //YourTestHere?;
[TestMethod]
public void TestInventory_Remove10Items() => //YourTestHere?;
}
}
More detailed information can be found here MSTest v2 at meziantou.net
I strongly recommend atleast a quick read through of the link, as this will likely help you solve and understand the issue with the tests run in parallel or sequential.
I would like to provide a potential solution that I started but did not pursue.
First, I made a class that I could use as an attribute on my test methods.
[AttributeUsage(AttributeTargets.Method, Inherited = true, AllowMultiple =true)]
public class NoParallel : Attribute
{
public NoParallel(string nonParallelGroupName)
{
SetName = nonParallelGroupName;
}
public string SetName { get; }
}
Then I went and added it to my test methods that will conflict.
[NoParallel("Tax")]
public void TestPrice_5PercentTax();
[NoParallel("Tax")]
public void TestPrice_10PercentTax();
[NoParallel("Tax")]
public void TestPrice_NoTax();
// This test doesn't care
public void TestInventory_Add10Items();
// This test doesn't care
public void TestInventory_Remove10Items();
I gave my test class a static dictionary of mutexes keyed by their names.
private static Dictionary<string, Mutex> exclusiveCategories = new Dictionary<string, Mutex>();
Finally, using a helper to grab all of the "NoParallel" strings the test method has...
public static List<string> NonparallelSets(this TestContext context, ContextHandler testInstance)
{
var result = new List<string>();
var testName = context.TestName;
var testClassType = testInstance.GetType();
var testMethod = testClassType.GetMethod(testName);
if (testMethod != null)
{
var nonParallelGroup = testMethod.GetCustomAttribute<NoParallel>(true);
if (nonParallelGroup != null)
{
result = nonParallelGroups.Select(x => x.SetName).ToList();
}
}
result.Sort();
return result;
}
... I set up a TestInitialize and TestCleanup to make the tests with matching NoParallel strings execute in order.
[TestInitialize]
public void PerformSetup()
{
// Get all "NoParallel" strings on the test method currently being run
var nonParallelSets = testContext.NonparallelSets(this);
// A test can have multiple "NoParallel" attributes so do this for all of them
foreach (var setName in nonParallelSets)
{
// If this NoParallel set doesn't have a mutex yet, make one
Mutex mutex;
if (exclusiveCategories.ContainsKey(setName))
{
mutex = exclusiveCategories[setName];
}
else
{
mutex = new System.Threading.Mutex();
exclusiveCategories[setName] = mutex;
}
// Wait for the mutex before you can run the test
mutex.WaitOne();
}
}
[TestCleanup]
public void PerformTeardown()
{
// Get the "NoParallel" strings on the test method again
var nonParallelSets = testContext.NonparallelSets(this);
// Release the mutex held for each one
foreach (var setName in nonParallelSets)
{
var mutex = exclusiveCategories[setName];
mutex.ReleaseMutex();
}
}
We decided not to pursue this because it wasn't really worth the effort. Ultimately we decided to pull the tests that can't run together into their own test class, and mark them with [DoNotParallelize] as H.N suggested.
I have a project that supports multiple deployments mode: InMem, OnPremise, Cloud.
Also each projects have small services like TimeDistance which can be conected either to WCF, either to an API.
In the unitTestMockup i can say which one i want to use:
Service.TimeDistance = new WCFTimeDistance() / new APITimeDistance().
Until now i had only WCFTimeDistance but now we are in transition mode to move to APITimeDistance but in the meantime i want when i run the tests to run twice, once with WCF once with API.
What's a good approach to do this?
I use C# 4.5
Microsoft.VisualStudio.QualityTools.UnitTestFramework as framework for unitTests
A simple example of desired workflow would be this:
1)Mockup: Service.TimeDistance = new WCFTimeDistance();
2)UnitTest: CheckDistanceBetweenTwoLocationsTest()
{
Service.TimeDistance.CalculateDistance(Location1, Location2) // WCFTimeDistance
}
3)Mockup: Service.TimeDistance = new APITimeDistance();
UnitTest: CheckDistanceBetweenTwoLocationsTest()
{
4)Service.TimeDistance.CalculateDistance(Location1, Location2) // APITimeDistance
}
Create two unit tests. Also look into using abstractions instead of static classes. It would make mocking and testing easier.
This is for your current setup
[TestClass]
public class TimeDistanceTests {
//change these to the needed types
object Location1;
object Location2;
double expected;
[TestInitialize]
public void Init() {
//...setup the two locations and expectations
Location1 = //...
Location2 = //...
expected = //...the distance.
}
[TestMethod]
public void Check_Distance_Between_Two_Locations_Using_WCF() {
//Arrange
var sut = new WCFTimeDistance();
//Act
var actual = sut.CalculateDistance(Location1, Location2);
//Assert
//...some assertion proving that test passes or failed
Assert.AreEqual(expected, actual);
}
[TestMethod]
public void Check_Distance_Between_Two_Locations_Using_API() {
//Arrange
var sut = new APITimeDistance();
//Act
var actual = sut.CalculateDistance(Location1, Location2);
//Assert
//...some assertion proving that test passes or failed
Assert.AreEqual(expected, actual);
}
}
Ideally you would want to abstract the service. Assuming something like
public interface ITimeDistance {
double CalculateDistance(Location location1, Location location2);
}
Your Service would use the abstraction instead of a concretion.
public class Service {
ITimeDistance timeDistance
public Service(ITimeDistance timeDistance) {
this.timeDistance = timeDistance;
}
public ITimeDistance TimeDistance { get { return timeDistance; } }
}
So now in your unit tests you can swap the different implementations of your time distance service.
Here is a simplified version of a problem I encountered:
public interface IService
{
IProvider Provider { get; }
}
public interface IProvider
{
List<int> Numbers{ get; }
string Text { get; }
}
[TestMethod]
public void ServiceTest()
{
var service = new Mock<IService>();
var provider = new Mock<IProvider>();
service.Setup(s => s.Provider).Returns(provider.Object); // A
service.Setup(s => s.Provider.Text).Returns("some text"); // B - incorrect
// they actually meant to do this, instead of 'B'
// provider.Setup(p => p.Text).Returns("some text");
provider.Setup(p => p.Numbers).Returns(new List<int> { 1, 2, 3 });
DummyApplicationCode(service.Object);
}
int DummyApplicationCode(IService service)
{
// will throw, because the Provider was replaced at 'B'
int shouldBeOne = service.Provider.Numbers.First();
return shouldBeOne;
}
A unit test was failing because way down in the application code under test, the mocked IService was returning the wrong IProvider.
I eventually spotted the line (bear in mind the code I was looking at was not as simple as above) which had caused it, labelled 'B' above, which someone else had added due to misunderstanding the Moq Setup.
I'm aware that subsequent Setups on a mock will override previous ones but I hadn't spotted this issue because the Return of the offending line was for a separate sub-property.
I expect this is by design but it threw me as I hadn't anticipated someone would do this.
My question: Since the Setup at 'B' is only concerned with the return of the provider Text, why does the service 'Provider' property need to replace that which was defined at 'A'?
This is clearly intentional when looking at the source:
https://github.com/moq/moq4/blob/master/Source/Mock.cs
https://github.com/moq/moq4/blob/master/Source/Interceptor.cs
Setup creates a "call" by using AddCall on Interceptor. This contains the following block of code which, as long as we're creating a non-conditional setup, removes all previous setups. It's even commented.
if (!call.IsConditional)
{
lock (calls)
{
// if it's not a conditional call, we do
// all the override setups.
// TODO maybe add the conditionals to other
// record like calls to be user friendly and display
// somethig like: non of this calls were performed.
if (calls.ContainsKey(key))
{
// Remove previous from ordered calls
InterceptionContext.RemoveOrderedCall(calls[key]);
}
calls[key] = call;
}
I'm just starting out with BDD/TDD using MSpec (with AutoMocking by James Broome) and RhinoMocks. Here's an excerpt from my practice project:
namespace Tests.VideoStore.Controllers
{
public abstract class context_for_movie_controller :
Specification<MovieController>
{
private static IList<Movie> movies;
protected static IMovieRepository _movieRepository;
protected static ActionResult _result;
protected static string title;
protected static string director;
Establish context = () =>
{
_movieRepository = DependencyOf<IMovieRepository>();
};
}
[Subject(typeof(MovieController))]
public class when_searching_for_movies_with_director :
context_for_movie_controller
{
Establish context = () =>
{
title = null;
director = "James Cameron";
var movie4 = new Movie {
Title = "Terminator", Director = "James Cameron"};
var movie6 = new Movie {
Title = "Avatar", Director = "James Cameron"};
movies = new List<Movie> {movie4, movie6};
// Repository returns all movies.
_movieRepository.Stub(x => x.FindMovies(title, director))
.Return(movies);
};
Because of = () => _result = subject.Find(title, director);
It should_fetch_movies_from_the_repository = () =>
_movieRepository.AssertWasCalled(x =>
x.FindMovies(title, director));
It should_return_a_list_of_movies_matching_the_director = () =>
_result.ShouldBeAView().And()
.ShouldHaveModelOfType<IEnumerable<Movie>>)
.And().ShouldContainOnly(movies);
}
As you can see, I stubbed out the FindMovies() method on the MovieRepository class. Then I'm calling the MoviesController.Find() action. My question is, should there be an assert to check if the stubbed method (FindMovies) was called by the controller? Or perhaps should I only care about the returned result and not where it was taken from? Furthermore, a spec that says "should_fetch_movies_from_the_repository" looks a lot like an engineering task, not something that a client might understand - does it have its place in BDD?
the general rule to follow for assertions is that you assert against output interactions, not input interactions.
the FindMovies stub is returning a "movies" collection to the class that called it and you are then verifying that the class received the correct list via the "it should return a list of movies matching the director" assertion. if the FindMovies method is not called, then this assertion will fail.
therefore, you do not need to assert the calls against the FindMovies method.
to counterpoint this, if you have an mock or stub that is purely output - let's say an IView interface being called by a Presenter class, then you do want to assert against the IView being called. for example, this code:
public class MyPresenter
{
... other code here
public DoSomething()
{
IList data = GetSomeData();
myView.DisplayData(data);
}
}
you would want to assert that the view.DisplayData method is called in this case, because you are not retrieving anything from this call that can be asserted by another test.
as for the "fetch from repository" - of course your customers care about that. they want the system to save movies to the storage and load them from the storage. however ... the FindMovies call is an input into the class being tested, so it's not necessary to have this assetion or test, at all. if the FindMovies method is not called, then the other test will fail and let you know that there is a problem.