Is there a way or approach like a method decorator or attribute for a test method that can say for example:
"Run Method C Before running Method B"
So basically you are creating a dependancy between C and B. I know tests are better off being atomic and should be but sometimes in it's better to keep your tests small and to the point. It makes sense not run a 'RemoveItem' test method when the item it is looking for is simply not there.
Most people would add the item before hand and then test to see if they can remove - 'All In the same test'. I don't like this approach and want to make my tests smaller, more to to point and more atomic as possible.
Like you said, you dont want interdependencies between your test. If you are not comfortable having an "Add" before the "Remove" in your remove test, thus testing the Add method in the wrong place, then I recommend using testInitialize to setup some objects the tests can act on. I do however recommend the practice of actually running Add before you run Remove, in the test of Remove.
[Testclass]
public class TestStacks
{
private Stack<string> emptyStack;
private Stack<string> singleItemStack;
[TestInitialize]
public void Setup()
{
singleItemStack = new Stack<string>();
singleItemStack.Push("Item");
emptyStack = new Stack<string>();
}
[TestMethod]
public void TestPush()
{
emptyStack.Push("Added");
Assert.AreEqual(1, emptyStack.Count);
}
[TestMethod]
public void TestRemove()
{
singleItemStack.Pop();
Assert.AreEqual(0, singleItemStack.Count);
}
[TestMethod]
[ExpectedException(typeof(InvalidOperationException))]
public void TestPopFromEmpty()
{
emptyStack.Pop();
}
}
If you need to have some item added before testing removal, then best place to add item is arrange part of removal test. This will make context of removal test clear.
But DRY principle also works here - you can move addition logic to separate helper method. Then call it twice - when testing addition, and when arranging context for removal:
[Testclass]
public class Tests
{
[TestMethod]
public void TestAddition()
{
AddItem();
// Assert addition
}
[TestMethod]
public void TestRemoval()
{
AddItem();
// Remove item
// Assert removal
}
private void AddItem()
{
// Add item
}
}
Related
I have a logger class, which purpose is to be called from whatever class in my solution, who decides to log something.
I added an interface, which is why I applied a singleton pattern, and didn't use a static class.
My LogManager implementation (singleton):
https://pastebin.com/NHKmbj9c
I wanted to write simple unit tests, which are supposed to use local variables, testing the functionality of each ILogger methods, but as soon as my first Unit has passed, the Singleton will stay initialized in context, making subsequent unit tests to fail (while they are trying to Initialize the singleton...).
Unit Test:
[TestClass]
public class LogManagerTests
{
[TestMethod]
public void Error_ExpectedErrorLevel_ShouldBe_Error()
{
// Arrange
var actualLevel = ErrorLevel.Warning;
const ErrorLevel expectedLevel = ErrorLevel.Error;
var iLogger = LogManager.GetInstance;
iLogger.Initialize((level, msg) => { actualLevel = level; }, null);
// Act
iLogger.Error(new Exception(), string.Empty);
// Assert
Assert.AreEqual(expectedLevel, actualLevel);
}
[TestMethod]
public void Debug_ExpectedErrorLevel_ShouldBe_Verbose()
{
// Arrange
var actualLevel = ErrorLevel.Warning;
const ErrorLevel expectedLevel = ErrorLevel.Verbose;
var iLogger = LogManager.GetInstance;
iLogger.Initialize(null, (level, msg, ex) => { actualLevel = level; });
// Act
iLogger.Debug(string.Empty);
// Assert
Assert.AreEqual(expectedLevel, actualLevel);
}
}
Another tought is to initialize the LogManager as a private global variable within my TestClass, but this could give race conditions if the Unit test runs async, as multiple methods then will access the same output variable, which may override each others.
Is it possible to UnitTest a singleton in any way?
The design does not allow me to refactor the LogManager, and remove the singleton pattern from it.
It's possible to unit test a singleton, you just need to think about it differently. Don't try so hard to change your methodology to fit the test. Think about creating a method that is used only for testing, LogManager.Uninitialize().
Call this after every test in this group of tests to ensure your singleton is set back to a testable state.
[TestCleanup()]
public void Cleanup()
{
LogManager.Uninitialize();
}
It may not be pure but I think it's fine to write in a diagnostics method every once in a while. It's better than having bad test coverage where you need good test coverage.
hopefully it's pretty simple but I'm stuck trying to figure this out.
If I have a simple class with a method which is supposed to do something and then show a messagebox via MessageBox.Show("") when some values are in a certain state, how do I write a test/tests that can detect if a the messagebox was shown when expected and/or not shown when not expected?
You can't really test that though Unit tests. There is an easy way to test if the code was called correctly though.
I would hide showing the MessageBox behind an interface. Then mock that interface and create a counter or something like that in your mock implementation. Of course you can make this as specific as you need, providing the message shown, etc.
You shouldn't test that a MesssageBox appears, though it is possible to do so with one or another UI automation framework (like https://msdn.microsoft.com/en-us/library/ms747327(v=vs.110).aspx).
But it won't be a unittest. It would be an automated UI test.
So, if you want to create a proper unittest, then your class should be injectable with some
public interface IMessageBox
{
void Show(String message);
}
public class SUT
{
public SUT(IMessageBox messageBox)
{
this._messageBox = messageBox;
}
public void Test()
{
this._messageBox.Show("TEST);
}
}
So that you can mock that IMessageBox inside the unittest.
For example, with Moq:
[TestMethod]
public void Test()
{
// Arrange
var messageBox = new Mock<IMessageBox>();
messageBox
.Setup(m =>
m.Show("TEST))
.Verifiable();
var sut = new SUT(messageBox.Object);
// Act
sut.Test();
// Verify
messageBox.Verify();
}
This question already has answers here:
How can we run a test method with multiple parameters in MSTest?
(9 answers)
Closed 6 years ago.
Using MSTest I want to run a test like this ...
var range = Enumerable.Range(0, 9);
foreach(var i in range)
{
Test(i);
}
... one theory I had was to create a new Test attribute like this ...
[TestClass]
public class CubeTests
{
[TestMethod]
[TestRange(0, 9)]
public void Test(int i)
{
// Test stuff
}
}
...
The key here is that I have some quite memory intensive code that I would like MSTest to clean up between tests for me.
For something so simple I really don't want to be relying on files and using Datasource and Deployment items.
Can this be done, if so, is anyone prepared to offer up an idea of how?
Maybe that's what you're looking for. Some years ago, Microsoft made available an extension for visual studio called PEX.
PEX generate unit tests from a single parametric test, Pex finds
interesting input-output values of your methods, which you can save as
a small test suite with high code coverage.
You can use assumption and precondition for the parameters of your test, which ensure the best control of the tests generation.
Pex is no longer available(it was a research project), but is now available instead Intellitest, which still uses the same static analysis engine.
Intellitest generates a parameterized test that is modifiable and
general/global assertions can be added there. It also generates the
minimum number of inputs that maximize the code coverage; stores the
inputs as individual unit tests, each one calling the parameterized
test with a crafted input.
[PexMethod]
public void Test(int i)
{
PexAssume.IsTrue(i >= 0);
PexAssume.IsTrue(i < 10);
// Test stuff
}
You dont have to resort to built-in test runner magic. Simply add your range as a property of your test class:
private static IEnumerable<int> TestRange
{
get
{
int i = 0;
while(i < 10)
yield return i++;
}
}
now in your testmethod, you can do the for-loop as usual, using your uniquely defined testrange:
[TestMethod]
public void DoStuff_RangeIsValid_NoExceptions(){
// Act
foreach(var i in TestRange){
// do the unit test here
}
}
Implement test in separate method and call method from Test method
I think you can do this in NUnit, but I am pretty sure you can't do it in MS test.
If you want to do clean up then you can call the GC after every call, or create a TestCleanUpImpl method ( did this in snippet calling GC.Collect() to show how to force GC ).
Would suggest something like the following:
public void TestSetup()
{
//Setup tests
}
public void TestCleanUpImpl()
{
//unassign variables
//dispose disposable object
GC.Collect();
}
public void TestImpl(int i)
{
// Test stuff
// Do assert statements here
}
[TestMethod]
public void Test()
{
int fromNum = 0;
int untilNum = 9;
for(int i=fromNum;i<=untilNum;i++)
{
TestSetup();
TestImpl(i);
TestCleanUpImpl();
}
}
If you have complicated setup and clean up could possibly implement a class that handles disposing and creating, handle setup in constructor, disposal in Dispose method
I wouldn't use this as my first choice, prefer to keep my tests as simple as possible, even if my tests do violate DRY it makes them much easier to follow, which means less debugging, which is a good trade off in my opinion
public class TestImplObj : IDisposable
{
public TestImplObj()
{
//Setup test
}
public void TestImpl(int i)
{
//Do the actual test
}
public void Dispose()
{
Dispose(true);
GC.SuppressFinalize(this);
}
protected virtual void Dispose(bool disposing)
{
if (disposing)
{
// Do the clean up here
}
}
}
Background
I need some class to perform background initialization, which should start in constructor. Currently I'm using a Task which is started by constructor, and then all operations, depending on that initialization wait for that Task completion.
Please take a look at the following simplified example:
interface IEntry {}
interface IRepository
{
IQueryable<IEntry> Query { get; }
void Add(IEntry entry);
}
class PrefetchedRepository : IRepository
{
private readonly Task _prefetchingTask;
private readonly ICollection<IEntry> _entries = new List<IEntry>();
private readonly IRepository _underlyingRepository;
public PrefetchedRepository(IRepository underlyingRepository)
{
_underlyingRepository = underlyingRepository;
// Background initialization starts here
_prefetchingTask = Task.Factory.StartNew(Prefetch);
}
public IQueryable<IEntry> Query
{
get
{
EnsurePrefetchCompleted();
return _entries.AsQueryable();
}
}
public void Add(IEntry entry)
{
EnsurePrefetchCompleted();
_entries.Add(entry);
_underlyingRepository.Add(entry);
}
private void EnsurePrefetchCompleted()
{
_prefetchingTask.Wait();
}
private void Prefetch()
{
foreach (var entry in _underlyingRepository.Query)
{
_entries.Add(entry);
}
}
}
This works. The problem starts when I want to test initialization in Unit Test. I'm creating the instance and providing the mock of underlying repository. I want to ensure that all entries were fetched from the mock as expected.
[TestFixture]
public class PrefetchingRepositoryTests
{
[Test]
public void WhenInitialized_PrefetchingIsDone()
{
// Arrange
var underlyingRepositoryMock = A.Fake<IRepository>();
// Act
var target = new PrefetchedRepository(_underlyingRepository);
// Assert
underlyingRepositoryMock.CallsTo(r => r.Query).MustHaveHappened(Repeated.Exactly(1));
}
}
As you can imagine, most of the time fails, because actually initialization didn't started yet at the assertion point.
Questions
Question 1 - Initialization: Is there more elegant way of asynchronous initialization rather than starting task in constructor and waiting for it in all dependent operations?
Question 2 - Testing: I thought of 2 possible ways to solve race between the test and testee:
Using event handle to the test:
[Test]
public void WhenInitialized_PrefetchingIsDone()
{
// Arrange ...
var invokedEvent = new ManualResetEvent(false);
underlyingRepositoryMock.CallsTo(r => r.Query).Invokes(_ => invokedEvent.Set());
// Act ...
// Assert
Assert.True(invokedEvent.WaitOne(1000));
}
Exposing EnsurePrefetchCompleted method as internal and using it in the Unit Test (assuming usage of [assembly: InternalsVisibleTo("...")])
The problem with both solutions is that in case of failure time duration long (actually in the second case - it is limited by test timeout).
Is there any simpler way to do this kind of testing?
Extract the prefetch logic into a separate Prefetcher class and when testing mock the Prefetcher with something that does the fetching without the use of a separate thread.
This will allow you to do the white-box testing of your PrefetchedRepository which I see you are attempting to do with
underlyingRepositoryMock.CallsTo(r => r.Query).MustHaveHappened(Repeated.Exactly(1)); (I would never do white-box testing, but that's just me.)
Once you are done with your white box testing, you can then do black-box testing of your PrefetchedRepository, without concern as to how it works internally. (Whether it invokes other objects to do its job, how many times it invokes them, etc.) Therefore, your testing code will not need to guess the point in time when it is okay to check whether query has been invoked, because it will not be concerned at all with whether query was invoked or not. Essentially, your testing code will be testing against interface IRepository, not against class PrefetchedRepository.
Don't expose a instance which is in invalid state. Client code may often notice a delay when calling any members in PrefetchedRepository just because the underlying repository is slow. You're trying to be clever by hiding these details by hiding all the grotty waiting logic inside EnsurePrefetchCompleted which client code doesn't even know. But this may surprise the client why does even this takes a lot of time??
Better approach is to expose the Task in the public surface of the API and let the client code await it before it does anything with the repository instance.
Something like this:
class PrefetchedRepository : IRepository
{
private readonly Task _prefetchingTask;
private readonly ICollection<IEntry> _entries = new List<IEntry>();
private readonly IRepository _underlyingRepository;
public PrefetchedRepository(IRepository underlyingRepository)
{
_underlyingRepository = underlyingRepository;
// Background initialization starts here
_prefetchingTask = Task.Factory.StartNew(Prefetch);
}
public Task Initialization
{
get
{
return _prefetchingTask;
}
}
...
}
Then you could do
var repo = new PrefetchedRepository(someOtherSlowRepo);
await repo.Initialization;
//Then do whatever with the repo.
Of course delete that EnsurePrefetchCompleted method and all calls to it.
But I do understand this introduces the smell so-called Temporal Coupling.
Better design is to introduce a Factory which does this for you.
public class PrefetchedRepositoryFactory
{
public Task<IRepository> CreateAsync()
{
//someOtherSlowRepo can be a parameter or instance field of this class
var repo = new PrefetchedRepository(someOtherSlowRepo);
await repo.Initialization;
return repo;
}
}
Then you could simply do
var repo = await prefetchedRepositoryFactory.CreateAsync();
//Do whatever with repo.
If you do so, you don't have to take any special care for testing as you'll always have the fully constructed repository in hand.
You can await inside the Test methods; Most of the major unittesting frameworks supports async Task returning methods.
This is my first attempt to do unit tests, so please be patient with me.
I'm still trying to unit test a library that converts lists of POCOs to ADO.Recordsets.
Right now, I'm trying to write a test that creates a List<Poco>, converts it into a Recordset (using the method I want to test) and then checks if they contain the same information (like, if Poco.Foo == RS.Foo, and so on...).
This is the POCO:
public class TestPoco
{
public string StringValue { get; set; }
public int Int32Value { get; set; }
public bool BoolValue { get; set; }
}
...and this is the test so far (I'm using xUnit.net):
[Fact]
public void TheTest()
{
var input = new List<TestPoco>();
input.Add(new TestPoco { BoolValue = true, Int32Value = 1, StringValue = "foo" });
var actual = input.ToRecordset();
Assert.Equal(actual.BoolValue, true);
Assert.Equal(actual.Int32Value, 1);
Assert.Equal(actual.StringValue, "foo");
}
What I don't like about this are the three asserts at the end, one per property of the POCO.
I've read lots of times that multiple asserts in one test are evil (and I understand the reasons why, and I agree).
The problem is, how can I get rid of them?
I have Roy Osherove's excellent book "The Art of Unit Testing" right in front of me, and he has an example which covers exactly this (for those who have the book: chapter 7.2.6, page 202/203):
In his example, the method under test returns an AnalyzedOutput object with several properties, and he wants to assert all the properties to check if each one contains the expected value.
The solution in this case:
Create another AnalyzedOutput instance, fill it with the expected values and assert if it's equal to the one returned by the method under test (and override Equals() to be able to do this).
But I think I can't do this in my case, because the method that I want to test returns an ADODB.Recordset.
And in order to create another Recordset with the expected values, I would first need to create it completely from scratch:
// this probably doesn't actually compile, the actual conversion method
// doesn't exist yet and this is just to show the idea
var expected = new ADODB.RecordsetClass();
expected.Fields.Append("BoolValue", ADODB.DataTypeEnum.adBoolean);
expected.Fields.Append("Int32Value", ADODB.DataTypeEnum.adInteger);
expected.Fields.Append("StringValue", ADODB.DataTypeEnum.adVarWChar);
expected.AddNew();
expected.BoolValue = true;
expected.Int32Value = 1;
expected.StringValue = "foo";
expected.Update();
I don't like this either, because this is basically a duplication of some of the code in the actual conversion method (the method under test), which is another thing to avoid in tests.
So...what can I do now?
Is this level of duplication still acceptable in this special situation, or is there a better way how to test this?
I'd argue that in the spirit of the thing, this is fine. The reason that multiple asserts are "evil", if I recall correctly, is that it implies that you are testing multiple things in one test. In this case, you are indeed doing that in that you are testing each field, presumably to make sure this works for several different types. Since that's all an object equality test would do anyway, I think you are in the clear.
If you really wanted to be militant about it, write one test per property (j/k!)
Multiple assertions per unit test are perfectly fine in my book, as long as the multiple assertions are all asserting the same test condition. In your case, they're testing that the conversion was successful, so the test passing is conditional on all of those assertions being true. As a result, it's perfectly fine!
I'd classify "one assertion per test" as a guideline, not a hard-and-fast rule. When you disregard it, consider why you're disregarding it.
That said, a way around it is to create a single test class that, on class setup, runs your test process. Then each test is just an assertion on a single property. For example:
public class ClassWithProperities
{
public string Foo { get; set; }
public int Bar { get; set; }
}
public static class Converter
{
public static ClassWithProperities Convert(string foo, int bar)
{
return new ClassWithProperities {Foo=foo, Bar=bar};
}
}
[TestClass]
public class PropertyTestsWhenFooIsTestAndBarIsOne
{
private static ClassWithProperities classWithProperties;
[ClassInitialize]
public static void ClassInit(TestContext testContext)
{
//Arrange
string foo = "test";
int bar = 1;
//Act
classWithProperties = Converter.Convert(foo, bar);
//Assert
}
[TestMethod]
public void AssertFooIsTest()
{
Assert.AreEqual("test", classWithProperties.Foo);
}
[TestMethod]
public void AssertBarIsOne()
{
Assert.AreEqual(1, classWithProperties.Bar);
}
}
[TestClass]
public class PropertyTestsWhenFooIsXyzAndBarIsTwoThousand
{
private static ClassWithProperities classWithProperties;
[ClassInitialize]
public static void ClassInit(TestContext testContext)
{
//Arrange
string foo = "Xyz";
int bar = 2000;
//Act
classWithProperties = Converter.Convert(foo, bar);
//Assert
}
[TestMethod]
public void AssertFooIsXyz()
{
Assert.AreEqual("Xyz", classWithProperties.Foo);
}
[TestMethod]
public void AssertBarIsTwoThousand()
{
Assert.AreEqual(2000, classWithProperties.Bar);
}
}
I agree with all the other comments that it is fine to do so, if you are logically testing one thing.
There is however a difference between have many assertions in a single unit test than having a separate unit test for each property. I call it 'Blocking Assertions' (Probably a better name out there). If you have many assertions in one test then you are only going to know about a failure in the first property that failed the assertion. If you have say 10 properties and 5 of them returned incorrect results then you will have to go through fixing the first one, re-run the test and notice another one failed, then fix that etc.
Depending on how you look at it this could be quite frustrating. On the flip side having 5 simple unit tests failing suddenly could also be off putting, but it might give you a clearer picture as to what have caused those to fail all at once and possibly direct you more quickly to a known fix (perhaps).
I would say if you need to test multiple properties keep the number down (possibly under 5) to avoid the blocking assertion issue getting out of control. If there are a ton of properties to test then perhaps it is a sign that your model is representing too much or perhaps you can look at grouping properties into multiple tests.
Those 3 asserts are valid. If you used a framework more like mspec, it would look like:
public class When_converting_a_TestPoco_to_Recordset
{
protected static List<TestPoco> inputs;
protected static Recordset actual;
Establish context = () => inputs = new List<TestPoco> { new TestPoco { /* set values */ } };
Because of = () => actual = input.ToRecordset ();
It should_have_copied_the_bool_value = () => actual.BoolValue.ShouldBeTrue ();
It should_have_copied_the_int_value = () => actual.Int32Value.ShouldBe (1);
It should_have_copied_the_String_value = () => actual.StringValue.ShouldBe ("foo");
}
I generally use mspec as a benchmark to see if my tests make sense. Your tests read just fine with mspec, and that gives me some semi-automated warm fuzzies that I'm testing the correct things.
For that matter, you've done a better job with multiple asserts. I hate seeing tests that look like:
Assert.That (actual.BoolValue == true && actual.Int32Value == 1 && actual.StringValue == "foo");
Because when that fails, the error message "expected True, got False" is completely worthless. Multiple asserts, and using the unit-testing framework as much as possible, will help you a great deal.
This should be something to check out http://rauchy.net/oapt/
Tool that generates a new test case for every assert.