So I've been trying to write a test for mass transit using the in-memory feature. I wondered what peoples approach was to waiting for consumers to execute. In the example below a use a sleep or I've also tried a while loop, but not a fan of either, any better ideas? I basically want to check that the consumer is executed.
[Fact]
public async Task SomeTest()
{
var busControl = Bus.Factory.CreateUsingInMemory(cfg =>
{
cfg.ReceiveEndpoint("commands", ec =>
{
ec.LoadFrom(context);
});
});
var address = new Uri(bus.Address, "commands")
await sendEndpoint.Send(MyExampleCommand());
Thread.Sleep(2000);
//Check nsubstitute mock received
}
Look at the test harness features that are built into MassTransit. They should give you some good ideas of how to test them.
You can look at the harness tests to see how they should be used. Note that they work with any test framework, not just NUnit.
https://github.com/MassTransit/MassTransit/blob/develop/tests/MassTransit.Tests/Testing/ConsumerTest_Specs.cs
The Testing documentation explains how to use the test harnesses.
We are using OpenCover for our solution test coverage and I noticed that
public async Task Build(ProcessorContext context)
{
context.ResultBetSlip.Bets.Clear();
// Here OpenCover tells me that I need to cover two branches
// so I assume I need to verify that GetAvailablePlugins can be empty and
// no Build method was called.
// How do I do that?
foreach (var plugin in _pluginFactory.GetAvailablePlugins())
{
await plugin.Build(context);
}
}
Now my test would look like that
[Test]
public async Task Build_ShouldntEnterForeachWhenThereAreNoPluginsRegistered()
{
// Arrange
var pluginFactoryMock = new Mock<IBetSlipProcessorServicePluginFactory>();
var sut = new BetSlipProcessorService(pluginFactoryMock.Object);
pluginFactoryMock
.Setup(pf => pf.GetAvailablePlugins())
.Returns(new List<IBetSlipProcessorServicePlugin>());
// Act
await sut.Build(AutoFixtureSimplified.Create<ProcessorContext>());
// Assert
???
}
Should I even consider testing such case if it is possible?
EDIT:
As requested this is the test coverage report:
And here you can find gist of all the tests that I do in order to achieve such coverage.
https://gist.github.com/kuskmen/df3f112b2b6d880741ee6ab559d64d53
I am assuming you are using the Moq framework for mocking. If this is the case you can do one of two things.
You can create your mock in strict mode
You can expect that when the plugin.Build(..) method is called that an exception is thrown.
A similar question was asked here: How to verify that method was NOT called in Moq?
edit:
I simulated the exact scenario that you are seeing and I have narrowed it down to the data type that you are iterating over. Due to the fact that you are using a list I would guess there is some internal workings of the list that are causing this problem. I changed all the list references to arrays and the branch coverage returned as expected.
I've been given a task of creating a restful web service with JSON formating using WCF with the below methods using TDD approach which should store the Product as a text file on disk:
CreateProduct(Product product)
GetAProduct(int productId)
URI Templates:
POST to /MyService/Product
GET to /MyService/Product/{productId}
Creating the service and its web methods are the easy part but
How would you approach this task with TDD? You should create a test before creating the SUT codes.
The rules of unit tests say they should also be independent and repeatable.
I have a number of confusions and issues as below:
1) Should I write my unit tests against the actual service implementation by adding a reference to it or against the urls of the service (in which case I'd have to host and run the service)? Or both?
2)
I was thinking one approach could be just creating one test method inside which I create a product, call the CreateProduct() method, then calling the GetAProduct() method and asserting that the product which was sent is the one that I have received. On TearDown() event I just remove the product which was created.
But the issues I have with the above is that
It tests more than one feature so it's not really a unit test.
It doesn't check whether the data was stored on file correctly
Is it TDD?
If I create a separate unit test for each web method then for example for calling GetAProduct() web method, I'd have to put some test data stored physically on the server since it can't rely on the CreateProduct() unit tests. They should be able to run independently.
Please advice.
Thanks,
I'd suggest not worrying about the web service end points and focus on behavior of the system. For the sake of this discussion I'll drop all technical jargon and talk about what I see as the core business problem you're trying to solve: Creating a Product Catalog.
In order to do so, start by thinking through what a product catalog does, not the technical details about how to do it. Use that as your starting points for your tests.
public class ProductCatalogTest
{
[Test]
public void allowsNewProductsToBeAdded() {}
[Test]
public void allowsUpdatesToExistingProducts() {}
[Test]
public void allowsFindingSpecificProductsUsingSku () {}
}
I won't go into detail about how to implement the tests and production code here, but this is a starting point. Once you've got the ProductCatalog production class worked out, you can turn your attention to the technical details like making a web service and marshaling your JSON.
I'm not a .NET guy, so this will be largely pseudocode, but it probably winds up looking something like this.
public class ProductCatalogServiceTest
{
[Test]
public void acceptsSkuAsParameterOnGetRequest()
{
var mockCatalog = new MockProductCatalog(); // Hand rolled mock here.
var catalogService = new ProductCatalogService(mockCatalog);
catalogService.find("some-sku-from-url")
mockCatalog.assertFindWasCalledWith("some-sku-from-url");
}
[Test]
public void returnsJsonFromGetRequest()
{
var mockCatalog = new MockProductCatalog(); // Hand rolled mock here.
mockCatalog.findShouldReturn(new Product("some-sku-from-url"));
var mockResponse = new MockHttpResponse(); // Hand rolled mock here.
var catalogService = new ProductCatalogService(mockCatalog, mockResponse);
catalogService.find("some-sku-from-url")
mockCatalog.assertWriteWasCalledWith("{ 'sku': 'some-sku-from-url' }");
}
}
You've now tested end to end, and test drove the whole thing. I personally would test drive the business logic contained in ProductCatalog and likely skip testing the marshaling as it's likely to all be done by frameworks anyway and it takes little code to tie the controllers into the product catalog. Your mileage may vary.
Finally, while test driving the catalog, I would expect the code to be split into multiple classes and mocking comes into play there so they would be unit tested, not a large integration test. Again, that's a topic for another day.
Hope that helps!
Brandon
Well to answer your question what I would do is to write the test calling the rest service and use something like Rhino Mocks to arrange (i.e setup an expectation for the call), act (actually run the code which calls the unit to be tested and assert that you get back what you expect. You could mock out the expected results of the rest call. An actual test of the rest service from front to back would be an integration test not a unit test.
So to be clearer the unit test you need to write is a test around what actually calls the rest web service in the business logic...
Like this is your proposed implementation (lets pretend this hasn't even been written)
public class SomeClass
{
private IWebServiceProxy proxy;
public SomeClass(IWebServiceProxy proxy)
{
this.proxy = proxy;
}
public void PostTheProduct()
{
proxy.Post("/MyService/Product");
}
public void REstGetCall()
{
proxy.Get("/MyService/Product/{productId}");
}
}
This is one of the tests you might consider writing.
[TestFixture]
public class TestingOurCalls()
{
[Test]
public Void TestTheProductCall()
{
var webServiceProxy = MockRepository.GenerateMock<IWebServiceProxy>();
SomeClass someClass = new SomeClass(webServiceProxy);
webServiceProxy.Expect(p=>p.Post("/MyService/Product"));
someClass.PostTheProduct(Arg<string>.Is.Anything());
webServiceProxy.VerifyAllExpectations();
}
}
Why won't this test method work? I keep getting requires a return value or an exception to throw.
public AuthenticateResponse Authenticate(string username, string password)
{
string response = GetResponse(GetUrl(username, password).ToString());
return ParseResponse(response);
}
[TestMethod()]
[ExpectedException(typeof(XmlException))]
public void Authenticate_BadXml_ReturnException()
{
MockRepository mockRepository = new MockRepository();
SSO sso = mockRepository.Stub<SSO>();
sso.Stub(t => t.GetResponse("")).Return("<test>d");
AuthenticateResponse response = sso.Authenticate("test", "test");
}
Your repository is still in "record" mode. You're mixing record/replay semantics (the "old" way of doing things) with the newer AAA (arrange/act/assert) style.
Instead of creating your own repository, simply use:
var sso = MockRepository.GeneateStub<SSO>();
Everything should work fine now.
Your last line is calling the Authenticate method on your stub object, you haven't set up a return or value or exception to throw when calling it, so Rhino Mocks doesn't know what the stub should do and it causes an error. You probably don't want to call a method on your stub - that seems kind of pointless to me, is there another object (that you're actually testing in this test) that you should be calling a method on?
Is that your whole test? If so, your test makes no sense. The only object in your test is the one you're stubbing--where is the subject of the test?
If you're trying to test the SSO class, you absolutely never want to mock/stub it. If SSO has one or more dependencies, use the mocking framework to set up canned interactions between those dependencies and your SUT. That is the exact purpose of a mocking framework.
I am currently writing some unit tests for a business-logic class that includes validation routines. For example:
public User CreateUser(string username, string password, UserDetails details)
{
ValidateUserDetails(details);
ValidateUsername(username);
ValidatePassword(password);
// create and return user
}
Should my test fixture contain tests for every possible validation error that can occur in the Validate* methods, or is it better to leave that for a separate set of tests? Or perhaps the validation logic should be refactored out somehow?
My reasoning is that if I decide to test for all the validation errors that can occur within CreateUser, the test fixture will become quite bloated. And most of the validation methods are used from more than one place...
Any great patterns or suggestions in this case?
Every test should only fail for one reason and only one test should fail for that reason.
This helps a lot with writing a maintainable set of unit tests.
I'd write a couple of tests each for ValidateUserDetails, ValidateUsername and ValidateUserPassword. Then you only need to test that CreateUser calls those functions.
Re read your question; Seems I misunderstood things a bit.
You might be interested in what J.P Boodhoo has written on his style of behaviour driven design.
http://blog.developwithpassion.com/2008/12/22/how-im-currently-writing-my-bdd-style-tests-part-2/
BDD is becoming a very overloaded term, everyone has a different definition and different tools to do it. As far as I see what JP Boodhoo is doing is splitting up test fixtures according to concern and not class.
For example you could create separate fixtures for testing Validation of user details, Validation of username, Validation of password and creating users. The idea of BDD is that by naming the testfixtures and tests the right way you can create something that almost reads like documentation by printing out the testfixture names and test names. Another advantage of grouping your tests by concern and not by class is that you'll probably only need one setup and teardown routine for each fixture.
I havn't had much experience with this myself though.
If you're interested in reading more, JP Boodhoo has posted a lot about this on his blog (see above link) or you can also listen to the dot net rocks episode with Scott Bellware where he talks about a similar way of grouping and naming tests http://www.dotnetrocks.com/default.aspx?showNum=406
I hope this is more what you're looking for.
You definitely need to test validation methods.
There is no need to test other methods for all possible combinations of arguments just to make sure validation is performed.
You seem to be mixing Validation and Design by Contract.
Validation is usually performed to friendly notify user that his input is incorrect. It is very related to business logic (password is not strong enough, email has incorrect format, etc.).
Design by Contract makes sure your code can execute without throwing exceptions later on (even without them you would get the exception, but much later and probably more obscure one).
Regarding application layer that should contain validation logic, probably the best is service layer (by Fowler) which defines application boundaries and is a good place to sanitize application input. And there should not be any validation logic inside this boundaries, only Design By Contract to detect errors earlier.
So finally, write validation logic tests when you want to friendly notify user that he has mistaken. Otherwise use Design By Contract and keep throwing exceptions.
Let Unit Tests (plural) against the Validate methods confirm their correct functioning.
Let Unit Tests (plural) against the CreateUser method confirm its correct functioning.
If CreateUser is merely required to call the validate methods, but is not required to make validation decisions itself, then the tests against CreateUser should confirm that requirement.
What is the responsibility of your business logic class and does it do something apart from the validation? I think I'd be tempted to move the validation routines into a class of its own (UserValidator) or multiple classes (UserDetailsValidator + UserCredentialsValidator) depending on your context and then provide mocks for the tests. So your class now would look something like:
public User CreateUser(string username, string password, UserDetails details)
{
if (Validator.isValid(details, username, password)) {
// what happens when not valid
}
// create and return user
}
You can then provide seperate unit tests purely for the validation and your tests for the business logic class can focus on when validation passes and when validation fails, as well as all your other tests.
I would add a bunch of test for each ValidateXXX method. Then in CreateUser create 3 test cases for checking what happens when each of ValidateUserDetails, ValidateUsername and ValidatePassword fails but the other succeed.
I'm using Lokad Shared Library for defining business validation rules. Here's how I test corner cases (sample from the open-source):
[Test]
public void Test()
{
ShouldPass("rinat.abdullin#lokad.com", "pwd", "http://ws.lokad.com/TimeSerieS2.asmx");
ShouldPass("some#nowhere.net", "pwd", "http://127.0.0.1/TimeSerieS2.asmx");
ShouldPass("rinat.abdullin#lokad.com", "pwd", "http://sandbox-ws.lokad.com/TimeSerieS2.asmx");
ShouldFail("invalid", "pwd", "http://ws.lokad.com/TimeSerieS.asmx");
ShouldFail("rinat.abdullin#lokad.com", "pwd", "http://identity-theift.com/TimeSerieS2.asmx");
}
static void ShouldFail(string username, string pwd, string url)
{
try
{
ShouldPass(username, pwd, url);
Assert.Fail("Expected {0}", typeof (RuleException).Name);
}
catch (RuleException)
{
}
}
static void ShouldPass(string username, string pwd, string url)
{
var connection = new ServiceConnection(username, pwd, new Uri(url));
Enforce.That(connection, ApiRules.ValidConnection);
}
Where ValidConnection rule is defined as:
public static void ValidConnection(ServiceConnection connection, IScope scope)
{
scope.Validate(connection.Username, "UserName", StringIs.Limited(6, 256), StringIs.ValidEmail);
scope.Validate(connection.Password, "Password", StringIs.Limited(1, 256));
scope.Validate(connection.Endpoint, "Endpoint", Endpoint);
}
static void Endpoint(Uri obj, IScope scope)
{
var local = obj.LocalPath.ToLowerInvariant();
if (local == "/timeseries.asmx")
{
scope.Error("Please, use TimeSeries2.asmx");
}
else if (local != "/timeseries2.asmx")
{
scope.Error("Unsupported local address '{0}'", local);
}
if (!obj.IsLoopback)
{
var host = obj.Host.ToLowerInvariant();
if ((host != "ws.lokad.com") && (host != "sandbox-ws.lokad.com"))
scope.Error("Unknown host '{0}'", host);
}
If some failing case is discovered (i.e.: new valid connection url is added), then the rule and the test gets updated.
More on this pattern could be found in this article. Everything is Open Source so feel free to reuse or ask questions.
PS: note that primitive rules used in this sample composite rule (i.e. StringIs.ValidEmail or StringIs.Limited) are thoroughly tested on their own and thus do not need excessive unit tests.