I am having difficulties while trying to understand the purpose of mocking my WCF applications in unit tests. I have read this article Testing with mock objects and I believe I got the idea why we use mocking in standard application - it is better to test behaviour rather than implementation of the method. So with using Mock objects, I can test whether certain method called certain mock object, whether it changed some properties etc.
But while I was searching for some good ways how to test WCF, everyone was suggesting to use Mocking as well. The thing is, that I feel like I should test whether method (which is communication with service) is really able to reach the service and obtain result, that is satisfactory...which is not the case I will achieve using mock object.
Question:
Is using mock objects in my WCF app unit tests really intended to test only whether the method tried to call the OperationContract(method) exposed by service (while not expecting the real results)?
Or am I missing something?
The purpose of Unit Testing is not to test your application, but to make sure the code you've written is doing what is intended and subsequently let you know when you've made a simple coding goof. When it comes to testing for communication-based services, there are a dozen other factors, not the least of which being actual connectivity, that could affect your connection but not actually tell you anything useful about your code. Thus, you should rely on running debug runs of your application to test end-to-end functionality, and Unit Tests with those mocks to test the actual code functionality.
Will you miss something here and there? Yes, but the definition of good Unit Tests is such that you'll eventually have to touch them up when this happens.
Related
I’ve seen lots of info on this topic but wanted to provide a specific example and ask some specific questions.
I’m currently in the middle of a development project in which I’m programming against a telephony system via the OEM provided SDK. I’ve created several interfaces & classes that extend the functionality of the SDK and have refactored these to support dependency injection for ease of testing. At the lowest level, I have methods like “retrieveUserInfo” that accepts a connection interface and a query object.
When Unit testing, I could actually create a connection to the telephony system, get back a given user, and check it for the correct data. This test is meaningful to me as it lets me know both my Middleware and the underlying OEM libraries are working correctly; however, because it’s actually creating a connection to an outside system, it seems more like an integration test to me (the test will fail if you can’t connect to the telephony system, a user is not configured as expected, I have some bug in my business logic, or there is an issue with the underlying libraries).
Should this test be labeled as an integration test? If so, how would I go about attempting to unit test methods like “retrieveUserInfo”? How do I properly segment these 2 types of tests?
Defining what a 'unit' is, is quite difficult. But if what you're testing is putting calls on external libraries that actually call to different systems, then you're definitely in the realm of integration testing.
What a unit is limited to is a bit subjective, but might be restricted to a class or public method on a class.
If you want to unit test a method that calls methods on an external dependency, then that's something you want to mock or stub (see Moq for a good mocking library).
Sorry if I am asking very basic question,
I have set of web services (developed using .Net WebApi). These services are either business layer or data access layer APIs. These APIs are either dependent on other services or database itself.
I want to write unit test cases for it. I have following questions
As business layer APIs has dependency on data access service or some other service. If I write unit test just to invoke business API then it would invoke data access API. Is this the correct way to write unit test case? or should I inject all dependency object with unit test? I think earlier one would be integration test not unit test.
Should I write unit tests for Data access layer at all? I checked this link (Writing tests for data access code: Unit tests are waste) and it says DAL does not require unit tests. Should I still write tests for data access layer. I think it would be integration test not unit tests?
Question 1:
I would say if you want to do TDD, then it's not the "correct" way, because as you said, you would be performing integration tests. Then again, maybe you don't want to do TDD and integration tests are good enough for you, but to answer the question: this wouldn't be the proper way to **unit-**test your code.
Question 2
I would say it depends what you have in your data access layer. For instance, if you implement repositories, you will probably want to write a few tests.
Save method
You want to make sure that given an entity that you have retrieved from your repository, editing some properties of this entity and persisting the changes will actually save the modifications and not create a new entity. Now: you might think this is an integration test, but it really depends on how well designed your code is. For instance, your repository could be just an extra layer of logic on top of a low-level ORM. In that case, when testing the save method, what you will do is that you will assert that the right methods are called with the right parameters on the ORM service injected in your repository.
Errors and exceptions
While accessing data, it is possible to have problems such as connection to the database being broken, or that the format of the data is not as expected, or deserialization problems. If you want to provide some good error handling and perhaps create custom exceptions and add more information to them depending on the context, then you definitely want to write tests to make sure the corrext information is propagated
on the other hand
If your DAL is just a few classes that wrap a non-mockable ORM, and you don't have any logic in there, then perhaps you don't need tests, but it seems that this doesn't happen too often, you will pretty much always have a bit of logic that can go wrong and that you want to test.
I am attempting to do TDD right ! I was reading about the TDD Inside Out as opposed to Outside In. Reason being is that i don't know how my layers up front so my idea was to start writing a test, have it fail and then start writing my first layer.
While writing my first layer I notice that I need another layer, lets call it a service layer. This is where I get confused, what do I do here ?
Do I stop and create a new Test that fails so that I can implement my new service layer using TDD? Once done, I go back to my original layer and should I create a mock of my service layer here ? Or use the service layer I just create via TDD?
This is TDD right ? So if I am mocking things out then maybe my TDD is not driving my development ? But of course if I don't mock things out, these technically are not unit tests but more of integration tests ?
If indeed my unit tests (written via TDD) use mocks then I need to have some other kind of tests to test the integration of each individual layer as one unit ??
An integration test or e2e test?
I think my problems are basically when I need to introduce new layers, should I mock these out, should I create a new test to drive the development of this new layer?
I hope somebody can help with untangling this muddle I have myself in !
Thanks
With more experience , you will become better with that.
But for now let me say this.
First of all, think about TDD as a tool to design clean code (check Uncle Bob's Clean Code to get more insight.). By no means it replaces any system design efforts. That means that you have to know what you want classes to design (at least roughly) and you have to define interfaces between those classes as well.
Second, unit tests according to Mike Cohn - Working Effectively with Legacy Code - Chapter 2 are tests that do not:
talk to a database
communiating accross a network
touching the file system
require you to do special things with your environment to run.
So, you should be well in the limits of a unit test.
In general, you want to write unit tests for each component (or class). That means you create fake classes or mocks for each of the interfaces, e.g. for each of your service layer classes. This means that you have to know the exact interface (method parameters and return value) that each of the call needs.
Try to get as far as possible with one instance and then move on to the next.
If you are unsure about how your design has to look like, consider building an untested prototype. Just as much code so you see the components working together and to help build your interfaces. Then sketch down the design, throw away your prototype and start over with a TDD approach.
When developing in TDD style you should work with interfaces as much as possible.
Unit testing means that you test every unit isolated from most (ideally all) other code.
So in your case: if the code you currently work on needs to make calls to some service layer. then jsut create an interface for the new modules and mock their correct behaviour (or expected error behaviour if you want to test error handling).
... and put testing your new service layer on your todo list ;)
This way you concentrate your work on your current unit and have an interface ready for your service layer, when you start working on this.
if you want to test how your layers work together, you need some kind of integration test.
I am trying to following TDD and I have come across a small issue. I wrote a Test to insert a new user into a database. The Insert new user is called on the MyService class, so I went ahead and created mytest. It failed and I started to implement my CreateUser method on my MyService Class.
The problem I am coming across is the MyService will call to a repository (another class) to do the database insertion.
So I figured I would use a mocking framework to mock out this Repository class, but is this the correct way to go?
This would mean I would have to change my test to actually create a mock for my User Repository. But is this recommended? I wrote my test initially and made it fail and now I realize I need a repository and need to mock it out, so I am having to change my test to cater for the mocked object. Smells a bit?
I would love some feedback here.
If this is the way to go then when would I create the actual User Repository? Would this need its own test?
Or should I just forget about mocking anything? But then this would be classed as an integration test rather than a unit test, as I would be testing the MyService and User Repository together as one unit.
I a little lost; I want to start out the correct way.
So I figured I would use a mocking framework to mock out this
Repository class, but is this the correct way to go?
Yes, this is a completely correct way to go, because you should test your classes in isolation. I.e. by mocking all dependencies. Otherwise you can't tell whether your class fails or some of its dependencies.
I wrote my test initially and made it fail and now I realize I need a
repository and need to mock it out, so I am having to change my test
to cater for the mocked object. Smells a bit?
Extracting classes, reorganizing methods, etc is a refactoring. And tests are here to help you with refactoring, to remove fear of change. It's completely normal to change your tests if implementation changes. I believe you didn't think that you could create perfect code from your first try and never change it again?
If this is the way to go then when would I create the actual User
Repository? Would this need its own test?
You will create a real repository in your application. And you can write tests for this repository (i.e. check if it correctly calls the underlying data access provider, which should be mocked). But such tests usually are very time-consuming and brittle. So, it's better to write some acceptance tests, which exercise the whole application with real repositories.
Or should I just forget about mocking anything?
Just the opposite - you should use mocks to test classes in isolation. If mocking requires lots of work (data access, ui) then don't mock such resources and use real objects in integration or acceptance tests.
You would most certainly mock out the dependency to the database, and then assert on your service calling the expected method on your mock. I commend you for trying to follow best practices, and encourage you to stay on this path.
As you have now realized, as you go along you will start adding new dependencies to the classes you write.
I would strongly advise you to satisfy these dependencies externally, as in create an interface IUserRepository, so you can mock it out, and pass an IUserRepository into the constructor of your service.
You would then store this in an instance variable and call the methods (i.e. _userRepository.StoreUser(user)) you need on it.
The advantage of that is, that it is very easy to satisfy these dependencies from your test classes, and that you can worry about instantiating of your objects, and your lifecycle management as a separate concern.
tl;dr: create a mock!
I have two set of testing libraries. One for UnitTests where I mock stuff. I only test units there. So if I would have a method of AddUser in the service I would create all the mocks I need to be able to test the code in that specific method.
This gives me a possibility to test some code paths that I would not be able to verify otherwise.
Another test library is for Integration tests or functional tests or whatever you want to call it. This one is making sure that a specific use case. E.g. Creating a tag from the webpage will do what i expect it to do. For this I use the sql server that shipps with Visual studio 2012 and after every test I delete the database and start over.
In my case I would say that the integration tests are much more important then the unit tests. This is because my application does not have so much logic, instead it is displaying data from the database in different ways.
Your initial test was incomplete, that's all. The final test is always going to have to deal with the fact the new user gets persisted.
TDD does not prescribe the kind of test you should create. You have to choose beforehand if it's going to be a unit test or some kind of integration test. If it's a unit test, then the use of mocking is practically inevitable (except when the tested unit has no dependencies to isolate from). If it's an integration test, then actual database access (in this case) would have to be taken into account in the test.
Either kind of test is correct. Common wisdom is that a larger unit test suite is created, testing units in isolation, while a separate but smaller test suite exercises whole use case scenarios.
Summary
I am a huge fan of Eiffel, but while the tools of Eiffel like Design-by-Contract can help significantly with the Mock-or-not-to-Mock question, the answer to the question has a huge management-decision component to it.
Detail
So—this is me thinking out loud as I ponder a common question. When contemplating TDD, there is a lot of twisting and turning on the matter of mock objects.
To Mock or Not to Mock
Is that the only binary question? Is it not more nuanced than that? Can mocks be approached with a strategy?
If your routine call on an object under test needs only base-types (i.e. STRING, BOOLEAN, REAL, INTEGER, etcetera) then you don't need a mock object anyhow. So, don't be worried.
If your routine call on an object under test either has arguments or attributes that require mock objects to be created before testing can begin then—that is where the trouble begins, right?
What sources do we have for constructing mocks?
Simple creation with:
make or default create
make with hand-coded base-type arguments
Complex creation with:
make with database-supplied arguments
make with other mock objects (start this process again)
Object factories
Production code based factories
Test code based factories
Data-repo based data (vs hand-coded)
Gleaned
Objects from prior bugs/errors
THE CHALLENGE:
Keeping the non-production test-code bloat to a bare minimum. I think this means asking hard but relevant questions before willy-nilly code writing begins.
Our optimal goal is:
No mocks needed. Strive for this above all.
Simple mock creation with no arguments.
Simple mock creation with base-type arguments.
Simple mock creation with DB-repo sourced base-type arguments.
Complex mock creation using production code object factories.
Complex mock creation using test-code object factories.
Objects with captured states from prior bugs/errors.
Each of these presents a challenge. As stated—one of the primary goals is to always keep the test code as small as possible and reuse production code as much as possible.
Moreover—perhaps there is a good rule of thumb: Do not write a test when you can write a contract. You might be able to side-step the need to write a mock if you just write good solid contract coverage!
EXAMPLE:
At the following link you will find both an object class and a related test class:
Class: https://github.com/ljr1981/stack_overflow_answers/blob/main/src/so_17302338/so_17302338.e
Test: https://github.com/ljr1981/stack_overflow_answers/blob/main/testing/so_17302338/so_17302338_test_set.e
If you start by looking at the test code, the first thing to note is how simple the tests are. All I am really doing is spinning up an instance of the class as an object. There are no "test assertions" because all of the "testing" is handled by DbC contracts in the class code. Pay special attention to the class invariant. The class invariant is either impossible with common TDD facilities, or nearly impossible. This includes the "implies" Boolean keyword as well.
Now—look at the class code. Notice first that Eiffel has the capacity to define multiple creation procedures (i.e. "init") without the need for a traffic-cop switch or pattern-recognition on creation arguments. The names of the creation procedures tell the appropriate story of what each creation procedure does.
Each creation procedure also contains its own preconditions and post-conditions to help cement code-correctness without resorting to "writing-the-bloody-test-first" nonsense.
Conclusion
Mock code that is test-code and not production-code is what will get you into trouble if you get too much of it. The facility of Design-by-Contract allows you to greatly minimize the need for mocks and test code. Yes—in Eiffel you will still write test code, but because of how the language-spec, compiler, IDE, and test facilities work, you will end up writing less of it—if you use it thoughtfully and with some smarts!
I've got NHibernate-based (constructor ISessionFactory injection) generic repository implementation which is stored inside DAL. It implements contract which is stored in `Domain Layer'.
Should I test real repository behavior using SQl CE or should I refactor my application to support agnostic-like (like in Tim Maccharty's book http://www.wrox.com/WileyCDA/WroxTitle/productCd-0470147563,descCd-authorInfo.html) Unit of Work and then give my fake implementation of IUnitOfWorkRepository?
Is it a valid approach to run tests on a local database exposing real repository implementation?
Thanks!
The issue is what your are testing and why. That will answer the question.
If:
I want to test a third party tool
That is, are you testing if NHibernate is working (not a kind of test
I do). Then do whatever it requires, so refactoring not required. Loose yourself.
I want to test how my code interacts with a thrid party tool
Then you are talking about what I like to call a interaction test. Refactoring is required as your more interested in how your using NHiberate than if it works.
I want to test my code
The abstract NHibernate entirely. Do whatever is necessary ... wrapper? Your now back into unit testing.
I want to test my application from a user point of view
I think this is above the scope your talking. But you can use this scope talking about components. So ... hmmm ... worthwhile but not easy. Not a unit test, so you want to instantiate the component/app and run the whole thing as its 'user' does. I call these 'UATs' and usually implement as 'Coded UATs'.
A unit test is testing a unit in isolation. So, no, it's not even a unit test if you're going to the database.
Abstract away and test your repositories with mocked up interfaces.
I think to test the repositories you need to use the actual scenario. Otherwise you don't have any other place to test the database access. Mocking the repositories is not a good practice. Because you don't have any logic which is need to test in the repositories. I think you need to write integration tests which is calling actual repositories to have any benefit from it.