I use NUnit integration tests.
I am trying to test to make sure that user can't create account with existing email. (test#example.com)
I need to have test data in the database (account with test#example.com email).
I can create this account in the test function, or in the sql script (and run it before integration tests).
Where is the better place to create this test data?
Neither option is wrong but there are a number of ways to extend and solidify your strategy:
Mocking which goes hand-in-hand with TDD
Generation of db test data with tools like RedGate's Sql Data Gen
Creation of pluggable data providers (IoC/DI) where you can swap between test and production
None of these solutions are mutually exclusive. I would recommend the last item especially (pluggable provider) and then a choice between object mocking or faux but quality db test data.
Your best bet is to look into Dependency Injection and Mocking frameworks. This way you can swap out data providers with mocked data providers and use the data that fits your needs for the specific test.
If you're using NHibernate or similar, you can always recreate your db schema before each test(fixture).
In a situation like you describe, I would prefer to create the account in the test function.
A unit test should be as self contained as possible. Also, it helps to be able to understand what you are testing, if you can see all the data required for the test in one place.
Here's a totally made up example that should illustrate:
[Test]
public void Test_CannotCreateDuplicateEmail()
{
// Arrange
CreateAccount("test#example.com"); // OK
// Act
try
{
CreateAccount("test#example.com");
// If control arrives here, then the test has failed.
Assert.Fail();
}
// Assert
catch(AccountException ex)
{
// Assert that the correct exception has been thrown.
Assert.AreEqual("Failed", ex.Message);
}
}
Related
I am in need of some testing advice.
I know that it's generally bad practice to hit a database in Unit Tests except in exceptional circumstances.
I'm taking a TDD approach to an MVC project using EF. My first test is:
void DatabaseShouldExist() { ... }
I would like to know... Is this an exceptional circumstance?
I want to check that EF generated the DB and my next test will be to check if it contains the the correct seed data.
How would you go about testing this?
Should it be tested?
You want to test behaviour, so not if a DB exists or not on it own.
As suggested in comments, start with business logic.
TDD starts small and is iterative, don't dive into DB logic test 1.
Simplistic example (For a App to store movies)
Test 1 - shouldAddAMoveToList()
Test 2 - shouldBeAbleToRetrieveAMovieFromList()
Test 3 - shouldPersistAMovieBeweenSessions() // Could Be DB here
When using TDD, pick something simple first. The DB part should come into play a bit later on.
Personally I would avoid testing against a DB with a Unit Test, and save this for Integration tests. DAO pattern is good for this as you could persist in memory, or simply mock the DB side in Unit Tests.
Unit Tests should try to adhere to the FIRST principle, introducing Databases can slow down tests, and prevent them being independent (unless clearing DB each time) - At very least try to use in-memory database for Unit Tests
I have a question on unit testing. Currently I have a large project that calls many SP and does not get a return for most methods. Really it is a large wrapper for many SQL calls. There is not a lot of logic as it is all held in the SP it also has sections of in line sql.
I need to unit test this c# project but it is becoming clear that the unit test would be pointless as it would call many SP which all would be mocked. Am I worried I am thinking about this incorrectly.
My question is that has anyone had this problem and what did they do? Should I be doing database unit tests instead, any insight would be a great help.
Thanks.
A unit test should not touch a data access layer as that would be an integration test/system test. What you can test is that your project in fact calls your data access layer. Doing this will give you peace of mind that during refactors that clicking a button does always call the data access layer.
//Arrange
var dataAccessMock = new Mock<IDataAccessMock>();
dataAccessMock(da => da.ExecuteSomething());
IYourApplication app = new YourApplication(dataAccessMock);
//Act
app.SomeProcessThatCallsExecuteSomething("1234567890");
/Assert
dataAccessMock.Verify(dp=>da.ExecuteSomething(), Times.Once());
note, in this example I am using Moq
After this is is tested to your liking you can focus on your integration test to verify your stored procedures are working as intended. For this you will potentially need to do quite a bit of work to attach a database in a known state, run your stored procedures, and then revert or trash your database so the tests are repeatable.
You should split your testing strategy into integration testing and unit testing.
For integration testing you can rely on your existing database. You will typically write more high-level tests here and verify that your application interacts with your database correctly.
For unit testing you should only pick selected scenarios that actually make sense for mocking out. These are typically scenarios where a lot of business logic "sits on top" of your database logic and you want to verify that business logic.
Over time you can mock out more and more database calls, but for the beginning identify the critical spots.
You have discovered one reason that business logic should generally go in the business, rather than data access, layer. Certainly there are exceptions dictated by performance and sometimes security concerns, but they should remain exceptions.
Having said that, you can still develop a strategy to test your sprocs (though depending on how extensive they are, it may or may not be correct to call those tests "unit tests").
You can use a unit testing framework either way.
In the initialization section, restore a testing copy of the database to a known state, e.g. by loading it from a previously saved copy.
Then, execute unit tests that exercise the stored procedures. Since the stored procedures generally do not return anything, your unit test code will have to select values from the database to check whether the expected changes were made or not.
It may be necessary, depending on possible interactions between stored procedures, to restore the database between each test, or between groups of related tests.
Data / Persistence Layer could is the often most neglected code from a unit testing perspective (true unit testing using test doubles: mocks, stubs, fakes, etc.). If you are connecting to a database then you are integration testing. I find value in a) well architected data/persistence layers that as a side effect are easy to test (uses interfaces, good data access framework abstractedion, etc. and b) are actually unit and integration tested property.
I have been doing a lot of unit testing lately with mocking. The one thing that strikes me as a bit of a problem are the differences between querying against an in memory list (via a mock of my repository) and querying directly against the database via entity framework.
Some of these situations might be:
Testing a filter parameter which would be case insensitive against a database but case sensitive
against an in memory collection leading to a false fail.
Linq statements that might pass against an in memory collection but would fail against entity framework because they arent supported leading to a false pass.
What is the correct way to handle or account for these differences so that there are not false passes or fails in tests? I really like mocking as it makes things so much quicker and easier to test. But it seems to me that the only way to get a really accurate test would be to just test against a the entity framework/database environment.
Besides the unit tests you do you should also create integration tests which run against a real database setup as encountered in production.
I'm not an expert for EF but with NHibernate for example you can create a configuration which points to an in-memory instance of SQLite where you then run your quick tests against (i.e. during a development cycle where you want to get through the test suite as fast as possible). When you want to run your integration tests against a real database you simply change the NHibernate config to point to a real database setup and run the same tests again.
Would be surprising if you could not achieve something similar with EF.
You can use DevMagicFake, this framework will fake the DB for you and can also generate data so you can test your application without testing the DB
First and most important is you can define any behavior data within your mock. Second is speed. From unit testing perspective testing speed counts. Database connections are bottleneck most of time so that's why you mock it with tests.
To implement testing properly you need to work on your overall arch first.
For instance to access data layer I use repository pattern sometimes. It's described really good in Eric Evans DDD book.
So let's say if your repository is defined as below
interface IRepository: IQueryable, ICollection
you can handle linq queries pretty straightforward.
Further reading Repository
I would make my mocks more granular, so that you don't actually query against a larger set in a mock repository.
I typically have setters on my mock repository that I set in each test to control the output of the mocked repository.
This way you don't have to rely on writing queries against a generic mock, and your focus can be on testing the logic in the method under test
public List<int> GetPortfolioList()
{
using (var connection = new SqlConnection("<connectionString>"))
using (var command = new SqlCommand("SELECT * FROM Portfolio", connection))
{
connection.Open();
var portfolioTable = SqlHelper.GetDataTable(command);
var portfolios = from DataRow row
in portfolioTable.Rows
select int.Parse(row["Portfolio"].ToString());
return portfolios.ToList();
}
}
Take this method in a SQL DAL provider to retrieve a list of portfolios, as the name (and code) suggests. Because the database table for integration testing contains a fairly static set of data we can Assert against several expectations. e.g. The list of portfolios will:
- not be empty
- contain certain known values
- contain no duplicates
Following a peer review, someone insisted that this code isn't being properly tested (in isolation) because it relies on database access. In the case that most of the value is found in ensuring that this method returns data from a database whose state is guaranteed, I've been unable to see the value in mocking away the database call in order to write a unit test for this method. Am I missing something?
I'll take the contrary view because I just finished writing a fake db (using in memory lists) to make linq to sql (or linq to anything) unit testable.
This is one question I used to pick a suitable way to fake/mock the database. (from reading your code though, the embedded "SELECT * FROM" means you are leaning on SQL more than linq, which will make it harder to factor your code into stuff that has to be executed by SQL Server and stuff that linq is capable of dealing with.
How are people unit testing code that uses Linq to SQL
I can now run unit tests that will succeed or fail depending on the suitability of my linq query even if the database is unplugged from the wall.
For example, how is your code to react if row["Portfolio"].ToString() is null, how does the code react when this doesn't return any rows, or returns 2?
And even if you are only doing integration tests, nunit is not a bad way to integration tests, just be careful not to call them unit tests, lest a purist get upset about it.
The method uses Linq to project some values from the DB into a list of integers - you may want to test that it does that correctly.
I would split the code into two - the data retrieval and the projection (Linq query) - you could then test the linq query with mock data, without needing to mock the database.
I would also say there is little value in unit testing data access code.
As a testing purist, I believe that I cannot "unit test" a DAL because I cannot do it in isolation. This means that a portion of my code, the one interacting with the database, goes without unit testing. The code above looks fine; just make sure that it exists as part of a larger contract that you can test in other objects.
I do perform integration testing, though, by building a database, seeding it, and ensuring that my DAL works.
I would agree with you, the only value here will be integration testing, there's really nothing to unit test without being pedantic.
The unit test code for any callers of this method should mock this method away, of course.
e: The exception cases MathewMartin mentions above would be the only things I would consider worth unit testing in this scenario.
I'm writing unit-tests for an app that uses a database, and I'd like to be able to run the app against some sample/test data - but I'm not sure of the best way to setup the initial test data for the tests.
What I'm looking for is a means to run the code-under-test against the same database (or schematically identical) that I currently use while debugging - and before each test, I'd like to ensure that the database is reset to a clean slate prior to inserting the test data.
I realize that using an IRepository pattern would allow me to remove the complexity of testing against an actual database, but I'm not sure that will be possible in my case.
Any suggestions or articles that could point me in the right direction?
Thanks!
--EDIT--
Thanks everyone, those are some great suggestions! I'll probably go the route of mocking my data access layer, combined with some simple set-up classes to generate exactly the data I need per test.
Here's the general approach I try to use. I conceive of tests at about three or four levels:: unit-tests, interaction tests, integration tests, acceptance tests.
At the unit test level, it's just code. Any database interaction is mocked out, either manually or using one of the popular frameworks, so loading data is not an issue. They run quick, and make sure the objects work as expected. This allows for very quick write-test/write code/run test cycles. The mock objects serve up the data that is needed by each test.
Interaction tests test the interactions of non-trivial class interactions. Again, no database required, it's mocked out.
Now at the integration level, I'm testing integration of components, and that's where real databases, queues, services, yada yada, get thrown in. If I can, I'll use one of the popular in-memory databases, so initialization is not an issue. It always starts off empty, and I use utility classes to scrub the database and load exactly the data I want before each test, so that there's no coupling between the tests.
The problem I've hit using in-memory databases is that they often don't support all the features I need. For example, perhaps I require an outer join, and the in-memory DB doesn't support that. In that case, I'll typically test against a local conventional database such as MySQL, again, scrubbing it before each test. Since the app is deployed to production in a separate environment, that data is untouched by the testing cycle.
The best way I've found to handle this is to use a static test database with known data, and use transactions to ensure that your tests don't change anything.
In your test setup you would start a transaction, and in your test cleanup, you would roll the transaction back. This lets you modify data in your tests but also makes sure everything gets restored to its original state when the test completes.
I know you're using C# but in the Java World there's the Spring framework. It allows you to run database minipulations in a transaction and after this transaction, you roll this one back. This means that you operate against a real database without touching the state after the test finishes. Perhaps this could be a hint to further investigation in C#.
Mocking is of cause the best way to unit test your code.
As far as integration tests go, I have had some issues using in-memory databases like SQLite, mainly because of small differences in behaviour and/or syntax.
I have been using a local instance of MySql for integration tests in several projects. A returning problem is the server setup and creation of test data.
I have created a small Nuget package called Mysql.Server (see more at https://github.com/stumpdk/MySql.Server), that simply sets up a local instance of MySql every time you run your tests.
With this instance running you can easily set up table structures and sample data for your tests without being concerned of either your production environment or local server setup.
I don't think there is an easy way to finish this. You just have to create those Pre-Test sql setup scripts and post-test Tear-down scripts. Then you need trigger those scripts for each run. A lot of people suggest SQLLite for unit test setup.
I found it best to have my tests go to a different db so I could wipe it clean and put in the data I wanted for the test.
You may want to have the database be something that can be set within the program, then your test can tell the classes to change the database.
This code clears all data from all user's tables in MS SQL Server:
private DateTime _timeout;
public void ClearDatabase(SqlConnection connection)
{
_timeout = DateTime.Now + TimeSpan.FromSeconds(30);
do
{
SqlCommand command = connection.CreateCommand();
command.CommandText = "exec sp_MSforeachtable 'DELETE FROM ?'";
try
{
command.ExecuteNonQuery();
return;
}
catch (SqlException)
{
}
} while (!TimeOut());
if (TimeOut())
Assert.Fail("Fail to clear DB");
}
private bool TimeOut()
{
return DateTime.Now > _timeout;
}
If you are thinking about a real database usage, then mostlikely we're talking integration tests here. I.e tests, which check app behavior as a composition of different components contrary to unit tests, where components are supposed to be tested in isolation.
Having the testing scope defined, I wouldn't recommend using things like in-memory databases or mocking libraries as the other authors suggested. The problem is that usually there is a slightly different behavior or reduced set of features for in-memory databases and there is no database at all with mocking, therefore you'll be testing some other application in general sense and not the one you'll be delivering to your customers.
I'd rather suggest to minimize the amount of integration tests by covering just a crucial parts of your logic leaving the rest for unit testing, while using a real database with the setup as close to the production one as possible. Test runs could be too slow and a real pain if there are a lot of integration ones.
Also you might use some tricks to optimize the speed of your tests execution:
Split tests to Read and Write in regard to the data mutations they introduce and run the former ones in parallel and without any cleanup. (E.g HTTP GET requests are safe to be run in parallel if the system under test is a webapp and tests are more like end-to-end);
Use the only insert/delete script for all the data and optimize as much as possible. You might find Reseed library I'm developing currently helpful. It's able to generate both insert and delete scripts for you. So basically what you asked for. Or check out Respawn which could be used for database cleanup;
Use database snapshots for the restore, which might be faster than full insert/delete cycle;
Wrap each test in transaction and revert it afterwards (this one is also not 100% honest and somewhat fragile);
Parallelize your tests by using a pool of databases instead of the only. Docker and TestContainers could be suitable here;