In my application I have multiple small entity framework dbcontexts which share the same database, for example:
public class Context1 : DbContext {
public Context1()
: base("DemoDb") {
}
}
public class Context2 : DbContext {
public Context2()
: base("DemoDb") {
}
}
All database updates are done via scripts and do not rely on migrations (nor will they going forward). The question is - how would you do integration testing against these contexts?
I believe there are three options here (there may be more I just don't know them)
Option 1 - Super context - a context which contains all models and configurations required for setting up the database:
public class SuperContext : DbContext
{
public SuperContext()
: base("DemoDb") {
}
}
In this option the test database would be setup against the super context and all subsequent testing would be done through the smaller contexts.
The reason I am not keen on this option is that I will be duplicating all the configurations and entity models that i have already built.
Option 2 - create a custom initialiser for integration tests that will run all the appropriate db initialisation scripts:
public class IntegrationTestInitializer : IDatabaseInitializer<DbContext> {
public void InitializeDatabase(DbContext context) {
/* run scripts to set up database here */
}
}
This option allows for testing against the true database structure but will also require updating everytime new db scripts are added
Option 3 - just test the individual contexts:
In this option one would just let EF create the test database based upon the context and all tests would operate within there own "sandbox".
The reason that I don't like this is that it doesn't feel like you would be testing against a true representation of the database.
I'm currently swaying towards options 2. What do you all think? Is there a better method out there?
I'm using integration testing a lot, because I still think it's the most reliable way of testing when data-dependent processes are involved. I also have a couple of different contexts, and DDL scripts for database upgrades, so our situations are very similar.
What I ended up with was Option 4: maintaining unit test database content through the regular user interface. Of course most integration tests temporarily modify the database content, as part of the "act" phase of the test (more on this "temporary" later), but the content is not set up when the test session starts.
Here's why.
At some stage we also generated database content at the start of the test session, either by code or by deserializing XML files. (We didn't have EF yet, but otherwise we would probably have had some Seed method in a database initializer). Gradually I started to feel misgivings with this approach. It was a hell of a job to maintain the code/XML when the data model or the business logic changed, esp. when new use cases had to be devised. Sometimes I allowed myself a minor corruption of these test data, knowing that it would not affect the tests.
Also, the data had to make sense, as in they had to be as valid and coherent as data from the real application. One way to ensure that is to generate the data by the application itself, or else inevitably you will somehow duplicate business logic in the seed method. Mocking real-world data is actually very hard. That's the most important thing I found out. Testing data constellations that don't represent real use cases isn't only a wast of time, it's false security.
So I found myself creating the test data through the application's front end and then painstakingly serializing this content into XML or writing code that would generate exactly the same. Until one day it occurred to me that I had the data readily available in this database, so why not use it directly?
Now maybe you ask How to make tests independent?
Integration tests, just as unit tests, should be executable in isolation. They should not depend on other tests, nor should they be affected by them. I assume that the background of your question is that you create and seed a database for each integration test. This is one way to achieve independent tests.
But what if there is only one database, and no seed scripts? You could restore a backup for each test. We chose a different approach. Each integration test runs within a TransactionScope that's never committed. It is very easy to achieve this. Each test fixture inherits from a base class that has these methods (NUnit):
[SetUp]
public void InitTestEnvironment()
{
SetupTeardown.PerTestSetup();
}
[TearDown]
public void CleanTestEnvironment()
{
SetupTeardown.PerTestTearDown();
}
and in SetupTeardown:
public static void PerTestSetup()
{
_tranactionScope = new TransactionScope();
}
public static void PerTestTearDown()
{
if (_tranactionScope != null)
{
_tranactionScope.Dispose(); // Rollback any changes made in a test.
_tranactionScope = null;
}
}
where _tranactionScope is a static member variable.
Option 2, or any variation thereof that runs the actual DB update scripts would be the best. Otherwise than this you are not necessarily integration testing against the same database you have in production (with respect to the schema, at least).
In order to address your concern about requiring updating every time new DB scripts are added, if you were to keep all the scripts in a single folder, perhaps within the project with a build action of "copy if newer", you could programmatically read each file and execute the script therein. As long as the place you're reading the files from is your canonical repository for the update scripts, you will never need to go in and make any further changes.
Related
I'm trying to create some unit tests for my project, after much digging around I found Effort, the idea is great, it mocks the database instead of the dealing with faking the DBContext which by the way is really hard to get it right when using a complex schema.
However I'm trying to get the Email of a user after I specifically added it to the in-memory database create by Effort, here is the code
MyContext contextx = new MyContext(Effort.DbConnectionFactory.CreateTransient());
var client = new Client
{
ClientId = 2,
PersonId = 3,
Person = new Person
{
PersonId = 3,
EMail = "xxxxx#gmail.com"
}
};
contextx.Client.Add(client); //<-- client got added, I checked it and is there
var email = contextx.Client.Select(c => c.Person.EMail).FirstOrDefault();
In the last line above I can't make it to return the email xxxx#gmail.com instead it always returns null.
Any ideas?
Answering Your Direct Question
For the specific question you asked, I would suggest two things:
Take a look at contextx.Client.ToArray() and see how many members you really have in that collection. It could be that the Client collection is actually empty, in which case you'll indeed get null. Or, it could be that the first element in the Client collection has a null value for EMail.
How does the behavior change if you call contextx.SaveChanges() before querying the Client collection on the DbContext? I'm curious to see if calling SaveChanges will cause the newly inserted value to exist in the collection. This really shouldn't be required, but there might be some strange interaction between Effort and the DbContext.
EDIT: SaveChanges() turns out to be the answer.
General Testing Suggestions
Since you tabbed this question with the "unit-testing" tag, I'll offer some general unit testing advice based on my ten years spent as a unit testing practitioner and coach. Unit testing is about testing various small parts of your application in isolation. Typically this means that unit tests only interact with a few classes at once. This also means that unit tests should not depend on external libraries or dependencies (such as the database). Conversely, an integration test exercises more parts of the system at once and may have external dependencies on things like databases.
While this may seem like a quibble over terminology, the terms are important for conveying the actual intent of your tests to other members of your team.
In this case, either you are really wanting to unit test some piece of functionality that happens to depend on DbContext, or you are attempting to test your data access layer. If you're trying to write an isolated unit test of something that depends on the DbContext directly, then you need to break the dependency on the DbContext. I'll explain this below in Breaking the Dependency on DbContext below. Otherwise, you're really trying to integration test your DbContext including how your entities are mapped. In this case, I've always found it best to isolate these tests and use a real (local) database. You probably want to use a locally installed database of the same variety you're using in production. Often, SqlExpress works just fine. Point your tests at an instance of the database that the tests can completely trash. Let your tests remove any existing data before running each test. Then, they can setup whatever data they need without concern that existing data will conflict.
Breaking the Dependency on DbContext
So then, how do you write good unit tests when your business logic depends on accessing DbContext? You don't.
In my applications that use Entity Framework for data persistence, I make sure access to the DbContext is contained within a separate data access project. Typically, I will create classes that implement the Repository pattern and those classes are allowed to take a dependency on DbContext. So, in this case, I would create a ClientRepository that implements an IClientRepository interface. The interface would look something like this:
public interface IClientRepository {
Client GetClientByEMail(string email);
}
Then, any classes that need access to the method can be unit tested using a basic stub / mock / whatever. Nothing has to worry about mocking out DbContext. Your data access layer is contained, and you can test it thoroughly using a real database. For some suggestions on how to test your data access layer, see above.
As an added benefit, the implementation of this interface defines what it means to find a Client by email address in a single, unified place. The IClientRepository interface allows you to quickly answer the question, "How do we query for Client entities in our system?"
Taking a dependency on DbContext is roughly the same scale of a testing problem as allowing domain classes to take a dependency on the connection string and having ADO.Net code everywhere. It means that you have to create a real data store (even with a fake db) with real data in it. But, if you contain your access to the DbContext within a specific data access assembly, you'll find that your unit tests are much easier to write.
As far as project organization, I typically only allow my data access project to take a reference to Entity Framework. I'll have a separate Core project in which I define the entities. I'll also define the data access interfaces in the Core project. Then, the concrete interface implementations get put into the data access project. Most of the projects in your solution can then simply take a dependency on the Core project, and only the top level executable or web project really needs to depend on the data access project.
I am trying to learn how to mock my generic repository so i can unit test all my services.
Im using NHibernate Fluent to handle data access and a Ninject for dependency (I'm not interested in testing that)
My repository interface looks like:
public interface IRepository<TEntity> where TEntity : class
{
IQueryable<TEntity> GetAll();
TEntity Get(int key);
void Insert(TEntity entity);
void Update(TEntity entity);
void Delete(int id);
}
And the actual repository looks like:
public class GenerRepository<TEntity> : IRepository<TEntity>where TEntity : Entity
{
protected ISession Session{get { return NHibernateHelper.OpenSession(); }}
public IQueryable<TEntity> GetAll(){return Session.Query<TEntity>();}
public TEntity Get(int key){return Session.Get<TEntity>(key);}
public void Insert(TEntity entity){Session.Save(entity);}
public void Update(TEntity entity){Session.Update(entity);}
public void Delete(int id){Session.Delete(Session.Load<TEntity>(id));}
}
All my services do the following take the created repository in and use it.
I've read so many articles on how to do this but none are simple or well explained. So any advice between creating a test generic repository or even mocking it. I would also be interested in creating a in memory database but how do i set the configuration up for fluent nhibernate in my test project without editing code in my real project?
Is it possible just to make the generic repository hit a list of Tentity rather than the database or in memory database.
Thanks for reading and look forward to the advice.
My answer should/could be a comment, maybe. Because I would like to tell you: do not do it. Do not waste your time to create a fake of the data to be returned from persistence. And do not invest your time to: take the data from a client and put them into some virtual DB in memory.
You need to be sure, that your services (consuming repository) can really serialize/render the real data. And deserialize/persist the changed. And that would really require a real data.
Rather spend some time to create scripts, which will populate the test data. The data which you can expect in your tests: when doing Business validation, Service data serialization...
Also take a look here: Ayende: NHibernate Unit Testing. An extract:
When using NHibernate we generally want to test only three things,
that properties are persisted, that cascade works as expected and that
queries return the correct result. In order to do all of those, we
generally have to talk to a real database, trying to fake any of those
at this level is futile and going to be very complicated.
A note: some time ago, we used to wrap all the tests in Transaction Begin() and Rollback(). Which was looking good. But we realized, that lot of stuff - because of missing Flush() call - was not tested all the way down (e.g. setting not-null).
I have to agree with Radim, that unit testing nhibernate code by mocking the nhibernate functionality in most cases in not what you want to do.
Unless you want to test complex business logic which is based on data you retrieve via nhibernate, then this is perfectly fine.
But to test if your mappings, data retrieval and persistence works fine, you have to test against a real database.
If you target MSSQL Server, I would not use another type of database. Instead there is SQL Express which has all features of the real server.
MSSQL Express can optionally be installed with local DB. This will allow you to load mdf files via connection string which will more or less instantiate an instance of MSSQL Server...
I used that for integration testing and it works really nice.
Create a data base file in your unit test project
Depending on your model (code first/db first) let nhibernate create the scheme, otherwise simple populate the scheme into that database file
Add the file to the deployment items of your test settings so that the file gets copied to the test target directory
Generate a connection string which uses the copied database file.
Example connection string: Data Source=(LocalDB)\v11.0;AttachDbFileName=[whateverthepathis]\DatabaseFileName.mdf;InitialCatalog=DatabaseName;Integrated Security=True;MultipleActiveResultSets=True
Run your tests
This way your tests will run with an empty database every time and you will have reproduceable integration tests without the need of a real server where you would have to create a DB or reset it everytime...
There are couple of ways to achieve this,
Use a real database for testing using scripts to setup and revert the database, but with this approach it would take time and effort to create and maintain these scripts when there are changes to the database
Use a real database, and use transaction scope for testing (starting the transaction persist, and do the test and once all is done only rolling back the transaction), this is a really good approach and I use this for a large scale project. However one problem with this is it takes a lot of time to run tests (I have around 3500 tests and it takes total of 40 minutes to run them all)
Use a fake repositories (having an internal list of entities) for business logic test and use actually repositories to verify the mappings. This approach would require additional effort to create and maintain fake repositories. The same tests executed on actual repositories can be executed on fake repositories to verify fakes are working. With this approach test execution would be faster.
I started an internship where my job consists of seting up unit tests on some projects.
I have already implemented dozens of tests and now I want to create a mechanism which restores the original database after or before each run.
I have the necessary scripts to create, populate and drop my database, but I don't want to call this mechanism each test classes, but instead call it just once.
Is it possible to create a ClassIniatialize() who is called only once when I start one or all the tests?
EDIT
All I/you need is to use AssemblyInitialize() and AssemblyCleanUp() and all is resolved, ;)
All I/you need is to use AssemblyInitialize() and AssemblyCleanUp() and all is resolved, ;)
You could start a transaction on TestInitialize and do a rollback on TestCleanup. This approach would allow you to avoid data changes to perdure outside the scope of individual tests.
[TestInitialize()]
public void Initialize()
{
//Init DB Transaction
}
[TestCleanup()]
public void Cleanup()
{
//Rollback DB Transaction, database returns to the initial state
}
You'll need to have a previously populated test database to run test on.
As a side note, database access should be tested on integration test and not on unit test. Adding external dependencies is against unit test definition.
Check also about the possibility of using in memory SQLite, will be more suitable for testing database access that a production db engine.
You could create a base class and decorate a method in that base class with the ClassInitialize-attribute.
After that you just need to make sure that all your test-classes inherit from this base class.
ClassInitilize method will be called once for every testclass you have. You may implement it just once in a base testclass for example, but it will still be called once for every test class.
I use NUnit integration tests.
I am trying to test to make sure that user can't create account with existing email. (test#example.com)
I need to have test data in the database (account with test#example.com email).
I can create this account in the test function, or in the sql script (and run it before integration tests).
Where is the better place to create this test data?
Neither option is wrong but there are a number of ways to extend and solidify your strategy:
Mocking which goes hand-in-hand with TDD
Generation of db test data with tools like RedGate's Sql Data Gen
Creation of pluggable data providers (IoC/DI) where you can swap between test and production
None of these solutions are mutually exclusive. I would recommend the last item especially (pluggable provider) and then a choice between object mocking or faux but quality db test data.
Your best bet is to look into Dependency Injection and Mocking frameworks. This way you can swap out data providers with mocked data providers and use the data that fits your needs for the specific test.
If you're using NHibernate or similar, you can always recreate your db schema before each test(fixture).
In a situation like you describe, I would prefer to create the account in the test function.
A unit test should be as self contained as possible. Also, it helps to be able to understand what you are testing, if you can see all the data required for the test in one place.
Here's a totally made up example that should illustrate:
[Test]
public void Test_CannotCreateDuplicateEmail()
{
// Arrange
CreateAccount("test#example.com"); // OK
// Act
try
{
CreateAccount("test#example.com");
// If control arrives here, then the test has failed.
Assert.Fail();
}
// Assert
catch(AccountException ex)
{
// Assert that the correct exception has been thrown.
Assert.AreEqual("Failed", ex.Message);
}
}
I'm writing unit-tests for an app that uses a database, and I'd like to be able to run the app against some sample/test data - but I'm not sure of the best way to setup the initial test data for the tests.
What I'm looking for is a means to run the code-under-test against the same database (or schematically identical) that I currently use while debugging - and before each test, I'd like to ensure that the database is reset to a clean slate prior to inserting the test data.
I realize that using an IRepository pattern would allow me to remove the complexity of testing against an actual database, but I'm not sure that will be possible in my case.
Any suggestions or articles that could point me in the right direction?
Thanks!
--EDIT--
Thanks everyone, those are some great suggestions! I'll probably go the route of mocking my data access layer, combined with some simple set-up classes to generate exactly the data I need per test.
Here's the general approach I try to use. I conceive of tests at about three or four levels:: unit-tests, interaction tests, integration tests, acceptance tests.
At the unit test level, it's just code. Any database interaction is mocked out, either manually or using one of the popular frameworks, so loading data is not an issue. They run quick, and make sure the objects work as expected. This allows for very quick write-test/write code/run test cycles. The mock objects serve up the data that is needed by each test.
Interaction tests test the interactions of non-trivial class interactions. Again, no database required, it's mocked out.
Now at the integration level, I'm testing integration of components, and that's where real databases, queues, services, yada yada, get thrown in. If I can, I'll use one of the popular in-memory databases, so initialization is not an issue. It always starts off empty, and I use utility classes to scrub the database and load exactly the data I want before each test, so that there's no coupling between the tests.
The problem I've hit using in-memory databases is that they often don't support all the features I need. For example, perhaps I require an outer join, and the in-memory DB doesn't support that. In that case, I'll typically test against a local conventional database such as MySQL, again, scrubbing it before each test. Since the app is deployed to production in a separate environment, that data is untouched by the testing cycle.
The best way I've found to handle this is to use a static test database with known data, and use transactions to ensure that your tests don't change anything.
In your test setup you would start a transaction, and in your test cleanup, you would roll the transaction back. This lets you modify data in your tests but also makes sure everything gets restored to its original state when the test completes.
I know you're using C# but in the Java World there's the Spring framework. It allows you to run database minipulations in a transaction and after this transaction, you roll this one back. This means that you operate against a real database without touching the state after the test finishes. Perhaps this could be a hint to further investigation in C#.
Mocking is of cause the best way to unit test your code.
As far as integration tests go, I have had some issues using in-memory databases like SQLite, mainly because of small differences in behaviour and/or syntax.
I have been using a local instance of MySql for integration tests in several projects. A returning problem is the server setup and creation of test data.
I have created a small Nuget package called Mysql.Server (see more at https://github.com/stumpdk/MySql.Server), that simply sets up a local instance of MySql every time you run your tests.
With this instance running you can easily set up table structures and sample data for your tests without being concerned of either your production environment or local server setup.
I don't think there is an easy way to finish this. You just have to create those Pre-Test sql setup scripts and post-test Tear-down scripts. Then you need trigger those scripts for each run. A lot of people suggest SQLLite for unit test setup.
I found it best to have my tests go to a different db so I could wipe it clean and put in the data I wanted for the test.
You may want to have the database be something that can be set within the program, then your test can tell the classes to change the database.
This code clears all data from all user's tables in MS SQL Server:
private DateTime _timeout;
public void ClearDatabase(SqlConnection connection)
{
_timeout = DateTime.Now + TimeSpan.FromSeconds(30);
do
{
SqlCommand command = connection.CreateCommand();
command.CommandText = "exec sp_MSforeachtable 'DELETE FROM ?'";
try
{
command.ExecuteNonQuery();
return;
}
catch (SqlException)
{
}
} while (!TimeOut());
if (TimeOut())
Assert.Fail("Fail to clear DB");
}
private bool TimeOut()
{
return DateTime.Now > _timeout;
}
If you are thinking about a real database usage, then mostlikely we're talking integration tests here. I.e tests, which check app behavior as a composition of different components contrary to unit tests, where components are supposed to be tested in isolation.
Having the testing scope defined, I wouldn't recommend using things like in-memory databases or mocking libraries as the other authors suggested. The problem is that usually there is a slightly different behavior or reduced set of features for in-memory databases and there is no database at all with mocking, therefore you'll be testing some other application in general sense and not the one you'll be delivering to your customers.
I'd rather suggest to minimize the amount of integration tests by covering just a crucial parts of your logic leaving the rest for unit testing, while using a real database with the setup as close to the production one as possible. Test runs could be too slow and a real pain if there are a lot of integration ones.
Also you might use some tricks to optimize the speed of your tests execution:
Split tests to Read and Write in regard to the data mutations they introduce and run the former ones in parallel and without any cleanup. (E.g HTTP GET requests are safe to be run in parallel if the system under test is a webapp and tests are more like end-to-end);
Use the only insert/delete script for all the data and optimize as much as possible. You might find Reseed library I'm developing currently helpful. It's able to generate both insert and delete scripts for you. So basically what you asked for. Or check out Respawn which could be used for database cleanup;
Use database snapshots for the restore, which might be faster than full insert/delete cycle;
Wrap each test in transaction and revert it afterwards (this one is also not 100% honest and somewhat fragile);
Parallelize your tests by using a pool of databases instead of the only. Docker and TestContainers could be suitable here;