Develop Reference

Effort- FirstOrDefault returns null when Faking Database

I'm trying to create some unit tests for my project, after much digging around I found Effort, the idea is great, it mocks the database instead of the dealing with faking the DBContext which by the way is really hard to get it right when using a complex schema.
However I'm trying to get the Email of a user after I specifically added it to the in-memory database create by Effort, here is the code
MyContext contextx = new MyContext(Effort.DbConnectionFactory.CreateTransient());
var client = new Client
{
ClientId = 2,
PersonId = 3,
Person = new Person
{
PersonId = 3,
EMail = "xxxxx#gmail.com"
}
};
contextx.Client.Add(client); //<-- client got added, I checked it and is there
var email = contextx.Client.Select(c => c.Person.EMail).FirstOrDefault();
In the last line above I can't make it to return the email xxxx#gmail.com instead it always returns null.
Any ideas?

Answering Your Direct Question
For the specific question you asked, I would suggest two things:
Take a look at contextx.Client.ToArray() and see how many members you really have in that collection. It could be that the Client collection is actually empty, in which case you'll indeed get null. Or, it could be that the first element in the Client collection has a null value for EMail.
How does the behavior change if you call contextx.SaveChanges() before querying the Client collection on the DbContext? I'm curious to see if calling SaveChanges will cause the newly inserted value to exist in the collection. This really shouldn't be required, but there might be some strange interaction between Effort and the DbContext.
EDIT: SaveChanges() turns out to be the answer.
General Testing Suggestions
Since you tabbed this question with the "unit-testing" tag, I'll offer some general unit testing advice based on my ten years spent as a unit testing practitioner and coach. Unit testing is about testing various small parts of your application in isolation. Typically this means that unit tests only interact with a few classes at once. This also means that unit tests should not depend on external libraries or dependencies (such as the database). Conversely, an integration test exercises more parts of the system at once and may have external dependencies on things like databases.
While this may seem like a quibble over terminology, the terms are important for conveying the actual intent of your tests to other members of your team.
In this case, either you are really wanting to unit test some piece of functionality that happens to depend on DbContext, or you are attempting to test your data access layer. If you're trying to write an isolated unit test of something that depends on the DbContext directly, then you need to break the dependency on the DbContext. I'll explain this below in Breaking the Dependency on DbContext below. Otherwise, you're really trying to integration test your DbContext including how your entities are mapped. In this case, I've always found it best to isolate these tests and use a real (local) database. You probably want to use a locally installed database of the same variety you're using in production. Often, SqlExpress works just fine. Point your tests at an instance of the database that the tests can completely trash. Let your tests remove any existing data before running each test. Then, they can setup whatever data they need without concern that existing data will conflict.
Breaking the Dependency on DbContext
So then, how do you write good unit tests when your business logic depends on accessing DbContext? You don't.
In my applications that use Entity Framework for data persistence, I make sure access to the DbContext is contained within a separate data access project. Typically, I will create classes that implement the Repository pattern and those classes are allowed to take a dependency on DbContext. So, in this case, I would create a ClientRepository that implements an IClientRepository interface. The interface would look something like this:
public interface IClientRepository {
Client GetClientByEMail(string email);
}
Then, any classes that need access to the method can be unit tested using a basic stub / mock / whatever. Nothing has to worry about mocking out DbContext. Your data access layer is contained, and you can test it thoroughly using a real database. For some suggestions on how to test your data access layer, see above.
As an added benefit, the implementation of this interface defines what it means to find a Client by email address in a single, unified place. The IClientRepository interface allows you to quickly answer the question, "How do we query for Client entities in our system?"
Taking a dependency on DbContext is roughly the same scale of a testing problem as allowing domain classes to take a dependency on the connection string and having ADO.Net code everywhere. It means that you have to create a real data store (even with a fake db) with real data in it. But, if you contain your access to the DbContext within a specific data access assembly, you'll find that your unit tests are much easier to write.
As far as project organization, I typically only allow my data access project to take a reference to Entity Framework. I'll have a separate Core project in which I define the entities. I'll also define the data access interfaces in the Core project. Then, the concrete interface implementations get put into the data access project. Most of the projects in your solution can then simply take a dependency on the Core project, and only the top level executable or web project really needs to depend on the data access project.

Writing unit tests when the repository holds your most important code

I have an EAV system that stores entities in a SQL database, fetches them out and stores them in the cache. The application is written using the repository pattern because at some point in the future we will probably switch to using a NOSQL database for serving some or all of the data. I use Ninject to fetch the correct repository at runtime.
A large part of the system's functionality is around storing, retrieving and querying data in an efficient and timely manner. There is not a huge amount of functionality that doesn't fall into the realm of data access or user interface.
I've read up on unit testing - I understand the theory but haven't put it into practice yet for a few reasons:
An entity consists of fieldsets, fields, values, each of which has many properties. Creating any large number of these in code in order to test would require a lot of effort.
Some of the most crucial parts of my code are in the repositories. For instance all of the data access goes through a single highly optimised method that fetches entities from the database or cache.
Using a test database feels like I'm breaking one of the key tenets of unit testing - no external dependencies.
In addition to this the way the repositories are built feels like it's tied into how the data is stored in SQL. Entities go in one table, fields in another, values in another etc. So I have a repository for each. It is my understanding though that in a document store database that the Entity, its field and values would all exist as a single object, removing the need for multiple repositories. I've considered making my data access more granular in order to move sections of code outside of the repository, but this would compound the problem by forcing me to write the repository interfaces in a way that is designed for retrieving data from SQL.
Question: Based on the above, should I accept that I cannot write unit tests for large parts of my code and just test the things I can?

should I accept that I cannot write unit tests for large parts of my code?
No, you shouldn't accept that. In fact, this is never the case - with enough effort, you can unit test pretty much anything.
Your problem boils down to this: your code relies upon a database, but you cannot use it, because it is an external dependency. You can address this problem by using mock objects - special objects constructed inside your unit test code that present themselves as implementations of database interfaces, and feed your program the data that is required to complete a particular unit test. When your program sends requests to these objects, your unit test code can verify that the requests are correct. When your program expects a particular response, your unit tests give it the response as required by your unit test scenario.
Mocking may be non-trivial, especially in situations when requests and responses are complex. Several libraries exist to help you out with this in .NET, making the task of coding your mock objects almost independent of the structure of the real object. However, the real complexity is often in the behavior of the system that you are mocking - in your case, that's the database. The effort of coding up this complexity is entirely on you, and it does consume a very considerable portion of your coding time.

It appears, when you say a "unit test", you really mean an "integration test". Because in a unit-test-world there is no database. If you expect to get or insert some data into the external resource, you just fake it (using mocks, stubs, fakes, spies etc)
should I accept that I cannot write unit tests for large parts of my
code and just test the things I can?
Hard to tell without seeing your code, but it sounds like you can easily unit test it. This is based on your use of the interfaces and the repository pattern. As long as a unit test is independent from other tests, tests only a single piece of functionality, small, simple, does not depend on any external resources - you are good to go.
Do not confuse this with integration and other types of testing. Those may involve real data and may be a bit trickier to write.

If you're using the proper Repository pattern testing is easy, because
Business layer knows ONLY about the repository interface which deals ONLY with objects known by the layer and doesn't expose ANYTHING related to the actual Db (like EF). Here's where you're using fakes implementing the repository interface.
Testing the db access means you're testing the Repo implementation, you get test objects in and out. It's natural for the Repository to be coupled with the db.
Test for repo should be something like this
public void add_get_object()
{
var entity=CreateSomeTestEntity();
_repo.Save(entity);
var loaded=_repo.Get(entity.Id);
//assert those 2 entities are identical, but NOT using reference
// either make it implement IEquatable<Entity> or test each property manually
}
These repo tests can be reused with ANY repository implementation: in memory, EF, raven db etc, because it shouldn't matter the implementation, it matters that the repo does what it's required to do (saving/loading business objects).

How should I unit test a repository method implemented with Entity Framework?

I have a method like this in my repository layer:
public IEnumerable<User> GetActiveUsers()
{
return dbContext.Users
.Where(u => u.IsActive)
.OrderBy(u => u.Name)
.ToList();
}
Should I unit test this method by mocking the DbContext or should I test it with an actual database?
I know it stops being a "unit" test when I use an actual database, but I don't see the value in mocking the DbContext to test my repository methods which are thin in logic and usually just calls EF's method directly.
And if I have to use an actual database, is there any standard strategy to populate test data in the database so tests run independently and do not alter any state in the database?

Probably not what you want to hear, but you don't want to mock DbContext. I know it's been done all the time and in EF6 it's even been made easier than before. There are yet more interfaces and virtual methods available to implement mock objects. Technically it's not hard.
It's the behavior that matters.
Even in your small example there is a possible catch. A mock DbSet would do case-sensitive sorting. A connected DbSet would receive sorted data from the database and many database collations happen to be case-insensitive. So a unit test could produce different results than an integration test, even in this seemingly insignificant case.
The differences between in-memory and connected LINQ are overwhelming. Personally, I do only integration tests for anything that involves any LINQ to Entities query. It's just too easy to create a mock object graph that would look different if EF would have built it. In my service methods I sometimes compose pretty complex queries, maybe involving Includes, maybe deliberately omitting null guards, knowing that the statement is translated into SQL, maybe involving some lazy loading or relying on relationship fixup. I have to be aware of entity states, context lifespans, validation that kicks in when saving changes, concurrency ... I just don't believe green tests when it's all mocked.
Of course there's enough business logic left to test with pure unit test. Once you can make the assumption that correct objects are available (because you test that separately in integration tests) you can mock them and unit test their behavior in-memory.

On the other hand: what additional information would you get by setting up a database and testing against that?
I would simply mock the DbContext because it will be easiest to maintain and because there is no value in essentially unit testing the Entity-Framework's calls against a database.
The thing you want to be testing is whether or not your LINQ queries return the data from the datasource. However those LINQ queries are translated to SQL queries by EF is something you won't bother with.
You can consider it as an additional integration test if you really want to test your external dependencies but EF by itself is already very reliable so I doubt that this would be useful in any way.

If IEnumerable GetActiveUsers is implemented (and it should) as an interface, and the code you posted is a concrete class which implement the interface. Then you would normally use the mock framework to mock the interface you implement and return a result set you set up.
As Jeroen mentioned, you normally don't need to unit test what entity framework doing. Unit test is only test your logic, not the other library (EF here) logic.

Something to keep in mind when writing a Mock for your DBContext and executing your LINQ Queries is that they are running against the mock context object as LINQ to OBJECTs, not LINQ to Entities... (EF6)...
I personally would put your time into testing the behaviors. Julie Lerman has a series # pluralsight and MSDN Mag online covering how to test the dbContext if you must.

Is there any reason to use Mock objects when unit testing an Entity Framework DAL?

From what I understand, Mocking is used to remove dependency of another service so that you can properly test the execution of business logic without having to worry about those other services working or not.
However, if what you are testing IS that particular service (i.e. Entity Framework), Implementation-style unit tests against a preset test database are really the only tests that will tell you anything useful.
Am I missing anything? Does mocking have any place in my testing of an Entity Framework DAL?

You are correct in your assertion about mocking:
Mocking is used to remove dependency of another service so that you
can properly test the execution of business logic without having to
worry about those other services working or not.
In my words: the idea behind unit testing is to test a single code path through a single method. When that method hands execution over to another object there is a dependency. When control passes to an unmocked dependency you are no longer unit testing, but are instead integration testing.
Data access layer testing is typically an integration test. As you speculate you can utilize a predictable data set to ensure your DAL is returning results as expected. I would not expect a DAL to have any dependencies which would require mocking. Your testing that the values returned by your DAL match what you would expect given your dataset.
All of the above said it is not your responsibility to test the Entity Framework. If you find yourself testing the way EF works and are not creating tests about your specific DAL implementation then you are writing the wrong tests. Put another way: you test your code, let someone else test theirs.
Finally, three years ago I asked a similar question which elicited answers which greatly improved my understanding of testing. While not identical to your question I'd recommend reading through the responses.

In my opinion, mocking objects has nothing to do with the layer you are about to test. You have a component that you want to test and it has dependencies (that you can mock). So go ahead and mock.

One can assume EF works. You would test your code that interacts with EF. In this case, you fake EF in order to test your interacting code.
You basically shouldn't be testing EF. Leave that to Microsoft.

One thing you might consider is that EF can work with a local file and not your actual repository and you can switch between the two with connection strings. This example would create the .sdf file in an AppData folder or in the bin folder if it's a console application.
<connectionStrings>
<add name="SecurityContext"
connectionString="Data Source=|DataDirectory|YourDBContext.sdf"
providerName="System.Data.SqlServerCe.4.0" />
I like this when I'm starting a project or testing. You can load the DB with data and such and presto: EF has a mocked DB for you to run tests against without touching production data.

Differences between database/entity framework and in memory lists when mocking in unit tests

I have been doing a lot of unit testing lately with mocking. The one thing that strikes me as a bit of a problem are the differences between querying against an in memory list (via a mock of my repository) and querying directly against the database via entity framework.
Some of these situations might be:
Testing a filter parameter which would be case insensitive against a database but case sensitive
against an in memory collection leading to a false fail.
Linq statements that might pass against an in memory collection but would fail against entity framework because they arent supported leading to a false pass.
What is the correct way to handle or account for these differences so that there are not false passes or fails in tests? I really like mocking as it makes things so much quicker and easier to test. But it seems to me that the only way to get a really accurate test would be to just test against a the entity framework/database environment.

Besides the unit tests you do you should also create integration tests which run against a real database setup as encountered in production.
I'm not an expert for EF but with NHibernate for example you can create a configuration which points to an in-memory instance of SQLite where you then run your quick tests against (i.e. during a development cycle where you want to get through the test suite as fast as possible). When you want to run your integration tests against a real database you simply change the NHibernate config to point to a real database setup and run the same tests again.
Would be surprising if you could not achieve something similar with EF.

You can use DevMagicFake, this framework will fake the DB for you and can also generate data so you can test your application without testing the DB

First and most important is you can define any behavior data within your mock. Second is speed. From unit testing perspective testing speed counts. Database connections are bottleneck most of time so that's why you mock it with tests.
To implement testing properly you need to work on your overall arch first.
For instance to access data layer I use repository pattern sometimes. It's described really good in Eric Evans DDD book.
So let's say if your repository is defined as below
interface IRepository: IQueryable, ICollection
you can handle linq queries pretty straightforward.
Further reading Repository

I would make my mocks more granular, so that you don't actually query against a larger set in a mock repository.
I typically have setters on my mock repository that I set in each test to control the output of the mocked repository.
This way you don't have to rely on writing queries against a generic mock, and your focus can be on testing the logic in the method under test