Develop Reference

How to solve "there is already an open datareader associated with this connection"

The main problem is that when the web app is launched to the internet, when the load is high an exception is raised telling that there is already an opened data reader.
The following are the specs we use:
Entityframework 5.0.0
MySQL database
Is there a way of solving this problem without the using(){} block? Main problem of this approach is that when closed the using block I can't expand foreign key relations of entityframework objects inside the html view.
I also attach some source code, showing how we keep a single database context through the whole application
public abstract class AbstractService
{
public Entities db_model
{
get
{
return DbContext.Instance.db_model;
}
}
}
public class DbContext
{
public Entities db_model = new Entities();
private static DbContext _dbContext;
public static DbContext Instance
{
get
{
if(_dbContext == null)
{
_dbContext = new DbContext();
}
return _dbContext;
}
}
}

This answer is specifically related to the issue mentioned in the question about using the loaded entities in an ASP.NET View. The question asks about a way of solving this problem without a using block or disposing of the DbContext, however I am suggesting doing exactly this.
The reason being that it's generally desirable not to use Entity Framework objects in the ASP.NET Views because those objects are a lot more more than just plain POCO objects; they hide logic which allows them to act as a proxy to the underlying database, so they have a hidden dependency on the state of the DbContext which created them.
Here's a contrived example using EF models for Employee and Department with a DbContext:
public class CompanyDbContext : DbContext
{
public DbSet<Department> Departments { get; set; }
public DbSet<Employee> Employees { get; set; }
}
public class Department
{
public long Id { get; set; }
public virtual ICollection<Employee> Employees { get; set; }
}
public class Employee
{
public long Id { get; set; }
public long DepartmentId { get; set; }
public virtual Department Department { get; set; }
}
If these were used in an ASP.NET application, I would create some separate models which aren't tied to Entity Framework, to be used by ASP.NET. For example:
public class DepartmentModel
{
public long Id { get; set; }
public List<EmployeeModel> Employees { get; set; }
}
public class EmployeeModel
{
public long Id { get; set; }
public long DepartmentId { get; set; }
}
A few considerations:
According to the MSDN docs, "A DbContext represents a combination of the UnitOfWork and Repository patterns" - https://learn.microsoft.com/en-us/dotnet/api/system.data.entity.dbcontext?redirectedfrom=MSDN&view=entity-framework-6.2.0 - Therefore the DbContext should be short lived as far as possible.
When loading data from the context, related entities can be retrieved using DbSet<>.Include() - https://learn.microsoft.com/en-us/ef/ef6/querying/related-data
Generally speaking, it makes sense to de-couple the 'data' layer from the 'view' layer - for all kinds of reasons, some of which are listed here: https://learn.microsoft.com/en-us/aspnet/web-api/overview/data/using-web-api-with-entity-framework/part-5 -- this involves mapping between the EF objects and the POCO Models.
The logic which is used to query the DbContext would query the data using EF, and return that data using POCO models so that only logic which deals directly with DbContext has any involvement with the EF objects. For example:
public List<DepartmentModel> GetAllDepartments()
{
using (var ctx = new CompanyDbContext())
{
// Ensure that related data is loaded
var departments = ctx.Departments
.Include(d => d.Employees);
// Manual mapping by converting into a new set of models to be used by the Views
var models = departments
.Select(d => new DepartmentModel
{
Id = d.Id,
Employees = d.Employees
.Select(e => new EmployeeModel
{
Id = e.Id,
DepartmentId = e.DepartmentId
})
.ToList(),
})
.ToList();
return models;
}
}
Being able to use those POCO models, while requiring some extra boilerplate code, provides complete separation between the DbContext and ASP.NET, allowing the data to be used without ASP.NET Views/Controllers being concerned by the lifetime or state of the DbContext.
Sometimes this may look as if this approach violates the 'DRY' principle, however I would point out that EF objects and ViewModel objects exist to solve different problems, and it's not uncommon for the ViewModel objects to take a different shape, or even to require additional fields/attributes which wouldn't be suitable to add to the EF classes.
Lastly, the above uses 'manual' mapping, but if the mappings are really simple and straightforward, then it could make more sense to use AutoMapper instead: Cleanest Way To Map Entity To DTO With Linq Select?

ViewModels in Repository

I have read that the repository layer should not deal with ViewModels because of separation of concerns and should instead deal only with Models. This is also true for the service layer (in my case this is where my business logic is). So then the controller is left to deal with the population of the ViewModels.
I have a Model Category:
public class Category
{
public int ID { get; set; }
public int? ParentCategoryID { get; set; }
public virtual ICollection<Product> Products{ get; set; }
public virtual ICollection<CategoryName> CategoryNames{ get; set; }
}
I have a ViewModel CategoryListViewModel used when displaying all Categories
public class CategoryListViewModel
{
public int ID { get; set; }
public string Name { get; set; }
public string ParentName { get; set; }
}
My view takes IEnumerable<...CategoryListViewModel>
This is how I populate the ViewModel from the controller:
public ActionResult Index()
{
IEnumerable<CategoryListViewModel> model;
List<CategoryListViewModel> list = new List<CategoryListViewModel>();
IEnumerable<Category> categoryList = categoryService.GetAllCategoriesList(RouteData);
foreach (var item in categoryList)
{
CategoryListViewModel temp = new CategoryListViewModel()
{
ID = item.ID,
Name = categoryService.GetCategoryName(RouteData, item.ID)
};
if (item.ParentCategoryID != null)
{
temp.ParentName = categoryService.GetCategoryName(RouteData, (int)item.ParentCategoryID);
}
list.Add(temp);
}
model = list;
return View(model);
}
My service methods:
public IEnumerable<Category> GetAllCategoriesList(RouteData data)
{
LanguageService languageService = new LanguageService();
Languages langEnum = languageService.LanguageStringToEnum(languageService.DetermineSelectedLanguage(data));
IEnumerable<Category> allCategories = repository.getAllCategoriesTest();
return allCategories;
}
public string GetCategoryName(RouteData data, int categoryId)
{
LanguageService languageService = new LanguageService();
Languages langEnum = languageService.LanguageStringToEnum(languageService.DetermineSelectedLanguage(data));
return repository.GetCategoryName(langEnum, categoryId);
}
And finally my repository methods:
public IEnumerable<Category> getAllCategoriesTest()
{
return db.Category.ToList();
}
public string GetCategoryName(Languages lang, int categoryId)
{
return db.CategoryName.Where(cn => cn.CategoryID == categoryId && cn.Language == lang).Select(cn => cn.Name).FirstOrDefault();
}
This approach looks very bad to me. My Controller is not thin anymore and I am running a lot of queries for something that simple.
If I allow ViewModels in my repository I get a much cleaner solution.
My Controller method:
public ActionResult Index()
{
return View(categoryService.GetAllCategories(RouteData));
}
Service method:
public IEnumerable<CategoryListViewModel> GetAllCategories(RouteData data)
{
LanguageService languageService = new LanguageService();
Languages langEnum = languageService.LanguageStringToEnum(languageService.DetermineSelectedLanguage(data));
return repository.SelectAllCategories(langEnum);
}
And repository method:
public IEnumerable<CategoryListViewModel> SelectAllCategories(Languages lang)
{
var categories = db.Category.Include(c => c.CategoryNames).Select(names => new CategoryListViewModel
{
ID = names.ID,
Name = names.CategoryNames.Where(cn => cn.Language == lang).Select(cn => cn.Name).FirstOrDefault(),
ParentName = db.CategoryName.Where(cn => cn.Language == lang && cn.CategoryID == names.ParentCategoryID)
.Select(cn => cn.Name).FirstOrDefault()
}).ToList();
return categories;
}
This approach, although violating separation of concerns seems to be "cleaner" to me.
My question is isn't the other approach more efficient in term of queries? Also is there any other way that this could be done so as not to write heavy controller methods and not execute that many queries? It seems to me that I am missing something.

First, bear in mind that even though it has "MVC" in the name, ASP.NET MVC only very loosely implements the MVC pattern. MVC tells you to have thin controllers because the Model is an active record, which handles all the business logic, including that around querying itself. This does not apply to ASP.NET MVC. There, your Model is actually a combination of your DAL, service layer, entity and one or more view models. This means the controller inevitably must do at least a little more work than a controller in something like Ruby on Rails, if only to wire all this stuff together.
As #Liam suggested in the comments above, your best bet is factories. That way, the controller does not actually own the logic for how to map an entity to a view model. You'll of course still need to actually call the factory in your controller, but the logic remains abstracted.
Also, a proper service layer should roll up logic that would otherwise be in your controller. If you need the localized name for the category, your service should have a method that returns all the categories with their localized name already. If you're having to hit your service multiple times, that's a clear indication that you haven't provided a necessary endpoint for your application. You may need to introduce a DTO to handle this data, since the entity class may not have the appropriate properties. You'd then have a factory that maps your DTO to a view model.
Finally, for what it's worth, your repository is completely unnecessary. Just have your service interact directly with your Entity Framework context. Having a repository buys you nothing but just an additional thing you have to maintain.

Domain Model and related data (anemic domain model)

I'm currently working with ASP .NET Core 1.0 using Entity Framework Core. I have some complex calculations with data from the database and I'm not sure how to build a proper architecture using Dependency Injection without building an anemic domain model (http://www.martinfowler.com/bliki/AnemicDomainModel.html)
(Simplified) Example:
I have the following entities:
public class Project {
public int Id {get;set;}
public string Name {get;set;}
}
public class TimeEntry
{
public int Id {get;set;}
public DateTime Date {get;set;}
public int DurationMinutes {get;set;}
public int ProjectId {get;set;}
public Project Project {get;set;}
}
public class Employee {
public int Id {get;set;}
public string Name {get;set;}
public List<TimeEntry> TimeEntries {get;set;}
}
I want to do some complex calculations to calculate a monthly TimeSheet. Because I can not access the database within the Employee entity I calculate the TimeSheet in a EmployeeService.
public class EmployeeService {
private DbContext _db;
public EmployeeService(DbContext db) {
_db = db;
}
public List<CalculatedMonth> GetMonthlyTimeSheet(int employeeId) {
var employee = _db.Employee.Include(x=>x.TimeEntry).ThenInclude(x=>x.Project).Single();
var result = new List<CalculatedMonth>();
//complex calculation using TimeEntries etc here
return result;
}
}
If I want to get the TimeSheet I need to inject the EmployeeService and call GetMonthlyTimeSheet.
So - I end up with a lot of GetThis() and GetThat() methods inside my service although this methods would perfectly fit into the Employee class itself.
What I want to achieve is something like:
public class Employee {
public int Id {get;set;}
public string Name {get;set;}
public List<TimeEntry> TimeEntries {get;set;}
public List<CalculatedMonth> GetMonthlyTimeSheet() {
var result = new List<CalculatedMonth>();
//complex calculation using TimeEntries etc here
return result;
}
}
public IActionResult GetTimeSheets(int employeeId) {
var employee = _employeeRepository.Get(employeeId);
return employee.GetTimeSheets();
}
...but for that I need to make sure that the list of TimeEntries is populated from the database (EF Core does not support lazy loading). I do not want to .Include(x=>y) everything on every request because sometimes I just need the employee's name without the timeentries and it would affect the performance of the application.
Can anyone point me in a direction how to architect this properly?
Edit:
One possibility (from the comments of the first answer) would be:
public class Employee {
public int Id {get;set;}
public string Name {get;set;}
public List<TimeEntry> TimeEntries {get;set;}
public List<CalculatedMonth> GetMonthlyTimeSheet() {
if (TimeEntries == null)
throw new PleaseIncludePropertyException(nameof(TimeEntries));
var result = new List<CalculatedMonth>();
//complex calculation using TimeEntries etc here
return result;
}
}
public class EmployeeService {
private DbContext _db;
public EmployeeService(DbContext db) {
_db = db;
}
public Employee GetEmployeeWithoutData(int employeeId) {
return _db.Employee.Single();
}
public Employee GetEmployeeWithData(int employeeId) {
return _db.Employee.Include(x=>x.TimeEntry).ThenInclude(x=>x.Project).Single();
}
}
public IActionResult GetTimeSheets(int employeeId) {
var employee = _employeeService.GetEmployeeWithData(employeeId);
return employee.GetTimeSheets();
}

Do not try to solve querying problems with your aggregates. Your aggregates are meant to process commands and protect invariants. They form a consistency boundary around a set of data.
Is the Employee object responsible for protecting the integrity of an employee's timesheet? If it doesn't then this data doesn't belong into the Employee class.
Lazy-loading may be fine for CRUD models, but is usually considered an anti-pattern when we design aggregates because those should be as small and cohesive as possible.
Are you taking business decisions based on the calculated result from timesheets? Is there any invariants to protect? Does it matter if the decision was made on stale timesheet data? If the answer to these questions is no then your calculation is really nothing more than a query.
Placing queries in service objects is fine. These service objects may even live outside the domain model (e.g. in the application layer), but there is no strict rule to follow. Also, you may choose to load a few aggregates in order to access the required data to process the calculations, but it's usually better to go directly in the database. This allows a better separation between your reads & writes (CQRS).

If I understood your question correctly you can use a trick with injecting a service into your entities that helps it do the job, e.g.:
public class Employee()
{
public object GetTimeSheets(ICalculatorHelper helper)
{
}
}
Then in your service that holds the employees you would obtain it in the constructor and pass to the employee class for calculations. This service can be a Facade e.g. for getting all the data and perform initialization or whatever you really need.
As for the TimeEntries, you can get them using a function like this:
private GetTimeEntries(ICalculationHelper helper)
{
if (_entries == null)
{
_entries = helper.GetTimeEntries();
}
return _entries;
}
It depends of course on you strategy of caching and so on if this pattern fits you.
Personally I find it rather easy to work with anemic classes and have a lot of the business logic in services. I do put some in the objects, like e.g. calculating FullName out of FirstName and LastName. Usually stuff that does not involve other services. Though it's a matter of preference.

Mapping database entities to domain entities via repositories - static property or separate service?

I am working on an application where my database schema does not match up well to my domain model, and modifications to the database schema are not under my control. Because of this, I end up doing a lot of mapping in my repository calls, and I have many concrete repos to handle the mapping to and from the database (using entity framework database-first). What I am looking for is an elegant way to make calls to my repositories based on the domain entity object type. Thus far, the domain model itself is still very anemic, as we are still in the process of defining business rules.
I have seen an example elsewhere (can't recall the link) where the repository calls were passed through the domain entities via a static property, but I do not know if this will present threading issues or whether it violates any domain model principles, especially if we decide to implement DI/IoC down the road.
Here is an example of what I have so far. For the sake of brevity, I have simplified the mapping calls to the database, as the mapping in the actual application is more complex.
Repository example:
public interface IRepository<T>
{
T GetById(int id);
void Save(T entity);
}
public abstract class RepositoryFactory<T> : IRepository<T>
{
protected MyDbContext db;
protected int userId;
protected RepositoryFactory()
{
this.db = new MyDbContext();
this.userId = WebSecurity.GetCurrentUser().UserId;
}
public abstract T GetById(int id);
public abstract void Save(T entity);
}
public class CustomerRepository : RepositoryFactory<Customer>
{
public override void Save(Customer customer)
{
var entity = db.customers.FirstOrDefault(p => p.customerid == customer.Id && p.users.userid == userId);
if (entity == null) return; // TODO: Add error trapping
// Mapping (lots of reshaping happening here)
entity.customername = customer.Name;
entity.customeraddress = customer.Address;
// ...
// Save changes to DB
db.Entry(entity).State = EntityState.Modified;
db.SaveChanges();
}
public override Customer GetById(int id)
{
var entity = db.customers.FirstOrDefault(p => p.customerid == id && p.users.userid == userId);
if (entity == null) return null; // TODO: Add error trapping
return new Customer
{
Name = entity.customername,
Address = entity.customeraddress,
// ...
};
}
}
Domain Entity example:
public class Entity { public int Id { get; set; } }
public class Customer : Entity
{
public string Name { get; set; }
public string Address { get; set; }
// Does this violate any domain principles?
public static IRepository<Customer> Repository
{
get { return new CustomerRepository(); }
}
}
With this code, from my controller I can do something like:
Customer customer = Customer.Repository.GetById(id);
Instead of:
IRepository<Customer> repo = new CustomerRepository();
Customer customer = repo.GetById(id);
This seems like a very elegant solution to my problem, and it also keeps me from needing to include the Repository namespace in my controllers (MVC). If this smells funny and there is a better way to handle this, I'd love to learn. The only other thing I can think of is creating a separate crud service to handle my repository calls, but I imagine for that I would need a dictionary or hash table to map my concrete repos to my domain model objects, and that seems like it would be a maintenance nightmare.

I'd suggest using an inversion of control (dependency injection) container and injecting your repositories into your controllers or wherever. This way you can use them like this:
public class HomeController : Controller
{
private readonly IRepository<Customer> _customerRepository;
public HomeController(IRepository<Customer> customerRepository)
{
_customerRepository = customerRepository;
}
public ActionResult Index(int id)
{
var customer = _customerRepository.GetById(id)
return View(customer);
}
}
This way, if you ever need to replace CustomerRepository class, or need to have multiple versions (say CustomerRepositoryEntityFramework or CustomerRepositoryNHibernate) you can simply replace a new class inheriting from IRepository and your controller code will still continue to work with no change.
I recommend using Castle Windsor, or Ninject, or one of the many other IoC containers.
Also, you generally want to keep your domain entities as poco's (Plain Old CLR Object). This means separating everything out of your entities including validation, business rules, etc. and simply having only its properties. This allows you to pass your domain entities through the pipeline more easily, specially since you are in the early stages of development. This will offer you the most flexibility in the future.

How to set this from Entity Framework using Linq

A while back, I fell into the fat controller trap when I was first working with MVC. My first app used EF4 to make all the models I needed. I just put all my logic into my controller actions. While it worked, it's definitely not the best practice way. To do it the right way I started trying to build my models based on my EF objects in an effort to follow the skinny controller concept.
I've run into a roadblock in trying to find the best way to populate my models. Is there a way to run a LINQ query and have it populate your model without having to iterate through the properties to set to another class?
Something like this:
// from EF model built from database
public class MyEFObject
{
public int ID {get; set;}
public string Name {get; set;}
public string Title {get; set;}
}
public class MyObjectModel : MyEFObject
{
private Entities _data = new Entities();
public MyObjectModel(int? id)
{
if(id.HasValue) // get an existing record
{
this = _data.MyEFObjects.Where(m => m.ID.Equals(id)).Single();
// or populate right out of the query
_data.MyEFObjects.Where(m => m.ID.Equals(id))
.Select(o => new {
this.ID = o.ID,
this.Name = o.Name,
this.Title = o.Title
});
}
else
{
// set defaults for a new MyObjectModel
}
}
public void Save()
{
// takes the current object and saves changes
}
}
I know you can add a function to the EF Entity object, but I like having the option to Create or Update all tied up in one call (Save method). I don't see the point messing with a model if I have to essentially recreate what I already have from my EF Object. If I simply have a method on a class that accepts a populated object, the concept of a usable model for my views is negated.

Slauma is right. LINQ to Entities won't accept it. I tried a couple of versions of what was posted and I only found my self with a kludgy mess. I got it to the point where I could set instance values, but by then EF wouldn't register a change had been made and defeating the whole purpose. There may be a way to do this, but as of now, the steps to make it work seem to be overkill.
I ended up with something like this:
public class MyObjectModel : MyEFObject
{
public void Save(int? id, MyObjectModel model)
{
var data = new Entities();
MyEFObject foo;
if(id.HasValue)
{
foo = data.MyEFObjects.Where(e => e.ID.Equals(id.Value)).Single();
}
else
{
foo = new MyEFObject();
}
foo.Name = model.Name;
foo.Title = model.Title;
if(!id.HasValue)
{
data.MyEFObjects.AddObject(foo);
}
data.SaveChanges();
}
}
I didn't want to have to work with two instances of my model, but it works and I have my lean controller action.

What you could do is have a domain model, ef model and and adapter. I think this keeps the code pretty clean and nicely separates the mapping logic.
//Domain model to decouple from EF
public class MyObjectModel
{
public int ID {get; set;}
public string Name {get; set;}
public string Title {get; set;}
}
//Auto generated Entity Framework class
public class MyEFObject
{
public int ID {get; set;}
public string Name {get; set;}
public string Title {get; set;}
}
//Adapter responsible for mapping your data to your domain model
public class MyObjectModelAdapter : MyEFObject
{
public MyObjectModelAdapter(MyEFObject entity)
{
if(entity != null)
{
this.ID = entity.ID;
this.Name = entity.Name;
this.Title = entity.Title;
}
else
{
// set defaults for a new MyObjectModel
}
}
}
Then the basic usage would be:
new Entities().MyEFObjects.ToList().Select(x => new MyObjectModelAdapter(x));
OR
new MyObjectModelAdapter(new Entities().MyEFObjects.FirstOrDefault(x => x.ID.Equals(objectId)));
If you specifically require a list of MyObjectModel then you could do the following:
new Entities().MyEFObjects.ToList().Select(x => new MyObjectModelAdapter(x) as MyObjectModel);
OR
new MyObjectModelAdapter(new Entities().MyEFObjects.FirstOrDefault(x => x.ID.Equals(objectId)) as MyObjectModel;
Of course you don't want to chain your entity context together like that, it is just to show usage.