I have a asp.net application using entity framework.
I have these two models:
public class CustomerModel
{
public int Id{get;set; }
[Required]
public string Name {get;set; }
[Required]
public string Surname { get; set; }
[Required]
[Range(18,110)]
public uint Age { get; set; }
[Required]
public virtual AdressModel Adress { get; set; }
[Required]
public DateTime Created { get; set; }
}
and
public class AdressModel
{
public int Id { get; set; }
[Required]
public int HouseNumber { get; set; }
[Required]
public string Town { get; set; }
[Required]
public string ZipCode { get; set; }
[Required]
public string Country { get; set; }
}
And a dbcontext class that looks like this:
public class DemoContext : DbContext
{
public DbSet<CustomerModel> Customers { get; set; }
public DbSet<AdressModel> Adresses { get; set; }
protected override void OnConfiguring(DbContextOptionsBuilder options)
{
options.UseLazyLoadingProxies();
options.UseSqlite(#"Data Source=/home/ask/RiderProjects/Parkz/identifier.sqlite");
}
}
and then I have a controller that just needs to load all the customers that I have in my database, and their adresses.
For that purpose I have this:
public IActionResult sendhere()
{
List<CustomerModel> customers = new List<CustomerModel>();
using (var db = new DemoContext()) {
customers = db.Customers
.Include(c => c.Adress)
.ToList();
}
return Content("hi");
}
Which I have tried to debug a bit.
The issue is that as soon as I exit my "using" block, all the related adress objects only consist of this error:
System.InvalidOperationException: An error was generated for warning 'Microsoft.EntityFrameworkCore.Infrastructure.LazyLoadOnDisposedContextWarning': An attempt was made to lazy-load navigation 'CustomerModelProxy.Adress' after the associated DbContext was disposed. This exception can be suppressed or logged by passing event ID 'CoreEventId.LazyLoadOnDisposedContextWarning' to the 'ConfigureWarnings' method in 'DbContext.OnConfiguring' or 'AddDbContext'.
at Microsoft.EntityFrameworkCore.Diagnostics.EventDefinition`2.Log[TLoggerCategory](IDiagnosticsLogger`1 logger, TParam1 arg1, TParam2 arg2)
at Microsoft.EntityFrameworkCore.Diagnostics.CoreLoggerExtensions.LazyLoadOnDisposedContextWarning(IDiagnosticsLogger`1 diagnostics, DbContext context, Object entityType, String navigationName)
at Microsoft.EntityFrameworkCore.Infrastructure.Internal.LazyLoader.ShouldLoad(Object entity, String navigationName, NavigationEntry& navigationEntry)
at Microsoft.EntityFrameworkCore.Infrastruc…
Which is because I exited the context.
How do I still get to access the adresses of my customers, even though I exited the Dbcontext, so I can return them to a view
My general advice when working with EF is that entities shouldn't be referenced outside of the scope of their DbContext. You certainly can work with detached entities, but you have to respect that they are provided in an "as-is" state from the moment they leave the scope of the DbContext. For that reason I recommend that anything passed outside of the scope of the DbContext should be a POCO ViewModel or DTO class to avoid confusing whether an entity class is actually a functional, attached entity representing data domain state, or a detached shell.
Option 1: Deal with DTO/ViewModels.
public IActionResult sendhere()
{
using (var db = new DemoContext()) {
var customerDTOs = db.Customers
.Select(c => new CustomerDTO
{
// fill in the fields you want here.
Addresses = c.Addresses.Select(a => new AddressDTO
{
// fill in the address fields you want here.
}).ToList()
}).ToList();
return View(customerDTOs);
}
}
You can alternatively leverage Automapper, set up the desired projection rules and use ProjectTo<CustomerDTO>(config) to replace the use of Select() above.
When leveraging projection, you do not need lazy loading proxies at all. This has arguably become the defacto recommended approach for EF.
The advantages of the projection method are that these DTOs (or ViewModels) cannot be confused with being entities. The issue with detached entities is that where you have methods in your code that might accept entities, these methods might expect to get entities and access members that aren't loaded. If they are attached and within the scope of a DbContext, those members can be lazy-loaded (not ideal for performance reasons, but functional) however if they are detached you get errors or NullRefExceptions. The other advantage of projection is the payload of data being pulled from the database and sent to the view logic or end consists of just the data needed.
Option 2: Don't de-scope the DbContext. With projects like ASP.Net MVC web applications, you can leverage an IoC Container to provide dependency injection into your Controllers. In this way you can set up the DbContext to be injected into the constructor with a lifetime scope set to the Request. In this way, for any given request, all services/classes you might call can be managed by the container and have access to the DbContext.
public class SomeController
{
private readonly DemoContext _context;
public SomeController(DemoContext context)
{
_context = context ?? throw new ArgumentNullException(nameof(context));
}
public IActionResult sendhere()
{
var customers = _context.Customers
.Include(c => c.Address)
.ToList();
return View(customers);
}
}
This can be combined with Option 1 to avoid needing to scope a DbContext with each request/action and better facilitate situations where you may want to make multiple calls against the DbContext and ensure the same context instance is used. For IoC containers there are a number of different ones available, and I believe ASP.Net Core comes with a default one, though I personally use and recommend Autofac. It has good documentation and examples on how to wire it up with MVC projects.
Option 3: Eager load everything you're going to need to reference. The example you provided should actually work, but your real code is likely missing an eager-load (.Include()) for the desired relationship given your example doesn't attempt to do anything with the Customers collection you load.
If your code does:
List<CustomerModel> customers = new List<CustomerModel>();
using (var db = new DemoContext()) {
customers = db.Customers
.Include(c => c.Address)
.ToList();
}
var firstAddressId = customers.FirstOrDefault()?.Address.Id;
This should work as Addresses was eager loaded. However, if you had:
List<CustomerModel> customers = new List<CustomerModel>();
using (var db = new DemoContext()) {
customers = db.Customers
.ToList();
}
var firstAddressId = customers.FirstOrDefault()?.Address.Id;
... without the Include(c => c.Address), then it would fail with that error.
With EF Core if you are going to want to return entities outside of the scope of a DbContext and you have lazy loading proxies enabled, you will want to temporarily turn off the proxy creation to avoid proxy errors. In this case anything you don't eager load will be left #null or default.
List<CustomerModel> customers = new List<CustomerModel>();
using (var db = new DemoContext()) {
db.ContextOptions.ProxyCreationEnabled = false;
customers = db.Customers
.Include(c => c.Address)
.ToList();
}
Return View(customers);
This should ensure that EF doesn't use proxies for the queries in the scope of that DbContext instance which should be avoided whenever you want to pass entities outside of the scope of the DbContext. This can be useful when you know you won't need the overhead of eager loading every reference. However, it is much better to use projection (Option 1) in this case to avoid future confusion around whether entities might actually have #null data, or merely it wasn't eager loaded.
Related
In an api I have built using dotnet 3.1, I have the Data access model for entity "Quotation" as follows.
public class QuotationDao : BaseEntity
{
public double Amount { get; set; }
public DateTime FromDate { get; set; }
public DateTime ToDate { get; set; }
public string Status { get; set; }
public Guid CustomerId { get; set; }
public CustomerDao Customer { get; set; }
public virtual ICollection<QuoteItemDao> Items { get; set; }
}
When I create a new instance of the entity 'QuoteItem', I want to increase the 'amount' attribute of the 'Quotation' entity by 1. And update the database.
The implementation of the BaseController.cs is shown below.
public class BaseController<TDao, TCreateRq, TUpdateRq, TResponse> : ControllerBase
where TDao : BaseEntity where TResponse : BaseResponse
{
private readonly IRepositoryWrapper _repositoryWrapper;
private readonly IRepositoryBase<TDao> _repository;
private readonly IMapper _mapper;
public BaseController(IRepositoryWrapper repositoryWrapper, IRepositoryBase<TDao> repository, IMapper mapper)
{
_repositoryWrapper = repositoryWrapper;
_repository = repository;
_mapper = mapper;
}
[HttpGet]
public virtual async Task<ActionResult<PagedResponse<TResponse>>> GetAll([FromQuery] StringQueryRequest request)
{
var paginationFilter = _mapper.Map<PaginationFilter>(request);
var pagedResponse = await _repository.ToPagedList
(_repository.FindAll(request.SearchString), paginationFilter);
var mapPagination = pagedResponse.MapPagination<TResponse, TDao>(_mapper);
return mapPagination.HandleToResponse();
}
[HttpGet("{id:guid}")]
public virtual async Task<ActionResult<Response<TResponse>>> GetById(Guid id)
{
var result = await GetDtoById(id);
return result.HandleGetToResponse();
}
[HttpPost]
public virtual async Task<ActionResult<Response<TResponse>>> Create([FromBody] TCreateRq request)
{
var entity = _mapper.Map<TDao>(request);
_repository.Create(entity);
await _repositoryWrapper.SaveAsync();
var baseResponse = _mapper.Map<TResponse>(entity);
var result = new Result<TResponse>(baseResponse);
return result.HandleToResponse();
}
[HttpPut("{id:guid}")]
public virtual async Task<ActionResult<Response<TResponse>>> UpdateById([FromRoute] Guid id,
[FromBody] TUpdateRq updateRq)
{
var getResult = await GetDtoById(id);
if (getResult.IsNone)
{
return getResult.HandleGetToResponse();
}
var value = await _repository.FindById(id);
//override the current DB objet from the values received from the request
//Not replacing the full object from the requests as some fields may be missing from the update RQ model (eg user password)
var entity = _mapper.Map(updateRq, value);
_repository.Update(entity);
await _repositoryWrapper.SaveAsync();
var baseResponse = _mapper.Map<TResponse>(entity);
var result = new Result<TResponse>(baseResponse);
return result.HandleToResponse();
}
[HttpDelete("{id:guid}")]
public virtual async Task<ActionResult<Response<TResponse>>> DeleteById([FromRoute] Guid id)
{
var getResult = await GetDtoById(id);
if (getResult.IsNone)
{
return getResult.HandleGetToResponse();
}
var baseResponse = getResult.ValueUnsafe();
var entity = _mapper.Map<TDao>(baseResponse);
_repository.Delete(entity);
await _repositoryWrapper.SaveAsync();
var result = new Result<TResponse>(baseResponse);
return result.HandleToResponse();
}
private async Task<Option<TResponse>> GetDtoById(Guid id)
{
var findById = await _repository.FindById(id);
return _mapper.Map<TResponse>(findById);
}
}
I cant figure out a way to implement this and I'd be grateful if someone could give some help.
This is the problem with relying on a purely Generic approach. Generic implies that regardless of what you pass in, they can be treated within the scope of the Generic class as identical. That said, as a base implementation, what you have isn't bad, but you are going to hit some limitations since a Generic implementation represents a "lowest common denominator" in terms of leveraging what EF can bring to the table.
Your issue can be addressed by structuring your controllers around aggregate roots and performing operations through those roots. The trouble you will find when working with related entities is that your controllers will need to know about these relationships in order to ensure required details are eager loaded to work with. You can leverage a Generic base class for controllers and repositories, but the actual controllers and repositories would be scoped around the aggregate root such as the Quotation. Adding quotation items would be done through its root (Quotation) rather than having something like a QuotationItemController and QuotationItemRepository. The goal of the QuotationController would be to support methods that can load a Quotation with it's respective items and have a method like AddQuotationItem. AddQuotationItem would then load the Quotation by ID, eager loading the items, validate the input for a QuotationItem, Create the QuotationItem, append it to the quote.Items, and increment the quote's Amount before saving the changes.
The key here would be to not think of fitting everything into a Generic pattern, but more-so leveraging a Generic if needed for elements of the controller or repositories etc. that are identical.
Some other tips: Which mapper are you using? If it is Automapper then consider using ProjectTo<TDestination>(config) rather than Map<TDestination>() when projecting to the view model. The benefit here is that the projection works across the IQueryable rather than introducing the overhead of fetching Entities and counting on related entities to be eager loaded or waking the performance troll that is lazy loading. This will be problematic with methods like FindAll if that is returning IEnumerable<TEntity> but if it is returning IQueryable<TEntity> then you should be set. Passing FindAll to something like ToPagedList won't save performance hits if FindAll is returning IEnumerable. When converting back from a view model to a DTO, provided your entities are tracked you should avoid using Update and just use Automapper's
mapper.Map(updateRq, value) which will copy the values from source to destination. If Destination is a tracked entity, then just call SaveChanges and EF will build an appropriate UPDATE statement for just the values that changed if any values actually changed. (Update will generate an UPDATE statement for all values even if nothing actually changed.)
That Update pattern is more a case of using:
var dao = mapper.Map<TEntity>(dto);
context.Update(dao);
context.SaveChanges();
which I would never recommend using over:
var dao = context.Entities.Single(x => x.Id == dto.Id);
mapper.Map(dto, dao);
context.SaveChanges();
As the later pattern validates that the dto passed in has a corresponding entity, Automapper should be configured to only copy across allowed properties according to configured rules, then SaveChanges would produce an efficient UPDATE SQL statement only if anything actually changed.
I have the following class:
public class User
{
public int Id { get; set; }
public List<User> Connections { get; set; }
//other properties
public User()
{
Connections = new List<User>();
}
}
Then I have a DataContext class for storage:
public class DataContext : DbContext
{
public DataContext() { }
public DataContext(DbContextOptions<DataContext> options) : base(options) { }
public virtual DbSet<User> Users { get; set; }
}
And a UserService class:
public class UserService: IUserService
{
private DataContext _context;
public UserService(DataContext context)
{
_context = context;
}
public User GetById(int id)
{
return _context.Users.Find(id);
}
...
}
Now suppose I correctly stored 2 users, and I add each other to their respective connection lists.
The problem is in the following piece of code:
var user1 = _userService.GetById(userId);
---> Here user1.Connections is an empty list (unexpected)
var results = anotherList.Select(x=>
{
---> Here user1.Connections have one object inside (the other user as expected)
});
I thought it was because the List was not populated yet since it was never accessed yet, but I also have a problem with the following endpoint in a controller:
var userId = int.Parse(User.Identity.Name);
var user1 = _userService.GetById(userId);
var connectionsInfo = user1.Connections.Select(x => new
{
Id = x.Id,
//map other properties
});
return Ok(connectionsInfo);
//this time an empty list is returned in the response, instead of a list with a single object
I read it might be regarding circular dependency, but I don't get any exception.
Also, I do not understand why in one case the list is populated after and in the other case is not populated at all.
Any idea what could be causing this?
Also I do not understand why in one case the list is populated after and in the other case is not populated at all.
It's the Lazy Loading feature in the entity framework. Lazy loading means delaying the loading of related data until you specifically request for it. For more explanation and a deep dive, you can review this good article.
Entity Framework supports three ways to load related data - eager loading, lazy loading, and explicit loading. for your scenario, It would prefer to use eager loading way. for achieving this goal EF has the Include() method. so, you can update your GetById method as below:
public User GetById(int id)
{
return _context.Users
.Include(item => item.Connections)
.Find(id);
}
With the above query when you find a specific user, its connections loads at the same time too. good luck.
The main problem is that when the web app is launched to the internet, when the load is high an exception is raised telling that there is already an opened data reader.
The following are the specs we use:
Entityframework 5.0.0
MySQL database
Is there a way of solving this problem without the using(){} block? Main problem of this approach is that when closed the using block I can't expand foreign key relations of entityframework objects inside the html view.
I also attach some source code, showing how we keep a single database context through the whole application
public abstract class AbstractService
{
public Entities db_model
{
get
{
return DbContext.Instance.db_model;
}
}
}
public class DbContext
{
public Entities db_model = new Entities();
private static DbContext _dbContext;
public static DbContext Instance
{
get
{
if(_dbContext == null)
{
_dbContext = new DbContext();
}
return _dbContext;
}
}
}
This answer is specifically related to the issue mentioned in the question about using the loaded entities in an ASP.NET View. The question asks about a way of solving this problem without a using block or disposing of the DbContext, however I am suggesting doing exactly this.
The reason being that it's generally desirable not to use Entity Framework objects in the ASP.NET Views because those objects are a lot more more than just plain POCO objects; they hide logic which allows them to act as a proxy to the underlying database, so they have a hidden dependency on the state of the DbContext which created them.
Here's a contrived example using EF models for Employee and Department with a DbContext:
public class CompanyDbContext : DbContext
{
public DbSet<Department> Departments { get; set; }
public DbSet<Employee> Employees { get; set; }
}
public class Department
{
public long Id { get; set; }
public virtual ICollection<Employee> Employees { get; set; }
}
public class Employee
{
public long Id { get; set; }
public long DepartmentId { get; set; }
public virtual Department Department { get; set; }
}
If these were used in an ASP.NET application, I would create some separate models which aren't tied to Entity Framework, to be used by ASP.NET. For example:
public class DepartmentModel
{
public long Id { get; set; }
public List<EmployeeModel> Employees { get; set; }
}
public class EmployeeModel
{
public long Id { get; set; }
public long DepartmentId { get; set; }
}
A few considerations:
According to the MSDN docs, "A DbContext represents a combination of the UnitOfWork and Repository patterns" - https://learn.microsoft.com/en-us/dotnet/api/system.data.entity.dbcontext?redirectedfrom=MSDN&view=entity-framework-6.2.0 - Therefore the DbContext should be short lived as far as possible.
When loading data from the context, related entities can be retrieved using DbSet<>.Include() - https://learn.microsoft.com/en-us/ef/ef6/querying/related-data
Generally speaking, it makes sense to de-couple the 'data' layer from the 'view' layer - for all kinds of reasons, some of which are listed here: https://learn.microsoft.com/en-us/aspnet/web-api/overview/data/using-web-api-with-entity-framework/part-5 -- this involves mapping between the EF objects and the POCO Models.
The logic which is used to query the DbContext would query the data using EF, and return that data using POCO models so that only logic which deals directly with DbContext has any involvement with the EF objects. For example:
public List<DepartmentModel> GetAllDepartments()
{
using (var ctx = new CompanyDbContext())
{
// Ensure that related data is loaded
var departments = ctx.Departments
.Include(d => d.Employees);
// Manual mapping by converting into a new set of models to be used by the Views
var models = departments
.Select(d => new DepartmentModel
{
Id = d.Id,
Employees = d.Employees
.Select(e => new EmployeeModel
{
Id = e.Id,
DepartmentId = e.DepartmentId
})
.ToList(),
})
.ToList();
return models;
}
}
Being able to use those POCO models, while requiring some extra boilerplate code, provides complete separation between the DbContext and ASP.NET, allowing the data to be used without ASP.NET Views/Controllers being concerned by the lifetime or state of the DbContext.
Sometimes this may look as if this approach violates the 'DRY' principle, however I would point out that EF objects and ViewModel objects exist to solve different problems, and it's not uncommon for the ViewModel objects to take a different shape, or even to require additional fields/attributes which wouldn't be suitable to add to the EF classes.
Lastly, the above uses 'manual' mapping, but if the mappings are really simple and straightforward, then it could make more sense to use AutoMapper instead: Cleanest Way To Map Entity To DTO With Linq Select?
Is it possible (with attributes or something) to configure DAO methods to return detached objects? I am keen to do this, because I want to make sure that the DAO pre-fetches any fields and relationships that might be required by downstream code on return from the DAO. If the entities are detached, then an exception will be thrown and we can identity the issue easily. With lazy resolution of relationships, you potentially get multiple additional requests to the DB without realising it.
For example, let's say I have a DAO class:
public class TestDao
{
private readonly MyContext _db;
public TestDao(MyContext db)
{
_db = db;
}
public List<Group> AllGroups()
{
return _db.Groups.ToList();
}
}
And then say that I have a client of the Dao:
public void TestGetAllGroups()
{
var groups = _testDao.AllGroups();
foreach (var group in groups)
{
var x = group.Memberships;
Console.WriteLine( group.id + ":" + x.Count );
}
}
This code works, but each iteration in the test harness causes a new hit to the DB because the DB hasn't pre-fetched (included) the Memberships relationship.
I'm looking for the best way to get this code to throw an exception, saying that group.Memberships is null or something. If the Group instances were detached upon exit from TestDao.AllGroups(), then this would do the trick, and alert us to the fact that the DAO needs to include the Memberships before returning from the AllGroups() method
Looks like you can disable Lazy Loading of relationships. In my Context class:
public partial class MyContext : DbContext
{
static MyContext()
{
Database.SetInitializer<MyContext>(null);
}
public MyContext() : this("Name=MyContext")
{
}
public MyContext(string name): base(name)
{
Configuration.LazyLoadingEnabled = false;
}
...
}
However, in addition to this, I need to make sure that the EF entity constructor doesn't create empty lists for the one-to-many relations. These seem to get added by default with the schema-first entity code generation tool:
public partial class Group
{
public Group()
{
//Remove this line!
//this.Memberships = new List<Membership>();
}
public int id { get; set; }
public string name { get; set; }
public virtual ICollection<Membership> Memberships { get; set; }
}
Now, my test harness will throw a NPE unless I put a .Include into the Dao to include Memberships
I updated my app from EF4.1 to EF6 and now I've got lazy loading issue. I used EF6.x DbContext Generator to generate new DbContext. All of the advices from this article are also applied.
My classes are public
Not sealed, not abstract
Have a public constructor
Don't implement neither IEntityWithChangeTracker nor IEntityWithRelationships
Both ProxyCreationEnabled and LazyLoadingEnabled are set to true
Navigation properties are virtual
What also looks wierd to me is that if I explicitly include navigation property with Include("...") it gets loaded.
Simplified version of my POCOs and DbContext:
public partial class Ideation
{
public Ideation()
{
}
public long Id { get; set; }
public Nullable<long> ChallengeId { get; set; }
public virtual Challenge Challenge { get; set; }
}
public partial class Challenge
{
public Challenge()
{
this.Ideations = new HashSet<Ideation>();
}
public long Id { get; set; }
public virtual ICollection<Ideation> Ideations { get; set; }
}
public partial class BoxEntities : DbContext
{
public TIBoxEntities()
: base("name=BoxEntities")
{
}
protected override void OnModelCreating(DbModelBuilder modelBuilder)
{
throw new UnintentionalCodeFirstException();
}
public virtual DbSet<Ideation> Ideations { get; set; }
public virtual DbSet<Challenge> Challenges { get; set; }
}
Also I tried to set ProxyCreationEnabled and LazyLoadingEnabled explicitly without no luck. The entity isn't loaded as a dynamic proxy as this debug session screenshot shows:
What else am I missing?
A situation where this could happen is that the entity you are trying to load with Find is already attached to the context as a non-proxy object. For example:
using (var context = new MyContext())
{
var ideation = new Ideation { Id = 1 }; // this is NOT a proxy
context.Ideations.Attach(ideation);
// other stuff maybe ...
var anotherIdeation = context.Ideations.Find(1);
}
anotherIdeation will be the non-proxy that is already attached and it is not capable of lazy loading. It even wouldn't help to run a DB query with var anotherIdeation = context.Ideations.SingleOrDefault(i => i.Id == 1); because the default merge option for queries is AppendOnly, i.e. the new entity would only be added if there isn't already an attached entity with that key. So, anotherIdeation would still be a non-proxy.
You can check if the entity is already attached by using Local before you call Find in your GetById method:
bool isIdeationAttached = context.Ideations.Local.Any(i => i.Id == id);
Per #ken2k's comment, the default for new models starting with EF 4 was to enable lazy loading by default. With EF 1, it was not allowed. If you migrated your model from 1 to 4, it is kept off by default. You would need to modify the context to turn that on.
That being said, you indicate through debugging that it is true. In that case, check your usage scenarios. Is it possible your context has been disposed prior to fetching the child objects. Typically we see this when data binding to the LINQ query where the context is configured in a Using block and the actual iteration doesn't happen until after the using block's scope has passed.