I have two tables(Teacher and Course) in my database and EF mapping:
public partial class Teacher
{
public long Id { get; set; }
public virtual ICollection<Course> Courses { get; set; }
}
And I load a teacher like this:
public Teacher GetTeacher(long id)
{
using(var entities = new MyEntities())
{
var teacher = entities.Teachers.Where(t => t.Id == id).FirstOrDefault();
return teacher;
}
}
Now sometimes I want to load Courses list for my Teacher. So I changed implementation like this:
public Teacher GetTeacher(long id, bool loadCourses)
{
using(var entities = new MyEntities())
{
var teacherQuery = entities.Teachers.Where(t => t.Id == id);
if(loadCourses)
{
teacherQuery.Include(t => t.Courses);
}
return teacherQuery.FirstOrDefault();
}
}
And this worked fine. But after that, I decided to turn off LazyLoading for Courses property since I've decided to control this manually by loadCourses field.
So I've just removed virtual from Courses collection:
public ICollection<Course> Courses { get; set; }
This did help to turn off LazyLoading but Include stopped working and my Courses collection never loads.
So the question is: is it possible to do eager loading for Courses collection with LazyLoading disabled?
P.S. I don't actually use entity objects in my application but convert them to my Domain objects. So that's why I decided to use bool loadCourses field instead of actually using LazyLoading.
Also, I'd like to have one SELECT query(with JOIN, of course) sent to the database instead of two separate SELECTs.
Shortly after asking the question, I found an answer which is dead simple: there is a bug under if(loadCourses) that I was not assigning teacherQuery. Fixed code looks like this and works fine:
if(loadCourses)
{
teacherQuery = teacherQuery.Include(t => t.Courses);
}
Also would like to mention a useful link that #VidmantasBlazevicius provided. It contains answers on how virtual keyword affects your entities.
Related
i am struggeling for a while now to understand how EF loads / updates entities.
First of all i wanna explain what my app (WPF) is about. I am developing
an application where users can store Todo Items in Categories, these categories are predefined by the application. Each user can read all items but can only delete / update his own items. It's a multiuser system, means the application is running multiple times in the network accessing the same sql server database.
When a user is adding/deleting/updating items the UI on all the other running apps has to update.
My model looks like this:
public class Category
{
public int Id { get; set; }
public string Name { get; set; }
public List<Todo> Todos { get; set; }
}
public class Todo
{
public int Id { get; set; }
public string Content { get; set; }
public DateTime LastUpdate { get; set; }
public string Owner { get; set; }
public Category Category { get; set; }
public List<Info> Infos { get; set; }
}
public class Info
{
public int Id { get; set; }
public string Value { get; set; }
public Todo Todo { get; set; }
}
I am making the inital load like this, which works fine:
Context.dbsCategories.Where(c => c.Id == id).Include(c => c.Todos.Select(t => t.Infos)).FirstOrDefault();
Now i was trying to load only the Todos which are from the current user therefore i tried this:
Context.dbsCategories.Where(c => c.Id == id).Include(c => c.Todos.Where(t => t.Owner == Settings.User).Select(t => t.Infos)).FirstOrDefault();
This does not work because it's not possible to filter within include, so I tried this:
var cat = Context.dbsCategories.Where(c => c.Id == id).FirstOrDefault();
Context.dbsTodos.Where(t => t.Category.Id == cat.Id && t.Owner == Settings.User).Include(t=>t.Infos);
After executing the second line where i look for the Todo Items, these Items were automatically added to cat's Todos collection. Why? I would have expected that i have to add them manually to cat's Todos collection.
Just for my understanding what is EF doing here exactly?
Now to my main problem -> the synchronization of the data between database and client. I am using a long running Context which lives as long as the application is running to save changes to the database which are made on owned items. The user does not have the possibility to manipulate / delete data from other users this is guarantee by the user interface.
To synchronize the data i build this Synch Method which will run every 10 second, right now it's triggere manually.
Thats my synchronization Code, which only synchronizes Items to the client that do not belong to it.
private async Task Synchronize()
{
using (var ctx = new Context())
{
var database = ctx.dbsTodos().Where(x => x.Owner != Settings.User).Select(t => t.Infos).AsNoTracking();
var loaded = Context.dbsTodos.Local.Where(x => x.Owner != Settings.User);
//In local context but not in database anymore -> Detachen
foreach (var detach in loaded.Except(database, new TodoIdComparer()).ToList())
{
Context.ObjectContext.Detach(detach);
Log.Debug(this, $"Item {detach} detached");
}
//In database and local context -> Check Timestamp -> Update
foreach (var update in loaded.Intersect(database, new TodoIdTimeStampComparer()))
{
await Context.Entry(update).ReloadAsync();
Log.Debug(this, $"Item {update} updated");
}
//In database but not in local context -> Attach
foreach (var attach in database.ToList().Except(loaded, new TodoIdComparer()))
{
Context.dbsTodos().Attach(attach);
Log.Debug(this, $"Item {attach} attached");
}
}
}
I am having following problems / issues of unknow origin with it:
Detaching deleted Items seems to work, right now i am not sure if only the Todo Items are detached or also the Infos.
Updating Items works only for the TodoItem itsself, its not reloading the Infos within? How can i reload the whole entity with all it's relations?
I am thankful for every help on this, even if you are saying it's all wrong what i am doing here!
Attaching new Items and Infos does not work so far? What am i doing wrong here?
Is this the right approach to synchronize data between client and database?
What am i doing wrong here? Is there any "How to Sync" Tutorial? I have not found anything helpful so far?
Thanks!
My, you do like to deviate from entity framework code-first conventions, do you?
(1) Incorrect class definitions
The relations between your tables are Lists, instead of ICollections, they are not declared virtual and you forgot to declare the foreign key
There is a one-to-many relation between Todo and Category: every Todo belongs to exactly one Category (using a foreign key), every Category has zero or more Todos.
You choose to give Category a property:
List<Todo> Todos {get; set;}
Are you sure that category.Todos[4] has a defined meaning?
What would category.Todos.Insert(4, new Todo()) mean?
Better stick to an interface where you can't use functions that have no proper meaning in your database: use ICollection<Todo> Todos {get; set;}. This way you'll have only access to functions that Entity Framework can translate to SQL.
Besides, a query will probably be faster: you give entity framework the possibility to query the data in its most efficient way, instead of forcing it to put the result into a List.
In entity framework the columns of a table are represented by non-virtual properties; the virtual properties represent the relations between the tables (one-to-many, many-to-many)
public class Category
{
public int Id { get; set; }
public string Name { get; set; }
... // other properties
// every Category has zero or more Todos (one-to-many)
public virtual ICollection<Todo> Todos { get; set; }
}
public class Todo
{
public int Id { get; set; }
public string Content { get; set; }
... // other properties
// every Todo belongs to exactly one Category, using foreign key
public int CategoryId { get; set }
public virtual Category Category { get; set; }
// every Todo has zero or more Infos:
public virtual ICollection<Info> Infos { get; set; }
}
You'll probably guess Info by now:
public class Info
{
public int Id { get; set; }
public string Value { get; set; }
... // other properties
// every info belongs to exactly one Todo, using foreign key
public int TodoId {get; set;}
public virtual Todo Todo { get; set; }
}
Three major improvements:
ICollections instead of Lists
ICollections are virtual, because it is not a real column in your table,
foreign key definitions non-virtual: they are real columns in your tables.
(2) Use Select instead of Include
One of the slower parts of a database query is the transport of the selected data from the Database Management System to your local process. Hence it is wise to limit the amount of transported data.
Suppose Category with Id [4] has a thousand Todos. Every Todo of this Category will have a foreign key with a value 4. So this same value 4 will be transported 1001 times. What a waste of processing power!
In entity framework use Select instead of Include to query data and select only the properties you actually plan to use. Only use Include if you plan to update the Selected data.
Give me all Categories that ... with their Todos that ...
var results = dbContext.Categories
.Where(category => ...)
.Select(category => new
{
// only select properties that you plan to use
Id = category.Id,
Name = category.Name,
...
Todos = category.Todos
.Where(todo => ...) // only if you don't want all Todos
.Select(todo => new
{
// again, select only the properties you'll plan to use
Id = todo.Id,
...
// not needed, you know the value:
// CategoryId = todo.CategoryId,
// only if you also want some infos:
Infos = todo.Infos
.Select(info => ....) // you know the drill by now
.ToList(),
})
.ToList(),
});
(3) Don't keep DbContext alive for such a long time!
Another problem is that you keep your DbContext open for quite some time. This is not how a dbContext was meant. If your database changes between your query and your update, you'll have troubles. I can hardly imagine that you query so much data that you need to optimize it by keeping your dbContext alive. Even if you query a lot of data, the display of this huge amount of data would be the bottle-neck, not the database query.
Better fetch the data once, dispose the DbContext, and when updating fetch the data again, update the changed properties and SaveChanges.
fetch data:
RepositoryCategory FetchCategory(int categoryId)
{
using (var dbContext = new MyDbContext())
{
return dbContext.Categories.Where(category => category.Id == categoryId)
.Select(category => new RepositoryCategory
{
... // see above
})
.FirstOrDefault();
}
}
Yes, you'll need an extra class RepositoryCategory for this. The advantage is, that you hide that you fetched your data from a database. Your code would hardly change if you'd fetch your data from a CSV-file, or from the internet. This is way better testable, and also way better maintainable: if the Category table in your database changes, users of your RepositoryCategory won't notice it.
Consider creating a special namespace for the data you fetch from your database. This way you can name the fetched Category still Category, instead of RepositoryCategory. You even hide better where you fetched your data from.
Back to your question
You wrote:
Now i was trying to load only the Todos which are from the current user
After the previous improvements, this will be easy:
string owner = Settings.User; // or something similar
var result = dbContext.Todos.Where(todo => todo.Owner == owner)
.Select(todo => new
{
// properties you need
})
I have a Customer class that has a relationship to an Address class:
public class Customer
{
public int Id { get; set; }
public string Name { get; set; }
public virtual ICollection<Address> Addresses { get; set; }
}
public class Address
{
public int Id { get; set; }
public string Street1 { get; set; }
//Snip a bunch of properties
public virtual Customer Customer { get; set; }
}
I have an edit form which displays all the fields for both the customer and address. When this form is submitted, it calls the Edit method in the controller:
public ActionResult Save(Customer customer)
{
if (!ModelState.IsValid)
{
var viewModel = new CustomerFormViewModel
{
Customer = customer,
CustomerTypes = _context.CustomerTypes.ToList()
};
return View("CustomerForm", viewModel);
}
if (customer.Id == 0)
_context.Customers.Add(customer);
else
{
var existingCustomer = _context.Customers
.Include(c => c.Addresses)
.Single(c => c.Id == customer.Id);
existingCustomer.Name = customer.Name;
existingCustomer.TaxId = customer.TaxId;
existingCustomer.CustomerTypeId = customer.CustomerTypeId;
existingCustomer.CreditLimit = customer.CreditLimit;
existingCustomer.Exempt = customer.Exempt;
existingCustomer.Addresses = customer.Addresses;
}
_context.SaveChanges();
return RedirectToAction("Index", "Customers");
}
This doesn't work and creates duplicate entries in the Addresses table in the DB. I think I understand why (EF isn't smart enough to know the Addresses inside the collection need to be added/modified/deleted as the case may be). So, what is the best way to fix this?
My instinct is that I need to iterate over the Addresses collections and compare them manually, adding any new ones from the form that don't exist for the customer, updating ones that do exist, and deleting ones that were not sent by the form but exist in the DB for the customer. Something like (ignoring the delete functionality for now):
foreach(Address address in customer.Addresses)
{
if (address.Id == 0)
// Add record
else
// Fetch address record from DB
// Update data
}
// Save context
Is this the best way to go about this, or are there any EF tricks to iterating and syncing a child collection to the DB?
Oh, and one question which has me scratching my head - I can sort of understand how a new address record is getting created in the DB, but what I don't get is the existing address record is also updated to have its customer_id set to NULL...how the heck does that happen? That leads me to believe that EF does see the original address record is somehow linked (as it is modifying it) but it's not smart enough to realize the record I'm passing in should replace it?
Thanks -- also, this is EF6 and MVC5
The problem comes from the line
existingCustomer.Addresses = customer.Addresses;
in your code. This like assigns field Addresses from customer coming from the model. So far ok. The point is that customer does not have any relation to the database model at this point (it's not coming from the database but from the view).
If you would like to update existingCustomer.Addresses with the data coming from the model, you need to merge the data instead of replacing it. The following "pseudo code" might give you a direction:
void MergeAddresses(var existingAddresses, var newAddresses) {
foreach(var address in newAddresses) {
if (existingAddresses.Contains(newAddress)) {
// merge fields if applicable
}
else {
// add field to existingAddresses - be ware to use a "cloned" list
}
}
// now delete items from existing list
foreach (var address in existingAddresses.CloneList()) {
if (!newAddresses.Contains(address)) {
// remove from existingAddresses
}
}
}
Is this the best way to go about this, or are there any EF tricks to iterating and syncing a child collection to the DB?
No, there aren't such tricks. EF designers left saving detached entities totally up to us - the developers.
However there is a package called GraphDiff which is addressing that, so you could give it a try. Here is how your code would look like using it:
using RefactorThis.GraphDiff;
...
_context.UpdateGraph(customer, map => map.OwnedCollection(
e => e.Addresses, with => with.AssociatedEntity(e => e.Customer)));
_context.SaveChanges();
I have a Customer model with a MailingAddress property - this is a navigation property to an Address class:
public class Customer
{
public int Id { get; set; }
public string Name { get; set; }
public virtual Address MailingAddress { get; set; }
}
I have a form that posts a viewmodel back which contains a customer, along with the mailing address. Here is the save method in the controller:
public ActionResult Save(CustomerFormViewModel viewModel)
{
if (!ModelState.IsValid)
{
return View("CustomerForm", viewModel);
}
if (viewModel.Customer.Id == 0)
{
_context.Customers.Add(viewModel.Customer);
}
else
{
var customerInDb = _context.Customers
.Single(c => c.Id == viewModel.Customer.Id);
_context.Entry(customerInDb).CurrentValues.SetValues(viewModel.Customer);
}
_context.SaveChanges();
return RedirectToAction("Index", "Customers");
}
When posting a new customer, everything works fine (mostly, see note below) and the customer is created along with a corresponding address record. However, when I edit an existing entry, the customer is updated but the address is not. I verified the updated address is being passed in the customer object. If I add a line like this:
_context.Entry(customerInDb.MailingAddress).CurrentValues.SetValues(viewModel.Customer.MailingAddress);
Then it is updated.
Is the child here still considered a detached entity? I assumed since it is a property of the Customer I am fetching it would be automatically saved with the parent. Why does this work with a new record and not with an update?
One note about the new record creation - a Customer record is created and has a MailingAddress_Id pointing to the address. The Address record is also created, but its Customer_Id is null...an idea why EF is not adding the key on that side of the relationship? Address model and view model code in case it helps:
public class Address
{
public int Id { get; set; }
public string Street1 { get; set; }
// Snip a bunch of address data properties
public virtual Customer Customer { get; set; }
}
public class CustomerFormViewModel
{
// Snip irrelevant properties
public Customer Customer { get; set; }
}
First of all, if your Customer and Address are in one-to-one relationship, then no foreign key is needed. Actually, in one-to-one relatioonships primary key on dependant side of relationship is also foreign key to principal side. Secondly, when you create new Customer you use context.Customers.Add(viewModel.Customer); and it adds model with all of its child models, but when you try to update using _context.Entry(customerInDb).CurrentValues.SetValues(viewModel.Customer); it does not add all child navigation properties, to do so, you have to tell it to EntityFramework explicitly:
var customerInDb = _context.Customers
.Single(c => c.Id == viewModel.Customer.Id);
_context.Entry(customerInDb)
.CurrentValues
.SetValues(viewModel.Customer);
var mailingAddressInDb = _context.Addresses
.Single(m => m.Id = viewModel.Customer.MailingAddress.Id);
_context.Entry(mailingAddressInDb)
.CurrentValues
.SetValues(viewModel.Customer.MailingAddress);
It should work for you. But it is a bit awkward. When you have dozens of models, you would not even want to imagine it.
Good news
The good news is that, there is an API to solve this problem from its roots. Your problem will be solved in just a few steps. You install it from NuGet using Install-Package Ma.EntityFramework.GraphManager, configure your models to meet prerequisites (which are so easy) and handle whole graph using single line of code:
public ActionResult Save(CustomerFormViewModel viewModel)
{
if (!ModelState.IsValid)
{
return View("CustomerForm", viewModel);
}
// Define state of whole graph with single line
_context.AddOrUpdate(viewModel.Customer);
_context.SaveChanges();
return RedirectToAction("Index", "Customers");
}
Please, have a look at CodeProject article for a quick wallktrough. It has example code, so you can download and examine it. I am the owner of this API and I am ready to answer to your questions.
I have a teacher entity that has a child entity of ICollection called YogaClasses. When I modify the list of YogaClasses for a teacher and save, entity framework adds the new rows of YogaClasses to the table 'YogaClass' but it doesn't remove or edit the old rows. So I'm left with double the data. Shouldn't "context.SaveChanges()" in my repo be smart enough to know to delete removed classes and add the new ones (edit), not just add new ones?
In my controller I have something like this for a Teacher edit.
string userId = User.Identity.GetUserId();
Teacher teacher = teacherRepository.Find(userId);
//other code left out here
teacher.YogaClasses = GetSelectedClasses(Request.Form[2]);
// other stuff here
teacherRepository.Save();
In my reposity I have this;
public void Save()
{
context.SaveChanges();
}
Here is my teacher and YogaClass entity
public class Teacher
{
public int TeacherId { get; set; }
public virtual ICollection<YogaClass> YogaClasses { get; set; }
}
public class YogaClass
{
public int YogaClassId { get; set; }
[Index]
[Required]
public int TeacherRefId { get; set; }
[ForeignKey("TeacherRefId")]
public virtual Teacher Teacher { get; set; }
}
The problem here is the relationship has not been loaded at the time you set teacher.YogaClasses. So when you set teacher.YogaClasses, it assumes the new relationship will be added. (lazyloading is still a bit late in this case). The solution is somehow you ensure to load all the relationships first (using either Include or some fake access to trigger lazyloading first), then you set the new value normally:
//fake access to trigger lazy loading (of course this works only if
//lazy loading is enabled)
var someClasses = teacher.YogaClasses;
teacher.YogaClasses = GetSelectedClasses(Request.Form[2]);
You can also clear the old classes and add new ones with a foreach loop:
teacher.YogaClasses.Clear();
foreach(var c in GetSelectedClasses(Request.Form[2])){
teacher.YogaClasses.Add(c);
}
I have a dirty solution but it doesn't seem like it's the best one.
before I save the teacher context in my repo I can call this
public void DeleteYogaClasses(Teacher teacher)
{
foreach (var yogaClass in teacher.YogaClasses.ToList())
{
context.Entry(yogaClass).State = EntityState.Deleted;
}
context.SaveChanges();
}
you need to use Attach method of the context and pass in your existing teacher object and then make the changes and call SaveChanges method. From your code it is not clear what is your context, so hard to give working code. Here is a link explaining how to add/modify
You said you need to modify YogaClasses, so from this what I assume is a Teacher object already has some YogaClasses entries and you want to update some of those entries. What you need to do is, have a list of ids of YogaClasses that you need to modify then iterate on that ids list and in that iteration loop find the existing yogaClass and attach it to the context, modify it and then call save changes (preferablly when all the changes are done so that it is not a performance hit)
Here is a suodocode for this
UpdateTeacher(int teacherId)
{
var teacher = teacherRepository.Find(teacherId);
UpdateYoga(teacher);
}
private void UpdateYoga(Teacher teacher)
{
foreach(var yoga in teacher.YogaClasses)
{
db.Context.Attach(yoga);
yoga.YogaStyle = whatEverValue;
}
db.context.SaveChanges();
}
I am using MVC.NET web api, EF with DB first, and I have lazy loading turned off on my context. EF is returning way too much data, even with LazyLoading turned off.
For example, I have Users with one Role. When I query for Users and Include Role, the Role.Users property is automatically filled with data since Users have been loaded into the context.
Why can't I get EF to give me JUST what I request? Or am I missing something big here?
public partial class User
{
public int UserID { get; set; }
public string Title { get; set; }
public string Email { get; set; }
public int RoleID { get; set; }
....
public virtual Role Role { get; set; }
}
public partial class Role
{
public int RoleID { get; set; }
public string RoleName { get; set; }
....
public virtual ICollection<User> Users { get; set; }
}
return db.Users.Include(u => u.Role);
// ^^ user.Role.Users is filled with 1000s of users
TL;DR - I want EF to never load data into navigation properties/collections unless I .Include() it directly. When serializing to JSON I want just what I ask for explicitly. It seems that even with lazy loading off, navigation properties that are already in the context (ie usually "circular references") will be loaded and returned.
The behaviour your are seeing is called Relationship Fixup and you cannot disable it.
If you are loading users with roles to serialize them and sent them to somewhere I guess that you don't want to track changes of entities in the context they have been loaded in. So, there is no need to attach them to the context and you can use:
return db.Users.Include(u => u.Role).AsNoTracking();
Or use a projection into an object specialized for serialization, as suggested by #STLRick.
You can select only what you need by using Select().
var users = _db.Users.Select(x => new
{
UserID = x.UserID,
Title = x.Title,
Email = x.Email,
RoleID = x.RoleID
}).AsEnumerable();
You are right that with lazy loading on, you will get back navigation properties because they are "touched" by the serializer which causes them to be loaded. Lazy loading should be off if you want the properties to come back as null. That said, it "seems" that once entities are loaded into the context (through other queries, for example), they will be processed by the serializer. So the answer is to tell the serializer not to return the navigation properties. The best way I've been able to find to do this is to use DTOs (Data Transfer Objects). This allows you to return exactly the data you want rather than your actual entities.
Your DTO might look something like this:
public partial class UserDto
{
public UserDto(user User)
{
UserID = user.UserID;
Title = user.Title;
//... and so on
}
public int UserID { get; set; }
public string Title { get; set; }
public string Email { get; set; }
public int RoleID { get; set; }
//exclude the Role navigation property from your DTO
}
...and then you could do something like this:
return db.Users.Include(u => u.Role).Select(user => new UserDto(user));
I don't want it to load anything besides what I tell it to include.
It looks like you need to use Explicit Loading. Basically, you could load the specific entities like this:
context.Include("Roles")
To my best knowledge that should not include related entities. Lazy loading should indeed be disabled and you could load navigational properties explicitly with Load.
First: Turn Lazy Loading on.
Second: If you want to filter down what you retrieve and return, then do a custom return object or something.
from u in db.Users
join r in db.Roles
on u.RoleID equals r.RoleID
select new { u.UserID, u.Title, u.Email, r.RoleName }
Or something like that. You will have a minimal return object and your object graph will be tiny.