Develop Reference

Why do I need to .Include() collections

I wrote a query which is pretty simple:
var locations = await _context.Locations
.Include(x => x.LocationsOfTheUsers)
.Include(x => x.Address)
.ThenInclude(x => x.County)
.Where(CalculateFilters(searchObj))
.ToListAsync(cancellationToken);
And everytime LocationsOfTheUsers were null so I decided to .Include(x => x.LocationsOfTheUsers) and I received results as expected but I'm not sure why do I have to include this collections since it's defined like this:
public class Location
{
public string Title { get; set; }
public long? RegionId { get; set; }
public Region Region { get; set; }
public long? AddressId { get; set; }
public Address Address { get; set; }
public long? CountyId { get; set; }
public County County { get; set; }
public ICollection<LocationsOfTheUsers> LocationsOfTheUsers { get; set; }
}
I thought this will be automatically included since it exist as ICollection in Location class.
So why is .Include() on LocationsOfTheUsers needed here?
Thanks guys
Cheers

In entity framework the non-virtual properties represent the columns of the tables, the virtual properties represent the relations between the tables (one-to-many, many-to-many, ...)
So your property should have been defined as:
public virtual ICollection<LocationsOfTheUsers> LocationsOfTheUsers { get; set; }
One of the slower parts of a database query is the transfer of the selected data from the database management system to your local process. Hence it is wise to limit the selected data to the values you actually plan to use.
If you have a one-to-many relation between Schools and Students, and you ask for School [10] you don't want automatically to fetch its 2000 Students.
Even if you would like to have "School [10] with all its Students" it would not be efficient to use Include to also fetch the Students. Every Student will have a foreign key SchoolId with a Value of [10]. If you would use Include you would transfer this foreign key 2000 times. What a waste!
When using entity framework always use Select to fetch data and select only the properties that you actually plan to use. Only use Include if you plan to change the included items.
This way you can separate your database table structure from the actual query. If your database structure changes, only the query changes, users of your query don't notice the internal changes.
Apart from better performance and more robustness against changes, readers of your code can more easily see what values are in their query.
Certainly don't use Include to save you some typing. Having to debug one error after future changes will take way more time than you will ever save by typeing include instead of Select
Finally: limit your data early in your process, so put the Where in front.
So your query should be:
var predicate = CalculateFilters(searchObj)
var queryLocations = dbContext.Locations
.Where(predicate)
.Select(location => new
{
// Select only the location properties that you plan to use
Id = location.Id,
Name = location.Name,
// Locations Of the users:
UserLocations = location.LocationsOfTheUsers
.Select(userLocation => new
{
// again: only the properties that you plan to use
Id = userLocation.Id,
...
// Not needed, you already know the value
// LocationId = userLocation.LocationId
})
.ToList(),
Address = new
{
Street = location.Address.Street,
PostCode = location.Addrress.PostCode,
...
County = location.Address.County.Name // if you only want one property
// or if you want more properties:
County = new
{
Name = location.Address.County.Name,
Abbr = location.Address.Count.Abbr,
...
}),
},
});

I thought this will be automatically included since it exist as ICollection in Location class.
Well, it's not automatically included, probably for performance reasons as the graph of related entities and their recursive child entities may be rather deep.
That's why you use eager loading to explicitly include the related entities that you want using the Include method.
The other option is to use lazy loading which means that the related entities are loaded as soon as you access the navigation property in your code, assuming some prerequisites are fulfilled and that the context is still around when this happens.
Please refer to the docs for more information.

I believe you are using EntityFrameworkCore. In EntityFramework (EF6), lazy loading is enabled by default, However, in EntityFrameworkCore, lazy loading related entities is handled by a separate package Microsoft.EntityFrameworkCore.Proxies.
To enable the behaviour you are seeking, install the above package and add the following code
protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
{
optionsBuilder.UseLazyLoadingProxies();
}
After this, the related entities will be loaded without the Include call.

Wrong column name weird issue in Entity Framework

I have been facing this problem some time, and to be honest I am myself confused with it so please excuse me if i don't succeed explaining it as I should.
I am trying to insert some data into a Table called CommunicationAttachment which is related as One to Many relationship with Communication; every communication could have many attachments.
The thing is that I get:
UpdateException: Invalid Column Name: "Communication_CommunicationId
when I try to insert list of attachments.
And please note that I am using the repository pattern but I even tried the normal way and the issue wasn't fixed.
I tried tracing the transaction that happens on the database and I figured out that it sends Communication_CommunicationId with the Insert statement, yet there is no such column. I am pretty sure I didn't send such a column.
Here is my code (this is happening when adding new Communication); first of all I call CasefileAttachments to make copies from them, and Communications are related to CaseFiles:
public List<CorrespondenceAttachment> GetCaseFileAttachments(List<Guid> CorrespondenceAttachmentIds)
{
List<CorrespondenceAttachment> originalAttachments = new List<CorrespondenceAttachment>();
foreach (var item in CorrespondenceAttachmentIds)
{
var attachment = QueryData.Query<CorrespondenceAttachment>().Where(att => att.CorrespondenceAttachmentID == item).FirstOrDefault();
originalAttachments.Add(attachment);
}
return originalAttachments;
}
Then I copy the CaseFileAttachments and create new objects of CommunicationAttachments :
public List<CommunicationAttachment> CopyCaseFileAttachmentsToCommunication(List<CorrespondenceAttachment> originalAttachments,Guid communicationId)
{
var communicationAttachments = new List<CommunicationAttachment>();
if (originalAttachments.Any())
{
foreach (var attachmentRef in originalAttachments)
{
var CommunicationAttachmentId = Guid.NewGuid();
communicationAttachments.Add(new CommunicationAttachment()
{
CommunicationAttachmentId = CommunicationAttachmentId,
DmsFileId = CommunicationAttachmentId,
CommunicationId = communicationId,
AttachmentTitle = attachmentRef.AttachmentTitle,
MimeType = attachmentRef.MimeType,
NewVersionID = null,
UploadDate = DateTime.Now,
Size = attachmentRef.Size,
Version = "0001",
AttachmentsGroupId = attachmentRef.AttachmentsGroupId,
DocumentId = attachmentRef.DocumentId,
RelativePath = attachmentRef.RelativePath,
Extension = attachmentRef.Extension,
AttachmentSubject = attachmentRef?.AttachmentSubject,
ExternalContactID = attachmentRef?.ExternalContactID,
AttachmentNumber = string.IsNullOrEmpty(attachmentRef?.AttachmentNumber) ? null : attachmentRef.AttachmentNumber,
TemplatedmsId = attachmentRef.TemplatedmsId,
State = eSense.Framework.Data.ObjectState.Added,
});
}
}
return communicationAttachments;
}
and the methods above are called something like this way:
public void AddNewCommunication(CommunicationDto communicationDto)
{
var communication = communicationDto
if (communicationDto.CommunicationAttachmentIdList.Any())
{
caseFileAttachments = GetCaseFileAttachments(communicationDto.CommunicationAttachmentIdList);
if (caseFileAttachments.Any())
{
commAttachments = CopyCaseFileAttachmentsToCommunication(caseFileAttachments, communication.CommunicationId);
}
}
communication.Attachments = commAttachments;
Save(communication)
}
So what could be the problem that I get a wrong column name?
Here is the relation between Communication and CommunicationAttachment
Note I added only the Important fields so don't bother if the declaring does not match the entity
Communication Entity:
public class Communication : BaseEntity
{
public Communication()
{
Attachments = new HashSet<CommunicationAttachment>();
}
[Key]
public Guid CommunicationId { get; set; }
public string Subject { get; set; }
public string CommunicationNumber { get; set; }
public virtual ICollection<CommunicationAttachment> Attachments { get; set; }
public DateTime DateCreated { get; set; }
public Guid? PreviousCommunicationId { get; set; }
[ForeignKey("PreviousCommunicationId")]
public virtual Communication PreviousCommunication { get; set; }
}
CommunicationAttachment Entity:
public class CommunicationAttachment : AttachmentBaseWithDelegation<Guid>
{
public override Guid PrimaryId
{
get
{
return this.CommunicationAttachmentId;
}
}
public CommunicationAttachment()
{
}
[Key]
public Guid CommunicationAttachmentId { get; set; }
private string _attachmentNumber;
public string AttachmentNumber { get; set; }
[ForeignKey("NewVersionID")]
public virtual CommunicationAttachment CaseFileAttachmentNewerVersion { get; set; }
public Guid CommunicationId { get; set; }
[ForeignKey("CommunicationId")]
public virtual Communication Communication { get; set; }
}
Sorry if you found it hard to understand my question I myself is confused!
Thanks in advance.

This is typically a case where a relationship between entities is not set up correctly. It would appear that EF should be resolving this relationship by convention if Communication's PK is "CommunicationId".
I notice that you've commented out a line to set the CommunicationId on the new entity:
//CommunicationId = communicationId,
What fields are in the CommunicationAttachment? is there a CommunicationId? Is there a Communication navigation property? What configuration settings are you are using?
For example, with fluent configuration I would have something like:
(CommunicationEntityConfiguration)
If CommunicationAttachment has a navigation property back to Communication and a FK field called CommunicationId...
HasMany(x => x.CommunicationAttachments)
.WithRequired(x => x.Communication)
.HasForeignKey(x => x.CommunicationId);
If the attachment entity has a navigation property without a mapped FK in the entity...
HasMany(x => x.CommunicationAttachments)
.WithRequired(x => x.Communication)
.Map(x => x.MapKey("CommunicationId"));
If the attachment entity does not have a navigation property, but has a FK in the entity...
HasMany(x => x.CommunicationAttachments)
.WithRequired()
.HasForeignKey(x => x.CommunicationId);
Or lastly if the attachment entity does not have a navigation property nor a mapped FK...
If the attachment entity does not have a navigation property, but has a FK in the entity...
HasMany(x => x.CommunicationAttachments)
.WithRequired()
.Map(x => x.MapKey("CommunicationId"));
I am a big fan of explicit mapping over convention as it is very clear as to what maps to what, and how, in order to resolve potential mapping conflicts. If the rest of the similar relations seem to be working and just this one is playing up, I'd be looking for possible typos in the field names. With a mapped collection like above, setting a Communcation.CommunicationAttachments.Add(attachment) should be setting the FK / related entity on the attachment without having to explicitly set the FK or related entity manually.
One additional note:
From your example I see you are setting Primary Keys manually client-side using Guid.NewGuid(). It is generally better to allow the database to manage PK generation and let EF manage FK assignment to ensure that related entities get the FKs to newly inserted rows automatically. Rather than SQL's NewId() or using Guid.NewGuid(), it is advisable to use sequential UUIDs. In SQL Server this is NewSequentialId(). For client-side setting, you can reproduce the sequential UUID pattern either with a system DLL call to get the ID, or a simple re-hash of the Guid bytes. see: Is there a .NET equalent to SQL Servers newsequentialid()
The GUIDs still carry the same uniqueness, the bytes are simply arranged to be more sequential and practical for database indexing to reduce page fragmentation. The downside is that IDs are more predictable. Depending on your database engine you might want to customize the algorithm based on whether the database is optimized for indexing on the lower-order or high-order bytes.
When using GUIDs for database, sequential or otherwise, you should ensure you have a scheduled index maintenance job on the database. With sequential IDs this job will run faster and keep the index tables more compact.

Updating object with child collection using Entity Framework causing duplicates in database

I have a Customer class that has a relationship to an Address class:
public class Customer
{
public int Id { get; set; }
public string Name { get; set; }
public virtual ICollection<Address> Addresses { get; set; }
}
public class Address
{
public int Id { get; set; }
public string Street1 { get; set; }
//Snip a bunch of properties
public virtual Customer Customer { get; set; }
}
I have an edit form which displays all the fields for both the customer and address. When this form is submitted, it calls the Edit method in the controller:
public ActionResult Save(Customer customer)
{
if (!ModelState.IsValid)
{
var viewModel = new CustomerFormViewModel
{
Customer = customer,
CustomerTypes = _context.CustomerTypes.ToList()
};
return View("CustomerForm", viewModel);
}
if (customer.Id == 0)
_context.Customers.Add(customer);
else
{
var existingCustomer = _context.Customers
.Include(c => c.Addresses)
.Single(c => c.Id == customer.Id);
existingCustomer.Name = customer.Name;
existingCustomer.TaxId = customer.TaxId;
existingCustomer.CustomerTypeId = customer.CustomerTypeId;
existingCustomer.CreditLimit = customer.CreditLimit;
existingCustomer.Exempt = customer.Exempt;
existingCustomer.Addresses = customer.Addresses;
}
_context.SaveChanges();
return RedirectToAction("Index", "Customers");
}
This doesn't work and creates duplicate entries in the Addresses table in the DB. I think I understand why (EF isn't smart enough to know the Addresses inside the collection need to be added/modified/deleted as the case may be). So, what is the best way to fix this?
My instinct is that I need to iterate over the Addresses collections and compare them manually, adding any new ones from the form that don't exist for the customer, updating ones that do exist, and deleting ones that were not sent by the form but exist in the DB for the customer. Something like (ignoring the delete functionality for now):
foreach(Address address in customer.Addresses)
{
if (address.Id == 0)
// Add record
else
// Fetch address record from DB
// Update data
}
// Save context
Is this the best way to go about this, or are there any EF tricks to iterating and syncing a child collection to the DB?
Oh, and one question which has me scratching my head - I can sort of understand how a new address record is getting created in the DB, but what I don't get is the existing address record is also updated to have its customer_id set to NULL...how the heck does that happen? That leads me to believe that EF does see the original address record is somehow linked (as it is modifying it) but it's not smart enough to realize the record I'm passing in should replace it?
Thanks -- also, this is EF6 and MVC5

The problem comes from the line
existingCustomer.Addresses = customer.Addresses;
in your code. This like assigns field Addresses from customer coming from the model. So far ok. The point is that customer does not have any relation to the database model at this point (it's not coming from the database but from the view).
If you would like to update existingCustomer.Addresses with the data coming from the model, you need to merge the data instead of replacing it. The following "pseudo code" might give you a direction:
void MergeAddresses(var existingAddresses, var newAddresses) {
foreach(var address in newAddresses) {
if (existingAddresses.Contains(newAddress)) {
// merge fields if applicable
}
else {
// add field to existingAddresses - be ware to use a "cloned" list
}
}
// now delete items from existing list
foreach (var address in existingAddresses.CloneList()) {
if (!newAddresses.Contains(address)) {
// remove from existingAddresses
}
}
}

Is this the best way to go about this, or are there any EF tricks to iterating and syncing a child collection to the DB?
No, there aren't such tricks. EF designers left saving detached entities totally up to us - the developers.
However there is a package called GraphDiff which is addressing that, so you could give it a try. Here is how your code would look like using it:
using RefactorThis.GraphDiff;
...
_context.UpdateGraph(customer, map => map.OwnedCollection(
e => e.Addresses, with => with.AssociatedEntity(e => e.Customer)));
_context.SaveChanges();

Why is my Seed() method not saving and connecting entities correctly?

I am building a registration site for a conference for my organization, with multiple VIPs and guest speakers. The requirement is to track many details about each attendee including their arrival and departure plans and their local lodging information. In order to facilitate discussion with stakeholders on the types of reports we need to build, I want to populate my dev database with a batch of records from a CSV containing randomly generated information like name, arrival/departure date/time, etc. This will allow us to look at a working site without having to register and re-register many times.
However, I simply cannot get the Seed method to persist the relevant records properly, yet my controller which handles the registration works perfectly.
My database structure is basically an Attendee entity with child entities for TravelSchedule, LodgingArrangement, and various lookups. Here are excerpts from my entities:
public class Attendee
{
public int Id { get; set; }
... other strings/etc ...
public virtual TravelSchedule TravelSchedule { get; set; }
public int TravelScheduleId { get; set; }
public virtual LodgingArrangment LodgingArrangement { get; set; }
public int LodgingArrangementId { get; set; }
}
public class TravelSchedule
{
public int Id { get; set; }
... other properties ...
public virtual Attendee Attendee { get; set; }
public int AttendeeId { get; set; }
}
public class LodgingArrangement
{
public int Id { get; set; }
... other properties ...
public virtual Attendee Attendee { get; set; }
public int AttendeeId { get; set; }
}
Here is the content of my context's OnModelCreating method:
modelBuilder.Entity<Attendee>()
.HasOptional(a => a.TravelSchedule)
.WithRequired(r => r.Attendee);
modelBuilder.Entity<TravelSchedule>()
.HasRequired(m => m.ArrivalMode)
.WithMany(m => m.Arrivals)
.HasForeignKey(m => m.ArrivalModeId)
.WillCascadeOnDelete(false);
modelBuilder.Entity<TravelSchedule>()
.HasRequired(m => m.DepartureMode)
.WithMany(m => m.Departures)
.HasForeignKey(m => m.DepartureModeId)
.WillCascadeOnDelete(false);
modelBuilder.Entity<Attendee>()
.HasOptional(a => a.LodgingArrangement)
.WithRequired(l => l.Attendee);
The following is an excerpt from my Seed method.
var attendees = GetAttendeesFromCsv();
context.Attendees.AddOrUpdate(a => a.Email, attendees.ToArray());
context.SaveChanges();
var dbAttendees = context.Attendees.ToList();
foreach (var attendee in dbAttendees)
{
attendee.TravelSchedule = CreateTravelSchedule();
context.Entry<Attendee>(attendee).State = EntityState.Modified;
context.SaveChanges();
}
GetAttendeesFromCsv() extracts the records from the CSV into Attendee objects, thanks to the CsvHelper package. CreateTravelSchedule creates a new TravelSchedule entity and populates it with data from lookup tables using the SelectRandom() method from extensionmethod.com. The bottom line is that I extract the CSV rows into Attendee objects, add a new randomly-generated TravelSchedule entity, and save the resulting Attendee with attached TravelSchedule.
Except this does not work. Instead the above code adds the TravelSchedule records to the database, but the AttendeeId is always set to 0 in the table. Also the Attendees table is populated with all of the records from the CSV, but the TravelScheduleId on each row is always 0 as well. However, when stepping through the update-database call with the debugger the attendee.Id is populated properly, so by my understanding EF should pick up that the two are related and persist the related TravelSchedule at the same time as the Attendee. So why isn't EF connecting the two records?
Changing the loop to this:
foreach (var attendee in dbAttendees)
{
var travel = CreateTravelSchedule();
travel.AttendeeId = attendee.Id; // I also tried just travel.Attendee = attendee, without success
context.TravelSchedules.Add(travel);
context.SaveChanges();
}
Results in this error:
System.Data.Entity.Core.UpdateException: An error occurred while updating the entries. See the inner exception for details. ---> System.Data.SqlClient.SqlException: The INSERT statement conflicted with the FOREIGN KEY constraint "FK_dbo.TravelSchedules_dbo.Attendees_Id". The conflict occurred in database "MySite.DAL.MyContext", table "dbo.Attendees", column 'Id'.
So it appears I cannot add the TravelSchedule entity to the Attendee, and I also cannot go "backwards" by creating the TravelSchedule and then attaching the Attendee.
The frustrating part is that the registration form logic in my controller works perfectly fine, excerpt below. The walkthrough is that the registration controller stores each screen's data (view models) in the session using a static WorkflowManager class which handles persistence between screens. After user confirmation the controller pulls each screen's details from the WorkflowManager, runs them through AutoMapper to convert them to the relevant populated DAL entities, attaches those entities to the attendee entity, and saves it all to the database.
Again, this works perfectly fine, saving the attendee and its two child entities without error. Here is the relevant excerpt of the controller action:
var attendeeRegistration = WorkflowManager.GetAttendeeRegistration();
var travelRegistration = WorkflowManager.GetTravelRegistration();
using (var db = new MyContext())
{
var attendee = Mapper.Map<Attendee>(attendeeRegistration);
attendee.AnotherChildEntity = db.ChildEntities.Find(attendeeRegistration.SelectedChildEntityId);
var travel = Mapper.Map<TravelSchedule>(travelRegistration);
travel.ArrivalMode = db.TravelModes.Find(travelRegistration.SelectedArrivalModeId);
travel.DepartureMode = db.TravelModes.Find(travelRegistration.SelectedDepartureModeId);
var lodging = Mapper.Map<LodgingArrangement>(lodgingRegistration);
lodging.LodgingLocation = db.LodgingLocations.Find(lodgingRegistration.SelectedLodgingLocationId);
attendee.Comments = comments;
attendee.TravelSchedule = travel;
attendee.LodgingArrangement = lodging;
db.Attendees.Add(attendee);
db.SaveChanges();
}
This works perfectly. None of these objects are in the database until after the user confirms the registration is complete. So I don't understand why I can persist new entities to the database here, yet I can't do what appears to me to be the same thing in the Seed method above.
Any help or ideas much appreciated. This has been causing me endless grief for days now. Thanks.

The association between Attendee and TravelSchedule is 1:1. EF implements 1:1 associations by creating a primary key in the dependent entity (here: TravelSchedule) that's also a foreign key to its principal entity (Attendee).
So Attendee.TravelScheduleId and TravelSchedule.AttendeeId are not used for the association and their values remain 0. Which means: the Seed works without these fields/properties (and I'd even expect it to work with them), it establishes the associations through the Id fields.

Entity Framework - seeding database - does PK need to be included in Seed method?

Following the MVC Music Store tutorial, the database is seeded with sample data. The Album class is as follows:
namespace MvcMusicStore.Models
{
public class Album
{
public int AlbumId { get; set;}
// ...
}
}
Then in the sample data class that's used in the tutorial, an Album object is created like this within the Seed() method:
new Album { Title = "The Best Of Billy Cobham",
Genre = genres.Single(g => g.Name == "Jazz"),
Price = 8.99M, Artist = artists.Single(a => a.Name == "Billy Cobham"),
AlbumArtUrl = "/Content/Images/placeholder.gif" }
In my project I have a class Tenant:
public class Tenant
{
[key]
public int TenantId { get; set;}
// ...
}
In my Seed() method, I create a Tenant like this:
new Tenant { UserId=1, UserName="Matthew" /*...*/ }
In my Seed() method I've included the Tenant PK which is UserId - Tenant is derived from User. I'm wondering, because in the Seed() method I've explicitly said that the Tenant's PK of UserId is 1, will that cause a problem when the method run? I ask because in the music store tutorial, the author hasn't included an Album's PK.
So the PK isn't explicitly stated when creating an Album. Will EF apply a PK by default? I've just spent a while creating quite a lot of sample data for my own project, and I've put the PK for a lot of entities in manually - will EF accept this or will it cause me problems? Is seeding the best way to generate sample data - alternatives?

When you don't add a [Key] attribute, EF will try to find an int readable - writeable property named Id or <yourClassName>Id.
That's all. And That's why AlbumId is taken as PK for class Album : No key attribute exists in that class, AlbumId = <nameOfClass>Id, it's a readable-writeable int.
If you have set a Key attribute, EF won't try to find another one. It's just a convention, which allow you to be less verbose.
For sample datas, Seed() method's body is a fine way to put them in Code First.