i am struggeling for a while now to understand how EF loads / updates entities.
First of all i wanna explain what my app (WPF) is about. I am developing
an application where users can store Todo Items in Categories, these categories are predefined by the application. Each user can read all items but can only delete / update his own items. It's a multiuser system, means the application is running multiple times in the network accessing the same sql server database.
When a user is adding/deleting/updating items the UI on all the other running apps has to update.
My model looks like this:
public class Category
{
public int Id { get; set; }
public string Name { get; set; }
public List<Todo> Todos { get; set; }
}
public class Todo
{
public int Id { get; set; }
public string Content { get; set; }
public DateTime LastUpdate { get; set; }
public string Owner { get; set; }
public Category Category { get; set; }
public List<Info> Infos { get; set; }
}
public class Info
{
public int Id { get; set; }
public string Value { get; set; }
public Todo Todo { get; set; }
}
I am making the inital load like this, which works fine:
Context.dbsCategories.Where(c => c.Id == id).Include(c => c.Todos.Select(t => t.Infos)).FirstOrDefault();
Now i was trying to load only the Todos which are from the current user therefore i tried this:
Context.dbsCategories.Where(c => c.Id == id).Include(c => c.Todos.Where(t => t.Owner == Settings.User).Select(t => t.Infos)).FirstOrDefault();
This does not work because it's not possible to filter within include, so I tried this:
var cat = Context.dbsCategories.Where(c => c.Id == id).FirstOrDefault();
Context.dbsTodos.Where(t => t.Category.Id == cat.Id && t.Owner == Settings.User).Include(t=>t.Infos);
After executing the second line where i look for the Todo Items, these Items were automatically added to cat's Todos collection. Why? I would have expected that i have to add them manually to cat's Todos collection.
Just for my understanding what is EF doing here exactly?
Now to my main problem -> the synchronization of the data between database and client. I am using a long running Context which lives as long as the application is running to save changes to the database which are made on owned items. The user does not have the possibility to manipulate / delete data from other users this is guarantee by the user interface.
To synchronize the data i build this Synch Method which will run every 10 second, right now it's triggere manually.
Thats my synchronization Code, which only synchronizes Items to the client that do not belong to it.
private async Task Synchronize()
{
using (var ctx = new Context())
{
var database = ctx.dbsTodos().Where(x => x.Owner != Settings.User).Select(t => t.Infos).AsNoTracking();
var loaded = Context.dbsTodos.Local.Where(x => x.Owner != Settings.User);
//In local context but not in database anymore -> Detachen
foreach (var detach in loaded.Except(database, new TodoIdComparer()).ToList())
{
Context.ObjectContext.Detach(detach);
Log.Debug(this, $"Item {detach} detached");
}
//In database and local context -> Check Timestamp -> Update
foreach (var update in loaded.Intersect(database, new TodoIdTimeStampComparer()))
{
await Context.Entry(update).ReloadAsync();
Log.Debug(this, $"Item {update} updated");
}
//In database but not in local context -> Attach
foreach (var attach in database.ToList().Except(loaded, new TodoIdComparer()))
{
Context.dbsTodos().Attach(attach);
Log.Debug(this, $"Item {attach} attached");
}
}
}
I am having following problems / issues of unknow origin with it:
Detaching deleted Items seems to work, right now i am not sure if only the Todo Items are detached or also the Infos.
Updating Items works only for the TodoItem itsself, its not reloading the Infos within? How can i reload the whole entity with all it's relations?
I am thankful for every help on this, even if you are saying it's all wrong what i am doing here!
Attaching new Items and Infos does not work so far? What am i doing wrong here?
Is this the right approach to synchronize data between client and database?
What am i doing wrong here? Is there any "How to Sync" Tutorial? I have not found anything helpful so far?
Thanks!
My, you do like to deviate from entity framework code-first conventions, do you?
(1) Incorrect class definitions
The relations between your tables are Lists, instead of ICollections, they are not declared virtual and you forgot to declare the foreign key
There is a one-to-many relation between Todo and Category: every Todo belongs to exactly one Category (using a foreign key), every Category has zero or more Todos.
You choose to give Category a property:
List<Todo> Todos {get; set;}
Are you sure that category.Todos[4] has a defined meaning?
What would category.Todos.Insert(4, new Todo()) mean?
Better stick to an interface where you can't use functions that have no proper meaning in your database: use ICollection<Todo> Todos {get; set;}. This way you'll have only access to functions that Entity Framework can translate to SQL.
Besides, a query will probably be faster: you give entity framework the possibility to query the data in its most efficient way, instead of forcing it to put the result into a List.
In entity framework the columns of a table are represented by non-virtual properties; the virtual properties represent the relations between the tables (one-to-many, many-to-many)
public class Category
{
public int Id { get; set; }
public string Name { get; set; }
... // other properties
// every Category has zero or more Todos (one-to-many)
public virtual ICollection<Todo> Todos { get; set; }
}
public class Todo
{
public int Id { get; set; }
public string Content { get; set; }
... // other properties
// every Todo belongs to exactly one Category, using foreign key
public int CategoryId { get; set }
public virtual Category Category { get; set; }
// every Todo has zero or more Infos:
public virtual ICollection<Info> Infos { get; set; }
}
You'll probably guess Info by now:
public class Info
{
public int Id { get; set; }
public string Value { get; set; }
... // other properties
// every info belongs to exactly one Todo, using foreign key
public int TodoId {get; set;}
public virtual Todo Todo { get; set; }
}
Three major improvements:
ICollections instead of Lists
ICollections are virtual, because it is not a real column in your table,
foreign key definitions non-virtual: they are real columns in your tables.
(2) Use Select instead of Include
One of the slower parts of a database query is the transport of the selected data from the Database Management System to your local process. Hence it is wise to limit the amount of transported data.
Suppose Category with Id [4] has a thousand Todos. Every Todo of this Category will have a foreign key with a value 4. So this same value 4 will be transported 1001 times. What a waste of processing power!
In entity framework use Select instead of Include to query data and select only the properties you actually plan to use. Only use Include if you plan to update the Selected data.
Give me all Categories that ... with their Todos that ...
var results = dbContext.Categories
.Where(category => ...)
.Select(category => new
{
// only select properties that you plan to use
Id = category.Id,
Name = category.Name,
...
Todos = category.Todos
.Where(todo => ...) // only if you don't want all Todos
.Select(todo => new
{
// again, select only the properties you'll plan to use
Id = todo.Id,
...
// not needed, you know the value:
// CategoryId = todo.CategoryId,
// only if you also want some infos:
Infos = todo.Infos
.Select(info => ....) // you know the drill by now
.ToList(),
})
.ToList(),
});
(3) Don't keep DbContext alive for such a long time!
Another problem is that you keep your DbContext open for quite some time. This is not how a dbContext was meant. If your database changes between your query and your update, you'll have troubles. I can hardly imagine that you query so much data that you need to optimize it by keeping your dbContext alive. Even if you query a lot of data, the display of this huge amount of data would be the bottle-neck, not the database query.
Better fetch the data once, dispose the DbContext, and when updating fetch the data again, update the changed properties and SaveChanges.
fetch data:
RepositoryCategory FetchCategory(int categoryId)
{
using (var dbContext = new MyDbContext())
{
return dbContext.Categories.Where(category => category.Id == categoryId)
.Select(category => new RepositoryCategory
{
... // see above
})
.FirstOrDefault();
}
}
Yes, you'll need an extra class RepositoryCategory for this. The advantage is, that you hide that you fetched your data from a database. Your code would hardly change if you'd fetch your data from a CSV-file, or from the internet. This is way better testable, and also way better maintainable: if the Category table in your database changes, users of your RepositoryCategory won't notice it.
Consider creating a special namespace for the data you fetch from your database. This way you can name the fetched Category still Category, instead of RepositoryCategory. You even hide better where you fetched your data from.
Back to your question
You wrote:
Now i was trying to load only the Todos which are from the current user
After the previous improvements, this will be easy:
string owner = Settings.User; // or something similar
var result = dbContext.Todos.Where(todo => todo.Owner == owner)
.Select(todo => new
{
// properties you need
})
Related
Usefull Context
I currently have two entities that look as below.
MovieSerie
public class MovieSerie
{
[Key]
public Guid MovieSerieId { get; set; }
[Required]
[MaxLength(128)]
public string Title { get; set; }
[Required]
[MaxLength(256)]
public string Description { get; set; }
public virtual ICollection<Movie> Movies { get; set; }
}
Movie
public class Movie
{
[Key]
public Guid MovieId { get; set; }
[Required]
[MaxLength(128)]
public string Title { get; set; }
public virtual MovieSerie MovieSerie { get; set; }
}
I have removed some properties that were unused so far so the example is a bit more readable.
These entities have a one-to-many relationship because a MovieSerie contains multiple movies but a movie can only belong to one MovieSerie.
The problem
When I am trying to make a new movie from Postman by providing an EXISTING MovieSerie, I am getting an exception. The exception looks as below.
Duplicate entry '\xA9\xCE\x0E\x1E\x9A\xAE\xA2G\x91<\xE6\xE3-\x88C\xE9' for key 'movieseries.PRIMARY'
So I figured out that it is trying to make a new MovieSerie when I am providing a MovieSerie object. The raw JSON from the request that I am trying to send from Postman looks like below.
{
"MovieId" : "6aa8c134-689c-45e2-bf60-cd0eb5473cc2",
"Title" : "TestMovie",
"MovieSerie" : {
"movieSerieId": "1e0ecea9-ae9a-47a2-913c-e6e32d8843e9",
"title": "Harry Potter",
"description": "This contains the Harry Potter serie"
}
}
The POST method to save the movie is shown below.
[HttpPost]
public async Task<ActionResult<Movie>> PostMovie(Movie movie)
{
if (movie == null)
{
return BadRequest("No movie object provided");
}
else if (movie.MovieSerie != null)
{
if (!_validator.MovieSerieExists(movie.MovieSerie.MovieSerieId))
{
return BadRequest("The movie serie does not exists in the database");
}
}
_context.Movies.Add(movie);
await _context.SaveChangesAsync();
return CreatedAtAction("GetMovie", new { id = movie.MovieId }, movie);
}
Could someone give me any insight into what I am doing wrong? Why is it trying to make a new entity while it already exists? What should I change to get the wished behavior?
I tried to provide all information required, however, let me know if I missed something.
EDIT ADDED DBCONTEXT
modelBuilder.Entity<MovieSerie>(entity =>
{
entity.HasKey(movieSerie => movieSerie.MovieSerieId);
entity.Property(movieSerie => movieSerie.Title).IsRequired();
entity.Property(movieSerie => movieSerie.Description).IsRequired();
entity.HasMany(ms => ms.Movies)
.WithOne(m => m.MovieSerie);
});
modelBuilder.Entity<Movie>(entity =>
{
entity.HasKey(movie => movie.MovieId);
entity.Property(movie => movie.Title).IsRequired();
entity.HasOne(m => m.MovieSerie)
.WithMany(s => s.Movies);
});
This is what happens when passing entities between server and client in ASP.Net. When your DbContext is lifetime scoped to a request, the entities are loaded by a DbContext and passed to the view, but then what you pass back on the Post call is a JSON object that is deserialized into an entity class definition. On this request, neither the Movie or it's associated related entities are tracked by the DbContext.
When you tell the Post's DbContext to Add the movie, any child entities on that movie will be treated as new entities as well, resulting in duplicate records.
How to avoid this:
Option 1: Use ViewModels to avoid confusing data coming from views with entities. (Data state) This is always my recommended option. This avoids confusion about what objects you are dealing with, and also means you can reduce the amount of data being sent over the wire. As entities get larger, sending entities back and forth means larger payloads for fields your view doesn't need. ViewModels can be populated to serve just the fields that the view will interact with. Automapper can help largely with turning entity graphs into ViewModels with it's ProjectTo method.
So if we had a view for creating a Movie (Movie/Create) and that view listed a the movie series to choose from, it might search/fetch series:
[Serializable]
public class MovieSeriesSummaryViewModel
{
public Guid MovieSeriesId { get; set; }
public string Name { get; set; }
}
Then when the controller goes to search/retrieve those series to choose from:
var series = _context.MovieSeries
// .Where(x => [search criteria...])
.ProjectTo<MovieSeriesSummaryViewModel>(config)
.ToList();
or
var series = _context.MovieSeries
// .Where(x => [search criteria...])
.Select( x = > new MovieSeriesSummaryViewModel
{
MovieSeriesId = x.MovieSeriesId,
Name = x.Name
}).ToList();
a PostMovie action accepts a PostMovieViewModel:
[Serializable]
public class PostMovieViewModel
{
public string MovieName { get; set; }
public Guid? MovieSeriesId { get; set; }
// ...
}
The create movie view model only needs to pass the series ID (if applicable) and the required fields to create a new movie. From there we associate the series from the DbContext when creating our new Movie:
[HttpPost]
public async Task<ActionResult<PostMovieViewModel>> PostMovie(PostMovieViewModel movieVM)
{
var movieSeries = movieVM.MovieSeriesId.HasValue
? _context.MovieSeries.Single(x => x.MovieSeriesId == movieVM.MovieSeriesId.Value)
: null;
var movie = new Movie
{
Name = movieVM.Name,
MovieSeries = movieSeries
};
_context.Movies.Add(movie);
await _context.SaveChangesAsync();
}
The key point here is that we fetch the existing series from the Context to associate to the new movie. Fetching entities by ID is quite fast and serves as a meaningful validation that the data we passed in is complete.
Option 2: Re-associate all references. The underlying problem with passing deserialize objects and treating them as entities is that the DbContext isn't tracking them. There are 2 ways you can fix this, either tell the DbContext to track them, or replace the references with tracked objects.
2a - Replacing references
[HttpPost]
public async Task<ActionResult<Movie>> PostMovie(Movie movie)
{
if (movie.MovieSeries != null)
{
var existingMovieSeries = _context.MovieSeries
.Single(x => MovieSeriesId == movie.MovieSeries.MovieSeriesId);
movie.MovieSeries = existingMovieSeries; // Replace the reference.
}
_context.Movies.Add(movie);
await _context.SaveChanges();
}
This still potentially means going to the DB for all references, and forgetting to will result in silent duplication issues.
2b - Track related entities. This one I saved for last as it can seem simple, but can trip you up...
[HttpPost]
public async Task<ActionResult<Movie>> PostMovie(Movie movie)
{
if (movie.MovieSeries != null)
_context.Attach(movie.MovieSeries);
_context.Movies.Add(movie);
await _context.SaveChanges();
}
That looks simple, and would work most of the time, but if the DbContext is already tracking that movie series for any reason, the Attach method will fail. This is an error that could appear intermittently at runtime depending on the particular actions/data combinations. (I.e. updating 2 movies /w same series or conditionally calling a method that loads that series) The proper check would be:
[HttpPost]
public async Task<ActionResult<Movie>> PostMovie(Movie movie)
{
if (movie.MovieSeries != null)
{
var existingMovieSeries = _context.MovieSeries.Local
.SingleOrDefault(x => x.MovieSeriesId == movie.MovieSeries.MovieSeriesId);
if (existingMovieSeries == null)
_context.Attach(movie.MovieSeries);
else
movie.MovieSeries = existingMovieSeries;
}
_context.Movies.Add(movie);
await _context.SaveChanges();
}
Checking MovieSeries.Local checks to see if the DbContext is tracking the series. (without hitting the DB) If not, we can attach it. If it is, we need to replace the reference. This can be a lot of boiler plate code to put in for every reference on a new object. When attaching entities coming from a view, it is also important not to ever set the entity state for that entity to Modified without first verifying the data is valid. (Which would require loading the entity first anyway) Doing so could allow users to alter data in ways you don't intend as setting an entity to Modified will update all fields on that entity. (Where loading an entity and then copying across values means only those values you change will be updated)
Your problem is that you are passing the whole movie serie object. This is not something you should do. The idea of relational databases is to, as the name suggest, relate tables. This relationships are done using keys (foreign keys).
In your particular case, you need to define a foreign key column in your Movie table, to relate it to MovieSeries, as follows:
public class Movie
{
[Key]
public Guid MovieId { get; set; }
public int MovieSerieId {get; set; }
[Required]
[MaxLength(128)]
public string Title { get; set; }
[ForeignKey("MovieSerieID")]
public virtual MovieSerie MovieSerie { get; set; }
}
As you can see, im specifying that the MovieSerieID attribute is a foreign key. The virtual MovieSerie attribute is used by EF to get all the details of your foreign key.
Now, you can create your movie passing only the MovieSerieid, as follows:
{
"MovieId" : "6aa8c134-689c-45e2-bf60-cd0eb5473cc2",
"Title" : "TestMovie",
"MovieSerieId": "1e0ecea9-ae9a-47a2-913c-e6e32d8843e9"
}
I wrote a query which is pretty simple:
var locations = await _context.Locations
.Include(x => x.LocationsOfTheUsers)
.Include(x => x.Address)
.ThenInclude(x => x.County)
.Where(CalculateFilters(searchObj))
.ToListAsync(cancellationToken);
And everytime LocationsOfTheUsers were null so I decided to .Include(x => x.LocationsOfTheUsers) and I received results as expected but I'm not sure why do I have to include this collections since it's defined like this:
public class Location
{
public string Title { get; set; }
public long? RegionId { get; set; }
public Region Region { get; set; }
public long? AddressId { get; set; }
public Address Address { get; set; }
public long? CountyId { get; set; }
public County County { get; set; }
public ICollection<LocationsOfTheUsers> LocationsOfTheUsers { get; set; }
}
I thought this will be automatically included since it exist as ICollection in Location class.
So why is .Include() on LocationsOfTheUsers needed here?
Thanks guys
Cheers
In entity framework the non-virtual properties represent the columns of the tables, the virtual properties represent the relations between the tables (one-to-many, many-to-many, ...)
So your property should have been defined as:
public virtual ICollection<LocationsOfTheUsers> LocationsOfTheUsers { get; set; }
One of the slower parts of a database query is the transfer of the selected data from the database management system to your local process. Hence it is wise to limit the selected data to the values you actually plan to use.
If you have a one-to-many relation between Schools and Students, and you ask for School [10] you don't want automatically to fetch its 2000 Students.
Even if you would like to have "School [10] with all its Students" it would not be efficient to use Include to also fetch the Students. Every Student will have a foreign key SchoolId with a Value of [10]. If you would use Include you would transfer this foreign key 2000 times. What a waste!
When using entity framework always use Select to fetch data and select only the properties that you actually plan to use. Only use Include if you plan to change the included items.
This way you can separate your database table structure from the actual query. If your database structure changes, only the query changes, users of your query don't notice the internal changes.
Apart from better performance and more robustness against changes, readers of your code can more easily see what values are in their query.
Certainly don't use Include to save you some typing. Having to debug one error after future changes will take way more time than you will ever save by typeing include instead of Select
Finally: limit your data early in your process, so put the Where in front.
So your query should be:
var predicate = CalculateFilters(searchObj)
var queryLocations = dbContext.Locations
.Where(predicate)
.Select(location => new
{
// Select only the location properties that you plan to use
Id = location.Id,
Name = location.Name,
// Locations Of the users:
UserLocations = location.LocationsOfTheUsers
.Select(userLocation => new
{
// again: only the properties that you plan to use
Id = userLocation.Id,
...
// Not needed, you already know the value
// LocationId = userLocation.LocationId
})
.ToList(),
Address = new
{
Street = location.Address.Street,
PostCode = location.Addrress.PostCode,
...
County = location.Address.County.Name // if you only want one property
// or if you want more properties:
County = new
{
Name = location.Address.County.Name,
Abbr = location.Address.Count.Abbr,
...
}),
},
});
I thought this will be automatically included since it exist as ICollection in Location class.
Well, it's not automatically included, probably for performance reasons as the graph of related entities and their recursive child entities may be rather deep.
That's why you use eager loading to explicitly include the related entities that you want using the Include method.
The other option is to use lazy loading which means that the related entities are loaded as soon as you access the navigation property in your code, assuming some prerequisites are fulfilled and that the context is still around when this happens.
Please refer to the docs for more information.
I believe you are using EntityFrameworkCore. In EntityFramework (EF6), lazy loading is enabled by default, However, in EntityFrameworkCore, lazy loading related entities is handled by a separate package Microsoft.EntityFrameworkCore.Proxies.
To enable the behaviour you are seeking, install the above package and add the following code
protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
{
optionsBuilder.UseLazyLoadingProxies();
}
After this, the related entities will be loaded without the Include call.
Given the following set up where there are many Teams and there are many LeagueSessions. Each Team belongs to zero or more LeagueSessions but only ever one LeagueSession is active. LeagueSessions have many teams, and the teams will be repeated. Many-to-many relationship is established between Teams and LeagueSessions with a join table called TeamsSessions.
Team model looks like this:
public class Team
{
public string Id { get; set; }
public string Name { get; set; }
public League League { get; set; }
public string LeagueID { get; set; }
public bool Selected { get; set; }
public ICollection<Match> Matches { get; set; }
public virtual ICollection<TeamSession> TeamsSessions { get; set; }
}
Team model fluent api configuration:
`
public class TeamConfiguration
{
public TeamConfiguration(EntityTypeBuilder<Team> model)
{
// The data for this model will be generated inside ThePLeagueDataCore.DataBaseInitializer.DatabaseBaseInitializer.cs class
// When generating data for models in here, you have to provide it with an ID, and it became mildly problematic to consistently get
// a unique ID for all the teams. In ThePLeagueDataCore.DataBaseInitializer.DatabaseBaseInitializer.cs we can use dbContext to generate
// unique ids for us for each team.
model.HasOne(team => team.League)
.WithMany(league => league.Teams)
.HasForeignKey(team => team.LeagueID);
}
}
`
Each team belongs to a single League. League model looks like this:
`public class League
{
public string Id { get; set; }
public string Type { get; set; }
public string Name { get; set; }
public IEnumerable<Team> Teams { get; set; }
public bool Selected { get; set; }
public string SportTypeID { get; set; }
public SportType SportType { get; set; }
public IEnumerable<LeagueSessionSchedule> Sessions { get; set; }
}`
fluent API for the League:
`public LeagueConfiguration(EntityTypeBuilder<League> model)
{
model.HasOne(league => league.SportType)
.WithMany(sportType => sportType.Leagues)
.HasForeignKey(league => league.SportTypeID);
model.HasMany(league => league.Teams)
.WithOne(team => team.League)
.HasForeignKey(team => team.LeagueID);
model.HasData(leagues);
}`
SessionScheduleBase class looks like this:
public class SessionScheduleBase
{
public string LeagueID { get; set; }
public bool ByeWeeks { get; set; }
public long? NumberOfWeeks { get; set; }
public DateTime SessionStart { get; set; }
public DateTime SessionEnd { get; set; }
public ICollection<TeamSession> TeamsSessions { get; set; } = new Collection<TeamSession>();
public ICollection<GameDay> GamesDays { get; set; } = new Collection<GameDay>();
}
Note: LeagueSessionSchedule inherits from SessionScheduleBase
The TeamSession model looks like this:
`public class TeamSession
{
public string Id { get; set; }
public string TeamId { get; set; }
public Team Team { get; set; }
public string LeagueSessionScheduleId { get; set; }
public LeagueSessionSchedule LeagueSessionSchedule { get; set; }
}`
I then configure the relationship with the fluent API like this:
`public TeamSessionConfiguration(EntityTypeBuilder<TeamSession> model)
{
model.HasKey(ts => new { ts.TeamId, ts.LeagueSessionScheduleId });
model.HasOne(ts => ts.Team)
.WithMany(t => t.TeamsSessions)
.HasForeignKey(ts => ts.TeamId);
model.HasOne(ts => ts.LeagueSessionSchedule)
.WithMany(s => s.TeamsSessions)
.HasForeignKey(ts => ts.LeagueSessionScheduleId);
}`
The problem arises whenever I attempt to insert a new LeagueSessionSchedule. The way I am adding a new TeamSession object onto the new LeagueSessionSchedule is like this:
`foreach (TeamSessionViewModel teamSession in newSchedule.TeamsSessions)
{
Team team = await this._teamRepository.GetByIdAsync(teamSession.TeamId, ct);
if(team != null)
{
TeamSession newTeamSession = new TeamSession()
{
Team = team,
LeagueSessionSchedule = leagueSessionSchedule
};
leagueSessionSchedule.TeamsSessions.Add(newTeamSession);
}
}`
Saving the new LeagueSessionSchedule code:
public async Task<LeagueSessionSchedule> AddScheduleAsync(LeagueSessionSchedule newLeagueSessionSchedule, CancellationToken ct = default)
{
this._dbContext.LeagueSessions.Add(newLeagueSessionSchedule);
await this._dbContext.SaveChangesAsync(ct);
return newLeagueSessionSchedule;
}
Saving the new LeagueSessionSchedule object throws an error by Entity Framework Core that it cannot INSERT a duplicate primary key value into the dbo.Teams table. I have no idea why its attempting to add to dbo.Teams table and not into TeamsSessions table.
ERROR:
INSERT INTO [LeagueSessions] ([Id], [Active], [ByeWeeks], [LeagueID], [NumberOfWeeks], [SessionEnd], [SessionStart])
VALUES (#p0, #p1, #p2, #p3, #p4, #p5, #p6);
INSERT INTO [Teams] ([Id], [Discriminator], [LeagueID], [Name], [Selected])
VALUES (#p7, #p8, #p9, #p10, #p11),
(#p12, #p13, #p14, #p15, #p16),
(#p17, #p18, #p19, #p20, #p21),
(#p22, #p23, #p24, #p25, #p26),
(#p27, #p28, #p29, #p30, #p31),
(#p32, #p33, #p34, #p35, #p36),
(#p37, #p38, #p39, #p40, #p41),
(#p42, #p43, #p44, #p45, #p46);
System.Data.SqlClient.SqlException (0x80131904): Violation of PRIMARY KEY constraint 'PK_Teams'. Cannot insert duplicate key in object 'dbo.Teams'. The duplicate key value is (217e2e11-0603-4239-aab5-9e2f1d3ebc2c).
My goal is to create a new LeagueSessionSchedule object. Along with the creation of this object, I also have to create a new TeamSession entry to the join table (or not if join table is not necessary) to then be able to pick any given team and see what session it is currently a part of.
My entire PublishSchedule method is the following:
`
public async Task<bool> PublishSessionsSchedulesAsync(List<LeagueSessionScheduleViewModel> newLeagueSessionsSchedules, CancellationToken ct = default(CancellationToken))
{
List<LeagueSessionSchedule> leagueSessionOperations = new List<LeagueSessionSchedule>();
foreach (LeagueSessionScheduleViewModel newSchedule in newLeagueSessionsSchedules)
{
LeagueSessionSchedule leagueSessionSchedule = new LeagueSessionSchedule()
{
Active = newSchedule.Active,
LeagueID = newSchedule.LeagueID,
ByeWeeks = newSchedule.ByeWeeks,
NumberOfWeeks = newSchedule.NumberOfWeeks,
SessionStart = newSchedule.SessionStart,
SessionEnd = newSchedule.SessionEnd
};
// leagueSessionSchedule = await this._sessionScheduleRepository.AddScheduleAsync(leagueSessionSchedule, ct);
// create game day entry for all configured game days
foreach (GameDayViewModel gameDay in newSchedule.GamesDays)
{
GameDay newGameDay = new GameDay()
{
GamesDay = gameDay.GamesDay
};
// leagueSessionSchedule.GamesDays.Add(newGameDay);
// create game time entry for every game day
foreach (GameTimeViewModel gameTime in gameDay.GamesTimes)
{
GameTime newGameTime = new GameTime()
{
GamesTime = DateTimeOffset.FromUnixTimeSeconds(gameTime.GamesTime).DateTime.ToLocalTime(),
// GameDayId = newGameDay.Id
};
// newGameTime = await this._sessionScheduleRepository.AddGameTimeAsync(newGameTime, ct);
newGameDay.GamesTimes.Add(newGameTime);
}
leagueSessionSchedule.GamesDays.Add(newGameDay);
}
// update teams sessions
foreach (TeamSessionViewModel teamSession in newSchedule.TeamsSessions)
{
// retrieve the team with the corresponding id
Team team = await this._teamRepository.GetByIdAsync(teamSession.TeamId, ct);
if(team != null)
{
TeamSession newTeamSession = new TeamSession()
{
Team = team,
LeagueSessionSchedule = leagueSessionSchedule
};
leagueSessionSchedule.TeamsSessions.Add(newTeamSession);
}
}
// update matches for this session
foreach (MatchViewModel match in newSchedule.Matches)
{
Match newMatch = new Match()
{
DateTime = match.DateTime,
HomeTeamId = match.HomeTeam.Id,
AwayTeamId = match.AwayTeam.Id,
LeagueID = match.LeagueID
};
leagueSessionSchedule.Matches.Add(newMatch);
}
try
{
leagueSessionOperations.Add(await this._sessionScheduleRepository.AddScheduleAsync(leagueSessionSchedule, ct));
}
catch(Exception ex)
{
}
}
// ensure all leagueSessionOperations did not return any null values
return leagueSessionOperations.All(op => op != null);
}
`
This is not a many-to-many relationship.
It is two separate one-to-many relationships, which happen to refer to the same table on one end of the relationship.
While it is true that on the database level, both use cases are represented by three tables, i.e. Foo 1->* FooBar *<-1 Bar, these two cases are treated differently by Entity Framework's automated behavior - and this is very important.
EF only handles the cross table for you if it is a direct many-to-many, e.g.
public class Foo
{
public virtual ICollection<Bar> Bars { get; set; }
}
public class Bar
{
public virtual ICollection<Foo> Foos { get; set; }
}
EF handles the cross table behind the scenes, and you are never made aware of the existence of the cross table (from the code perspective).
Importantly, EF Core does not yet support implicit cross tables! There is currently no way to do this in EF Core, but even if there were, you're not using it anyway, so the answer to your problem remains the same regardless of whether you're using EF or EF Core.
However, you have defined your own cross table. While this is still representative of a many-to-many relationship in database terms, it has ceased to be a many-to-many relationship as far as EF is concerned, and any documentation you find on EF's many-to-many relationships no longer applies to your scenario.
Unattached but indirectly added objects are assumed to be new.
By "indirectly added", I mean you that it was added to the context as part of another entity (which you directly added to the context). In the following example, foo is directly added and bar is indirectly added:
var foo = new Foo();
var bar = new Bar();
foo.Bar = bar;
context.Foos.Add(foo); // directly adding foo
// ... but not bar
context.SaveChanges();
When you add (and commit) a new entity to the context, EF adds it for you. However, EF also looks at any related entities that the first entity contains. During the commit in the above example, EF will look at both the foo and bar entities and will handle them accordingly. EF is smart enough to realize that you want bar to be stored in the database since you put it inside the foo object and you explicitly asked EF to add foo to the database.
It is important to realize that you've told EF that foo should be created (since you called Add(), which implies a new item), but you never told EF what it should do with bar. It's unclear (to EF) what you expect EF to do with this, and thus EF is left guessing at what to do.
If you never explained to EF whether bar already exists or not, Entity Framework defaults to assuming it needs to create this entity in the database.
Saving the new LeagueSessionSchedule object throws an error by Entity Framework Core that it cannot INSERT a duplicate primary key value into the dbo.Teams table. I have no idea why its attempting to add to dbo.Teams table
Knowing what you now know, the error becomes clearer. EF is trying to add this team (which was the bar object in my example) because it has no information on this team object and what its state in the database is.
There are a few solutions here.
1. Use the FK property instead of the navigational property
This is my preferred solution because it leaves no room for error. If the team ID does not yet exist, you get an error. At no point will EF try to create a team, since it doesn't even know the team's data, it only knows the (alleged) ID you're trying to create a relationship with.
Note: I am omitting LeagueSessionSchedule as it is unrelated to the current error - but it's essentially the same behavior for both Team and LeagueSessionSchedule.
TeamSession newTeamSession = new TeamSession()
{
TeamId = team.Id
};
By using the FK property instead of the nav prop, you are informing EF that this is an existing team - and therefore EF no longer tries to (re)create this team.
2. Ensure that the team is tracked by the current context
Note: I am omitting LeagueSessionSchedule as it is unrelated to the current error - but it's essentially the same behavior for both Team and LeagueSessionSchedule.
context.Teams.Attach(team);
TeamSession newTeamSession = new TeamSession()
{
Team = team
};
By attaching the object to the context, you are informing it of its existence. The default state of a newly attached entity is Unchanged, meaning "this already exists in the database and has not been changed - so you don't need to update it when we commit the context".
If you have actually made changes to your team that you want to be updated during commit, you should instead use:
context.Entry(team).State = EntityState.Modified;
Entry() inherently also attaches the entity, and by setting its state to Modified you ensure that the new values will be committed to the database when you call SaveChanges().
Note that I prefer solution 1 over solution 2 because it's foolproof and much less likely to lead to unexpected behavior or runtime exceptions.
String primary keys are undesirable
I'm not going to say that it doesn't work, but strings cannot be autogenerated by Entity Framework, making them undesirable as the type of your entity's PK. You will need to manually set your entity PK values.
Like I said, it's not impossible, but your code shows that you're not explicitly setting PK values:
if(team != null)
{
TeamSession newTeamSession = new TeamSession()
{
Team = team,
LeagueSessionSchedule = leagueSessionSchedule
};
leagueSessionSchedule.TeamsSessions.Add(newTeamSession);
}
If you want your PK's to be automatically generated, use an appropriate type. int and Guid are by far the most commonly used types for this.
Otherwise, you're going to have to start setting your own PK values, because if you don't (and the Id value thus defaults to null), your code is going to fail when you add a second TeamSession object using the above code (even though you're doing everything else correctly), since PK null is already taken by the first entity you added to the table.
I have been facing this problem some time, and to be honest I am myself confused with it so please excuse me if i don't succeed explaining it as I should.
I am trying to insert some data into a Table called CommunicationAttachment which is related as One to Many relationship with Communication; every communication could have many attachments.
The thing is that I get:
UpdateException: Invalid Column Name: "Communication_CommunicationId
when I try to insert list of attachments.
And please note that I am using the repository pattern but I even tried the normal way and the issue wasn't fixed.
I tried tracing the transaction that happens on the database and I figured out that it sends Communication_CommunicationId with the Insert statement, yet there is no such column. I am pretty sure I didn't send such a column.
Here is my code (this is happening when adding new Communication); first of all I call CasefileAttachments to make copies from them, and Communications are related to CaseFiles:
public List<CorrespondenceAttachment> GetCaseFileAttachments(List<Guid> CorrespondenceAttachmentIds)
{
List<CorrespondenceAttachment> originalAttachments = new List<CorrespondenceAttachment>();
foreach (var item in CorrespondenceAttachmentIds)
{
var attachment = QueryData.Query<CorrespondenceAttachment>().Where(att => att.CorrespondenceAttachmentID == item).FirstOrDefault();
originalAttachments.Add(attachment);
}
return originalAttachments;
}
Then I copy the CaseFileAttachments and create new objects of CommunicationAttachments :
public List<CommunicationAttachment> CopyCaseFileAttachmentsToCommunication(List<CorrespondenceAttachment> originalAttachments,Guid communicationId)
{
var communicationAttachments = new List<CommunicationAttachment>();
if (originalAttachments.Any())
{
foreach (var attachmentRef in originalAttachments)
{
var CommunicationAttachmentId = Guid.NewGuid();
communicationAttachments.Add(new CommunicationAttachment()
{
CommunicationAttachmentId = CommunicationAttachmentId,
DmsFileId = CommunicationAttachmentId,
CommunicationId = communicationId,
AttachmentTitle = attachmentRef.AttachmentTitle,
MimeType = attachmentRef.MimeType,
NewVersionID = null,
UploadDate = DateTime.Now,
Size = attachmentRef.Size,
Version = "0001",
AttachmentsGroupId = attachmentRef.AttachmentsGroupId,
DocumentId = attachmentRef.DocumentId,
RelativePath = attachmentRef.RelativePath,
Extension = attachmentRef.Extension,
AttachmentSubject = attachmentRef?.AttachmentSubject,
ExternalContactID = attachmentRef?.ExternalContactID,
AttachmentNumber = string.IsNullOrEmpty(attachmentRef?.AttachmentNumber) ? null : attachmentRef.AttachmentNumber,
TemplatedmsId = attachmentRef.TemplatedmsId,
State = eSense.Framework.Data.ObjectState.Added,
});
}
}
return communicationAttachments;
}
and the methods above are called something like this way:
public void AddNewCommunication(CommunicationDto communicationDto)
{
var communication = communicationDto
if (communicationDto.CommunicationAttachmentIdList.Any())
{
caseFileAttachments = GetCaseFileAttachments(communicationDto.CommunicationAttachmentIdList);
if (caseFileAttachments.Any())
{
commAttachments = CopyCaseFileAttachmentsToCommunication(caseFileAttachments, communication.CommunicationId);
}
}
communication.Attachments = commAttachments;
Save(communication)
}
So what could be the problem that I get a wrong column name?
Here is the relation between Communication and CommunicationAttachment
Note I added only the Important fields so don't bother if the declaring does not match the entity
Communication Entity:
public class Communication : BaseEntity
{
public Communication()
{
Attachments = new HashSet<CommunicationAttachment>();
}
[Key]
public Guid CommunicationId { get; set; }
public string Subject { get; set; }
public string CommunicationNumber { get; set; }
public virtual ICollection<CommunicationAttachment> Attachments { get; set; }
public DateTime DateCreated { get; set; }
public Guid? PreviousCommunicationId { get; set; }
[ForeignKey("PreviousCommunicationId")]
public virtual Communication PreviousCommunication { get; set; }
}
CommunicationAttachment Entity:
public class CommunicationAttachment : AttachmentBaseWithDelegation<Guid>
{
public override Guid PrimaryId
{
get
{
return this.CommunicationAttachmentId;
}
}
public CommunicationAttachment()
{
}
[Key]
public Guid CommunicationAttachmentId { get; set; }
private string _attachmentNumber;
public string AttachmentNumber { get; set; }
[ForeignKey("NewVersionID")]
public virtual CommunicationAttachment CaseFileAttachmentNewerVersion { get; set; }
public Guid CommunicationId { get; set; }
[ForeignKey("CommunicationId")]
public virtual Communication Communication { get; set; }
}
Sorry if you found it hard to understand my question I myself is confused!
Thanks in advance.
This is typically a case where a relationship between entities is not set up correctly. It would appear that EF should be resolving this relationship by convention if Communication's PK is "CommunicationId".
I notice that you've commented out a line to set the CommunicationId on the new entity:
//CommunicationId = communicationId,
What fields are in the CommunicationAttachment? is there a CommunicationId? Is there a Communication navigation property? What configuration settings are you are using?
For example, with fluent configuration I would have something like:
(CommunicationEntityConfiguration)
If CommunicationAttachment has a navigation property back to Communication and a FK field called CommunicationId...
HasMany(x => x.CommunicationAttachments)
.WithRequired(x => x.Communication)
.HasForeignKey(x => x.CommunicationId);
If the attachment entity has a navigation property without a mapped FK in the entity...
HasMany(x => x.CommunicationAttachments)
.WithRequired(x => x.Communication)
.Map(x => x.MapKey("CommunicationId"));
If the attachment entity does not have a navigation property, but has a FK in the entity...
HasMany(x => x.CommunicationAttachments)
.WithRequired()
.HasForeignKey(x => x.CommunicationId);
Or lastly if the attachment entity does not have a navigation property nor a mapped FK...
If the attachment entity does not have a navigation property, but has a FK in the entity...
HasMany(x => x.CommunicationAttachments)
.WithRequired()
.Map(x => x.MapKey("CommunicationId"));
I am a big fan of explicit mapping over convention as it is very clear as to what maps to what, and how, in order to resolve potential mapping conflicts. If the rest of the similar relations seem to be working and just this one is playing up, I'd be looking for possible typos in the field names. With a mapped collection like above, setting a Communcation.CommunicationAttachments.Add(attachment) should be setting the FK / related entity on the attachment without having to explicitly set the FK or related entity manually.
One additional note:
From your example I see you are setting Primary Keys manually client-side using Guid.NewGuid(). It is generally better to allow the database to manage PK generation and let EF manage FK assignment to ensure that related entities get the FKs to newly inserted rows automatically. Rather than SQL's NewId() or using Guid.NewGuid(), it is advisable to use sequential UUIDs. In SQL Server this is NewSequentialId(). For client-side setting, you can reproduce the sequential UUID pattern either with a system DLL call to get the ID, or a simple re-hash of the Guid bytes. see: Is there a .NET equalent to SQL Servers newsequentialid()
The GUIDs still carry the same uniqueness, the bytes are simply arranged to be more sequential and practical for database indexing to reduce page fragmentation. The downside is that IDs are more predictable. Depending on your database engine you might want to customize the algorithm based on whether the database is optimized for indexing on the lower-order or high-order bytes.
When using GUIDs for database, sequential or otherwise, you should ensure you have a scheduled index maintenance job on the database. With sequential IDs this job will run faster and keep the index tables more compact.
I have a Customer class that has a relationship to an Address class:
public class Customer
{
public int Id { get; set; }
public string Name { get; set; }
public virtual ICollection<Address> Addresses { get; set; }
}
public class Address
{
public int Id { get; set; }
public string Street1 { get; set; }
//Snip a bunch of properties
public virtual Customer Customer { get; set; }
}
I have an edit form which displays all the fields for both the customer and address. When this form is submitted, it calls the Edit method in the controller:
public ActionResult Save(Customer customer)
{
if (!ModelState.IsValid)
{
var viewModel = new CustomerFormViewModel
{
Customer = customer,
CustomerTypes = _context.CustomerTypes.ToList()
};
return View("CustomerForm", viewModel);
}
if (customer.Id == 0)
_context.Customers.Add(customer);
else
{
var existingCustomer = _context.Customers
.Include(c => c.Addresses)
.Single(c => c.Id == customer.Id);
existingCustomer.Name = customer.Name;
existingCustomer.TaxId = customer.TaxId;
existingCustomer.CustomerTypeId = customer.CustomerTypeId;
existingCustomer.CreditLimit = customer.CreditLimit;
existingCustomer.Exempt = customer.Exempt;
existingCustomer.Addresses = customer.Addresses;
}
_context.SaveChanges();
return RedirectToAction("Index", "Customers");
}
This doesn't work and creates duplicate entries in the Addresses table in the DB. I think I understand why (EF isn't smart enough to know the Addresses inside the collection need to be added/modified/deleted as the case may be). So, what is the best way to fix this?
My instinct is that I need to iterate over the Addresses collections and compare them manually, adding any new ones from the form that don't exist for the customer, updating ones that do exist, and deleting ones that were not sent by the form but exist in the DB for the customer. Something like (ignoring the delete functionality for now):
foreach(Address address in customer.Addresses)
{
if (address.Id == 0)
// Add record
else
// Fetch address record from DB
// Update data
}
// Save context
Is this the best way to go about this, or are there any EF tricks to iterating and syncing a child collection to the DB?
Oh, and one question which has me scratching my head - I can sort of understand how a new address record is getting created in the DB, but what I don't get is the existing address record is also updated to have its customer_id set to NULL...how the heck does that happen? That leads me to believe that EF does see the original address record is somehow linked (as it is modifying it) but it's not smart enough to realize the record I'm passing in should replace it?
Thanks -- also, this is EF6 and MVC5
The problem comes from the line
existingCustomer.Addresses = customer.Addresses;
in your code. This like assigns field Addresses from customer coming from the model. So far ok. The point is that customer does not have any relation to the database model at this point (it's not coming from the database but from the view).
If you would like to update existingCustomer.Addresses with the data coming from the model, you need to merge the data instead of replacing it. The following "pseudo code" might give you a direction:
void MergeAddresses(var existingAddresses, var newAddresses) {
foreach(var address in newAddresses) {
if (existingAddresses.Contains(newAddress)) {
// merge fields if applicable
}
else {
// add field to existingAddresses - be ware to use a "cloned" list
}
}
// now delete items from existing list
foreach (var address in existingAddresses.CloneList()) {
if (!newAddresses.Contains(address)) {
// remove from existingAddresses
}
}
}
Is this the best way to go about this, or are there any EF tricks to iterating and syncing a child collection to the DB?
No, there aren't such tricks. EF designers left saving detached entities totally up to us - the developers.
However there is a package called GraphDiff which is addressing that, so you could give it a try. Here is how your code would look like using it:
using RefactorThis.GraphDiff;
...
_context.UpdateGraph(customer, map => map.OwnedCollection(
e => e.Addresses, with => with.AssociatedEntity(e => e.Customer)));
_context.SaveChanges();