I'm using entity framework in .net core, Note that my knowledge of entity framework is somewhat limited, so take my assumptions with a pinch of salt.
Teams & Users are the two objects that I have trouble with.
They have a joining table, UserTeam because Users can have many Teams, and Teams can have many Users.
I figured out how to have them saved in a sqlite database using entityframework, and it's all fine. It took me a while to realize I had to use Include to get the joining property, but now it works in that part.
At some point, I have to expose that object on the API, so I have matching UserDTO & TeamDTO. There is my issue. UserDTOs have a list of TeamDTOs, and TeamDTOs have a list of UserDTOs, because that's how they should look like. Right?
But when I am mapping them, I get into a StackOverflowException, which is normal, because I'm looping on the list of teams, which contains Users, and for each of those Users, I end up recreating Teams, and so on.
Right now, I just added a boolean check in the loop, so when I call .ToDTO() I can decide to skip the users/teams from inner levels, but that does not look like a proper fix to me. Is it?
What do you guys suggest ?
Here is what I'm doing exactly :
public static TeamDTO ToDTO(this Team team, EatupContext context, bool doOnce = false)
{
var dto = new TeamDTO
{
Id = team.Id,
Name = team.Name,
};
var users = new List<UserDTO>();
foreach (var userTeam in team.UsersTeams)
{
var user = context.Users.Find(userTeam.UserId);
if (doOnce)
user.TeamsUsers = new List<UserTeam>();
users.Add(user.ToDTO(context, true));
}
dto.Users = users;
return dto;
}
public static UserDTO ToDTO(this User user, EatupContext context, bool doOnce = false)
{
var dto = new UserDTO
{
Id = user.Id,
NickName = user.NickName,
Email = user.Email,
Image = user.Image,
};
var teams = new List<TeamDTO>();
foreach (var userTeam in user.TeamsUsers)
{
var team = context.Teams.Find(userTeam.TeamId);
if (doOnce)
team.UsersTeams = new List<UserTeam>();
teams.Add(team.ToDTO(context, true));
}
dto.Teams = teams;
return dto;
}
Because I have a strong feeling my issue might come from there, I'm gonna share this piece of information :
When I'm looping through the entity objects UserTeams(of either a team or a user), the Team object is null, so I have to fetch the Team from the context using its ID, and then I have everything I need. It's very strange because everything appears to be working fine and the database is complete and healthy. Its just the UserTeam object that is missing the Team property.
But my assumption is that between fetching a new Team from the ID, and getting the Team from the UserTeam, both would create the same result I have now when I call ToDTO(): I still would need to loop through its users and have that overflow exception.
This seems like a very common issue that should have been dealt with by a lot of people, but I can't really find my answer using the keywords of my title.
What's my next step ?
You're getting the stackoverflow due to the fact that you're circularly calling the 2 methods.
In the Team Version of the ToDTO method, you call the user version, which in turn calls the team version and so on.
If your model is not structured something like below you should consider it. There's no need to actually model UserTeam as EFs mapping should take care of this. That's considering that you've specified one end of the relationship as the primary.
public class Team
{
public Team()
{
Users = new List<User>();
}
//properties we don't care about
public virtual ICollection<User> Users {get;set;}
}
public class User
{
public User()
{
Teams = new List<User>();
}
//properties we don't care about
public virtual ICollection<Team> Teams {get;set;}
}
just turn off lazy loading and use include method when you need to fetch lists of objects.
this is for DbContext class
public class BloggingContext : DbContext
{
public BloggingContext()
{
this.Configuration.LazyLoadingEnabled = false;
}
}
and this is for the controllers
using (var context = new BloggingContext())
{
// Load all blogs, all related posts, and all related comments.
var blogs1 = context.Blogs
.Include(b => b.Posts.Select(p => p.Comments))
.ToList();
// Load all users, their related profiles, and related avatar.
var users1 = context.Users
.Include(u => u.Profile.Avatar)
.ToList();
// Load all blogs, all related posts, and all related comments
// using a string to specify the relationships.
var blogs2 = context.Blogs
.Include("Posts.Comments")
.ToList();
// Load all users, their related profiles, and related avatar
// using a string to specify the relationships.
var users2 = context.Users
.Include("Profile.Avatar")
.ToList();
}
source : https://learn.microsoft.com/en-us/ef/ef6/querying/related-data
Related
I've been trying to take advantage of a new way of creating many-to-many relationships - nice article about EF 5 many-to-many relationships.
The article states that you no longer need to define relation class and the framework does the job for you.
However, for a couple of hours now I've been struggling to add an existing entity to the collection of another entity.
My models
public record Bottle
{
[Key]
public int Id { get; set; }
[Required]
public string Username { get; set; }
// some other properties
public Collection<User> Owners { get; set; }
}
public record User
{
[Key]
public int Id { get; set; }
// some other properties
public Collection<Bottle> Bottles { get; set; }
}
Say that I want to add a new bottle to the database. I also know owners of that bottle. I had thought that this bit of code could work:
public async Task<int> AddBottle(BottleForAddition bottle)
{
var bottleEntity = mapper.Map<Bottle>(bottle);
bottleEntity.Owners = bottle
.OwnerIds // List<int>
.Select(id => new User { Id = id })
.ToCollection(); // my extension method
var createdEntity = await context.AddEntityAsync(bottleEntity);
await context.SaveChangesAsync();
return createdEntity.Entity.Id;
}
but sadly it does not work (BottleForAddition is DTO with almost the same properties).
I get this error:
Unable to create bottle (error: Microsoft.EntityFrameworkCore.DbUpdateException: An error occurred while updating the entries. See the inner exception for details.
Microsoft.Data.Sqlite.SqliteException (0x80004005): SQLite Error 19: 'NOT NULL constraint failed: Users.Username'.
at Microsoft.Data.Sqlite.SqliteException.ThrowExceptionForRC(Int32 rc, sqlite3 db)
at Microsoft.Data.Sqlite.SqliteDataReader.NextResult()
at ...
So I came up with this
public async Task<int> AddBottle(BottleForAddition bottle)
{
var bottleEntity = mapper.Map<Bottle>(bottle);
bottleEntity.Owners = (await context.Users
.Where(u => bottle.OwnerIds.Contains(u.Id))
.ToListAsync())
.ToCollection();
var createdEntity = await context.AddEntityAsync(bottleEntity);
await context.SaveChangesAsync();
return createdEntity.Entity.Id;
}
That works but I have to fetch Users from the database.
Do you know about a better way how to deal with it?
The Users table in the database has a Username field does not allow NULL
You are creating new User entities from the OwnerIds which doesn't have Username value set
EF is trying to insert a new user to the Users table
Combining the pieces of information above, you'll get a clear picture why the error message says -
SQLite Error 19: 'NOT NULL constraint failed: Users.Username'.
Then comes the real question, why EF is trying to insert new users at all. Obviously, you created the User entities from the OwnerIds to add already existing users to the list, not to insert them.
Well, I'm assuming that the AddEntityAsync() method you are using (I'm not familiar with it) is an extension method, and inside it, you are using the DbContext.Add() or DbSet<TEntity>.Add() method. Even if that is no the case, apparently AddEntityAsync() at least works similarly as them.
The Add() method causes the entity in question (Bottle) and all it's related entities (Users) present in the entity-graph to be marked as Added. An entity marked as Added implies - This is a new entity and it will get inserted on the next SaveChanges call. Therefore, with your first approach, EF tried to insert the User entities you created. See details - DbSet<TEntity>.Add()
In your second approach, you fetched the existing User entities first. When you fetch existing entities using the DbContext, EF marks them as Unchanged. An entity marked as Unchanged implies - This entity already exists in the database and it might get updated on the next SaveChanges call. Therefore, in this case the Add method caused only the Bottle entity to be marked as Added and EF didn't try to re-insert any User entities you fetched.
As a general solution, in a disconnected scenario, when creating new entity with an entity-graph (with one or more related entities) use the Attach method instead. The Attach method causes any entity to be marked as Added only if it doesn't have the primary-key value set. Otherwise, the entity is marked as Unchanged. See details - DbSet<TEntity>.Attach()
Following is an example -
var bottleEntity = mapper.Map<Bottle>(bottle);
bottleEntity.Owners = bottle
.OwnerIds // List<int>
.Select(id => new User { Id = id })
.ToCollection(); // my extension method
await context.Bottles.Attach(bottleEntity);
await context.SaveChangesAsync();
Not related to the issue :
Also, since you are already using AutoMapper, if you define your BottleForAddition DTO as something like -
public class BottleForAddition
{
public int Id { get; set; }
public string Username { get; set; }
// some other properties
public Collection<int> Owners { get; set; } // the list of owner Id
}
then you will be able to configure/define your maps like -
this.CreateMap<BottleForAddition, Bottle>();
this.CreateMap<int, User>()
.ForMember(d => d.Id, opt => opt.MapFrom(s => s));
which could simplify the operation code like -
var bottleEntity = mapper.Map<Bottle>(bottle);
await context.Bottles.Attach(bottleEntity);
await context.SaveChangesAsync();
Fetching the Users is generally the correct course of action. This allows you to make the associations but also helps validate that the reference IDs passed from the client are valid. Fetching entities by ID is generally quite fast, so I'd consider avoiding async/await for this operation. async is suited for large or high-frequency operations where server responsiveness could be "hung up". Using it everywhere just leads to slower operations overall.
EF will want to use proxies for navigation properties both for lazy loading (not to be relied on as a crutch, but useful to avoid errors as a worst-case) as well as for change tracking.
public record Bottle
{
[Key]
public int Id { get; set; }
[Required]
public string Username { get; set; }
// some other properties
public virtual ICollection<User> Owners { get; set; } = new List<User>();
}
then in the applicable code...
var bottleEntity = mapper.Map<Bottle>(bottle);
var users = context.Users
.Where(u => bottle.OwnerIds.Contains(u.Id))
.ToList();
foreach(var user in users)
bottleEntity.Users.Add(user);
// Or since dealing with a new Entity could do this...
//((List<User>)bottleEntity.Users).AddRange(users);
await context.SaveChangesAsync();
return bottleEntity.Id;
It might be tempting to just create the users and attach them to the DbContext and much of the time this would work, except if there is ever the possibility that the DbContext might have been tracking an instance of any of those to-be-attached users, which will result in a runtime error that an entity with the same ID is already being tracked.
var bottleEntity = mapper.Map<Bottle>(bottle);
var proxyUsers = bottle.OwnerIds
.Select(x => new User { Id = x }).ToList();
foreach(var user in proxyUsers)
{
context.Users.Attach(user);
bottleEntity.Users.Add(user);
}
await context.SaveChangesAsync();
return bottleEntity.Id;
This requires either turning off all entity tracking or remember to always query entities with AsNoTracking which can lead to additional work and intermitted bugs appearing if this isn't adhered to consistently. To deal with possible tracked entities is a fair bit more work:
var bottleEntity = mapper.Map<Bottle>(bottle);
var proxyUsers = bottle.OwnerIds
.Select(x => new User { Id = x }).ToList();
var existingUsers = context.Users.Local
.Where(x => bottle.OwnerIds.Contains(x.Id)).ToList();
var neededProxyUsers = proxyUsers.Except(existingUsers, new UserIdComparer()).ToList();
foreach(var user in neededProxyUsers)
context.Users.Attach(user);
var users = neededProxyUsers.Union(existingUsers).ToList();
foreach(var user in users)
bottleEntity.Users.Add(user);
await context.SaveChangesAsync();
return bottleEntity.Id;
Any existing tracked entity needs to be found and referenced in place of an attached user reference. The other caveat of this approach is that the "proxy" users created for non-tracked entities are not complete user records so later code expecting to get User records from the DbContext could receive these attached proxy rows and result in things like null reference exceptions etc. for fields that were not populated.
Hence, fetching the references from the EF DbContext to get the relatable entities is generally the best/simplest option.
I noticed that when I store an object having a property being of another type, it gets saved but so is the linked in property. It's kind of neat when both are new or both are being updated.
class Customer
{
public Guid Id { get; set; }
public List<Category> Categories { get; set; } ...
}
class Category
{
public Guid Id { get; set; }
public string Name { get; set; } ...
}
However, when I create a new instance of customer but linked to a pre-existing category, it becomes a bit inappropriate. Category in this case is like a tag, so several customers will share a pointer to it and each new customer needs only to reuse the already present category. (If the ID of the category is set as unique, I get error for object already created and when it's not, I get duplicates.)
I understand that it has to do with the state of the objects in relation to EF engine. It simply isn't aware that the category in the new customer is already present in the DB, hence trying to create it.
I played with the status and got into some complicated algorithm, which sort of impacted on readability. Then, I came up with the following approach. I'm not actually doing anything with the data - I simply pull it into the client-side of the EF making it aware of their existence.
List<Guid> ids = data.Categories
.Select(_ => _.Id).ToList();
List<Category> existing = Context.Categories
.Where(_ => ids.Contains(_.Id)).ToList();
To make it simpler, let's assume that the number of the categories is limited, hence reformulating the retrieval to the code below. (It's basically like saying "Hmmm... Speaking of the categories... Well, nevermind.", which seems academically wrong.)
List<Categories> _ = Context.Categories.ToList();
Is there a pattern or approach that is best-practice recommended in this scenario?
I realize that the samples above are just Q&D workaround. The mentioned entity status is complicated. I get the sense that there's a neat way of resolving it.
You do not need to load all existing Categories into memory. Rather, just associate the new Customer with the existing Categories he has by loading them from the Context.
Guid[] categoryIdsOfNewCustomer; // passed as part of the create command
var newCustomer = new Customer();
newCustomer.Categories = Context.Categories
.Where(c => categoryIdsOfNewCustomer.Contains(c.Id))
.ToList();
Context.Customers.Add(newCustomer);
Context.SaveChanges();
A beginner in SQL/DB here - I'm designing a universal windows app where a DB might come in handy, so I'm reading on SQL/SQLite and I'm wondering - how does backwards compatibility (classes/tables wise) works? Suppose I add or remove a property from a class that I've been using as a base for a table - will I still be able to interact with the old - class data and cast it into the updated class? How does one go about monitoring the added/ removed properties while querying the db?
Thank you for your time.
Edit: Scenario - if I've got a User with 2 properties - 'Id', 'Name'.
public class User
{
public int Id {get; set; }
public string Name {get; set;}
}
And I've built an application with a 'Users' Db and I'm using that for a while. Then, I would like to add another property to my User- class. Suppose it's 'Age'. Can I add this new property to my original 'User' class and still be able to access the original Users table with:
var user = DbConnection.Table<User>().Where(x => x.Id == tId).FirstOrDefault();
Obviously, one option would be to keep the original User and create another class UserEx which will have the original properties + Age. Then I grab all 'User's, port them to UserEx and save to a new table.
This does seem a bit cumbersome to do for each added/ removed property though
Can I add this new property to my original 'User' class and still be
able to access the original Users table with ?
var user = DbConnection.Table<User>().Where(x => x.Id == tId).FirstOrDefault();
Does adding a property to the user class update the underlying SQLLite
table? You would need to extend the class to maintain compatibility.
public class User
{
public int Id {get; set; }
public string Name {get; set;}
public User getUser(int tId)
{
var user = DbConnection.Table<User>().Where(x => x.Id == tId).FirstOrDefault();
Id = user.Id;
Name = user.Name;
//age = user.age; // not possible
}
}
public class DetailedUser:User
{
public int age { get; set; }
public DetailedUser getUser(int tId)
{
var user = DbConnection.Table<User>().Where(x => x.Id == tId).FirstOrDefault();
base.Id = user.Id;
base.Name = user.Name;
//age = user.age; // not possible
}
public DetailedUser getDetailedUser(int tId)
{
var user = DbConnection.Table<DetailedUser>().Where(x => x.Id == tId).FirstOrDefault();
base.Id = user.Id;
base.Name = user.Name;
age = user.age;
}
}
The obvious 'cumbersome' process you outlined is another valid option. Either way when you are changing the data layer in your application there are consequences.
Two things come to mind, one is that in Android SqlLite has a onUpgrade function whey you can put your "cumbersome" data layer upgrade logic.. or you could use a NoSQL solution like MongoDB where the data layer is much more forgiving about the structure of your underlying records (ymmv).
Finally the better you plan your data layer structure the fewer times you will run into this kind of issue.
I hope that is of some help.
I'm trying to get my head around this issue where I am using the Entity Framework (6) in an N-tier application. Since data from the repository (which contains all communication with the database) should be used in a higher tier (the UI, services etc), I need to map it to DTOs.
In the database, there's quite a few many-to-many relationships going on, so the datastructure can/will get complex somewhere along the line of the applications lifetime. What I stumbled upon is, that I am repeating the exact same code when writing the repository methods. An example of this is my FirmRepository which contains a GetAll() method and GetById(int firmId) method.
In the GetById(int firmId) method, I have the following code (incomplete since there's a lot more relations that needs to be mapped to DTOs):
public DTO.Firm GetById(int id)
{
// Return result
var result = new DTO.Firm();
try
{
// Database connection
using (var ctx = new MyEntities())
{
// Get the firm from the database
var firm = (from f in ctx.Firms
where f.ID == id
select f).FirstOrDefault();
// If a firm was found, start mapping to DTO object
if (firm != null)
{
result.Address = firm.Address;
result.Address2 = firm.Address2;
result.VAT = firm.VAT;
result.Email = firm.Email;
// Map Zipcode and City
result.City = new DTO.City()
{
CityName = firm.City.City1,
ZipCode = firm.City.ZipCode
};
// Map ISO code and country
result.Country = new DTO.Country()
{
CountryName = firm.Country.Country1,
ISO = firm.Country.ISO
};
// Check if this firm has any exclusive parameters
if (firm.ExclusiveParameterType_Product_Firm.Any())
{
var exclusiveParamsList = new List<DTO.ExclusiveParameterType>();
// Map Exclusive parameter types
foreach (var param in firm.ExclusiveParameterType_Product_Firm)
{
// Check if the exclusive parameter type isn't null before proceeding
if (param.ExclusiveParameterType != null)
{
// Create a new exclusive parameter type DTO
var exclusiveParameter = new DTO.ExclusiveParameterType()
{
ID = param.ExclusiveParameterType.ID,
Description = param.ExclusiveParameterType.Description,
Name = param.ExclusiveParameterType.Name
};
// Add the new DTO to the list
exclusiveParamsList.Add(exclusiveParameter);
}
}
// A lot more objects to map....
// Set the list on the result object
result.ExclusiveParameterTypes = exclusiveParamsList;
}
}
}
// Return DTO
return result;
}
catch (Exception e)
{
// Log exception
Logging.Instance.Error(e);
// Simply return null
return null;
}
}
This is just one method. The GetAll() method will then have the exact same mapping logic which results in duplicated code. Also, when more methods gets added, i.e. a Find or Search method, the same mapping needs to be copied again. This is, of course, not ideal.
I have read a lot about the famous AutoMapper framework that can map entites to/from DTOs, but since I have these many-to-many relations it quickly feels bloated with AutoMapper config code. I've also read this article, which make sense in my eyes: http://rogeralsing.com/2013/12/01/why-mapping-dtos-to-entities-using-automapper-and-entityframework-is-horrible/
Is there any other way of doing this without copy/pasting the same code over and over again?
Thanks in advance!
You can make an extension method on Entity firm (DB.Firm) like this,
public static class Extensions
{
public static DTO.Firm ToDto(this DB.Firm firm)
{
var result = new DTO.Firm();
result.Address = firm.Address;
result.Address2 = firm.Address2;
//...
return result;
}
}
Then you can convert DB.Firm object anywhere in your code like firm.ToDto();
An alternate strategy is to use a combination of the class constructor and an explicit and/or implicit conversion operator(s). It allows you to cast one user-defined entity to another entity. The feature also has the added benefit of abstracting the process out so you aren't repeating yourself.
In your DTO.Firm class, define either an explicit or implicit operator (Note: I am making assumptions about the name of your classes):
public class Firm {
public Firm(DB.Firm firm) {
Address = firm.Address;
Email = firm.Email;
City = new DTO.City() {
CityName = firm.City.City1;
ZipCode = firm.City.ZipCode;
};
// etc.
}
public string Address { get; set;}
public string Email { get; set; }
public DTO.City City { get; set; }
// etc.
public static explicit operator Firm(DB.Firm f) {
return new Firm(f);
}
}
You can then use it in your repository code like this:
public DTO.Firm GetById(int id) {
using (var ctx = new MyEntities()) {
var firm = (from f in ctx.Firms
where f.ID == id
select f).FirstOrDefault();
return (DTO.Firm)firm;
}
}
public List<DTO.Firm> GetAll() {
using (var ctx = new MyEntities()) {
return ctx.Firms.Cast<DTO.Firm>().ToList();
}
}
Here's the reference in MSDN.
About mapping: it actually does not really matter if you use Automapper or prepare you mappings completely manually in some method (extension one or as explicit casting operator as mentioned in other answers) - the point is to have it in one place for reusability.
Just remember - you used FirstOrDefault method, so you actually called the database for a Firm entity. Now, when you are using properties of this entity, especiallly collections, they will be lazy loaded. If you have a lot of them (as you suggest in your question), you may face a huge amount of additional call and it might be a problem, especcially in foreach loop. You may end up with dozen of calls and heavy performace issues just to retrieve one dto. Just rethink, if you really need to get such a big object with all its relations.
For me, your problem is much deeper and considers application architecture. I must say, I personally do not like repository pattern with Entity Framework, in addition with Unit Of Work pattern. It seems to be very popular (at least of you take a look at google results for the query), but for me it does not fit very well with EF. Of course, it's just my opinion, you may not agree with me. For me it's just building another abstraction over already implemented Unit Of Work (DbContext) and repositories (DbSet objects). I found this article very interesing considering this topic. Command/query separation way-of-doing-things seems much more elegant for me, and also it fits into SOLID rules much better.
As I said, it's just my opinion and you may or may not agree with me. But I hope it gives you some perpective here.
This is my first experience with EF so I'm probably doing something stupid. Any comments on the architecture are welcome.
So I have the typical class of Users. Users have a username and a list of roles:
public class User
{
public string UserID{ get; set; }
public List<Role> Roles { get; set; }
public int Id { get; set; }
public User()
{
Roles = new List<Role>();
}
}
My domain objects live in their own code library along with the interfaces for their repositories. So in this case there would be an IUserRepository with all the CRUD methods plus any specialized data access methods I might need. What I'm trying to do is implement these repository interfaces with EF4 in another class library. Any problems with this design so far?
Now in the db (sql server) I have the typical tables: Users, Roles, and a many-to-many table mapping users to roles UsersRoles.
I have successfully set up most of the CRUD methods in the EF lib. Here is what Save looks like
public void Save(Drc.Domain.Entities.Staff.User member)
{
using (var ctx = new DrcDataContext())
{
var efUser = MapFromDomainObject(member);
if(member.Id < 1)
{
ctx.Users.AddObject(efUser);
}
else
{
ctx.Users.Attach(efUser);
ctx.ObjectStateManager.ChangeObjectState(efUser, EntityState.Modified);
}
ctx.SaveChanges();
member.Id = efUser.UserId;
}
}
Now I'm not sure if this is the proper way of accomplishing this but it works. However, I run into problems when doing a delete. The problem is with the related tables
public void Delete(Drc.Domain.Entities.Staff.User member)
{
using (var ctx = new DrcDataContext())
{
var efUser = MapFromDomainObject(member); ctx.Users.Attach(efUser);
while (efUser.Roles.Count > 0)
{
ctx.ObjectStateManager.ChangeObjectState(efUser.Roles.First(), EntityState.Deleted);
}
ctx.SaveChanges();
ctx.ObjectStateManager.ChangeObjectState(efUser, EntityState.Deleted);
ctx.SaveChanges();
}
}
If I don't delete the roles in the while loop I get a DELETE conflict with reference constraint error. If I run the code above it does delete the proper rows in the many-to-many table but it also deletes rows in the Roles table. I'm at a dead end now and considering scraping the ORM idea and writing my repository implementations in good ole reliable ADO.net.
--Edit I'm guessing that this is not the correct way to implement repositories with EF. Is it possible to do without littering your domain with a bunch of EF-centric stuff?
Use simply the standard approach and don't mess around with the entity state:
public void Delete(Drc.Domain.Entities.Staff.User member)
{
using (var ctx = new DrcDataContext())
{
var efUser = MapFromDomainObject(member);
ctx.Users.Attach(efUser);
ctx.Users.DeleteObject(efUser);
ctx.SaveChanges();
}
}
There is usually a cascading delete in the database from the User table to the join table (if you didn't disable it by hand). So deleting the user will delete the corresponding rows in the join table as well (but not the roles of course).
Setting the state of an entity to Deleted is not the same as calling DeleteObject. It will only set the parent to deleted and leave the children in an undeleted state in the context, leading to the constraint violation exception. DeleteObject will also mark the children in the context as Deleted and therefore avoid the exception.