I need to join multiple tables using repository pattern & Entity Framework (using C#). Is this possible? If so, please let me know how to do the same.
In EF, joining tables is done through the use of Navigation Properties. Basically, EF does it for you. When implementing in your Repositories, may it be Generic or not, you can call the Include method when building your query expression to tell EF to populate the navigation properties for you.
Let's say we have these POCO class:
public class Dog
{
public int DogId { get; set; }
public string Name { get; set; }
public int OwnerId { get; set;}
public Owner Owner { get; set; } // the navigation property
}
public class Owner
{
public int OwnerId { get; set; }
public string Name { get; set; }
// another navigation property
// all the dogs that are related or owned by this specific owner
public ICollection<Dog> DogList { get; set; }
public ICollection<Cat> CatList { get; set; }
}
Here's a sample code snippet using Include:
public virtual IEnumerable<Dog> Retrieve()
{
var _query = context.Dog.Include(a => a.Owner);
...
...// rest of your code
}
And for multiple tables you can nest the include method like so:
public virtual IEnumerable<Owner> Retrieve()
{
// you can nest as many as you want if there are more nav properties
var _query = context.Owner
.Include(a => a.DogList)
.Include(a => a.CatList);
...
...// rest of your code
}
Once you include nav properties then that is basically joining those other tables. Just look at the SQL being generated by the query. Hope this helps!
Related
Is there a better way to include additional fields/columns in EF Core without loading the whole navigation property
i have two entities say Employee and Department.
public class Employee
{
public int Id { get; set; }
public string Name { get; set; }
public int DepartmentId { get; set; }
public string DepartmentName { get; set; }
[JsonIgnore]
public Department Department { get; set; }
}
public class Department
{
public int Id { get; set; }
public string Name { get; set; }
// ... more fields.
}
i need to return list of Employees to the client, but i dont want to include the whole Department entity which is only for server side process, except the DepartmentId and DepartmentName.
i have two solutions.
Solution 1 is to use calculated property and navigation property.
public class Employee
{
// ...
public string DepartmentName
{
get
{
if (Department != null)
{
return Department.Name;
}
else
{
return null;
}
}
}
}
// Query for department name but also have to include all fields.
public static List<Employee> GetEmployees(MyDbContext context)
{
return context.Employees.Include(e => e.Department).ToList();
}
solution 2 is to use dynamic joining.
public class Employee
{
// ...
[NotMapped]
public string DepartmentName { get; set; }
}
// Query for department name without navigation property.
public static List<Employee> GetEmployees(MyDbContext context)
{
Func<Employee, Department, Employee> map = (em, dept) =>
{
em.DepartmentName = dept.Name;
return em;
};
var query = from em in context.Employees
join dept in context.Departments on em.DepartmentId equals dept.Id
select map(em, dept);
return query.ToList();
}
but if there 're more foreign fields to include, the linq query will be tedious. Another problem is, this is not generic way for arbitrary entity, and i have more entities to implement like above.
i want to know if there's any other elegent implementation or official way.
For methods which are running within the scope of a DbContext, as appears to be your case by passing through the DbContext instance, I would recommend having them return IQueryable<TEntity> rather than List<TEntity>. In this way the consumers can further refine the query as they see fit. This includes projecting the entity structure into what is needed, (Needing just a DepartmentName) handling things like sorting, pagination, etc. This is commonly the case when developing a Repository pattern.
public class Employee
{
public int Id { get; set; }
public string Name { get; set; }
public virtual Department Department { get; set; }
}
public class Department
{
public int Id { get; set; }
public string Name { get; set; }
// ... more fields.
}
// Then somewhere in a Repository class...
public IQueryable<Employee> GetEmployees(MyDbContext context)
{
return context.Employees.AsQueryable();
}
The first question that usually comes up is "what's the point? why not just use the DbContext?". There can be two good reasons for implementing a repository pattern like this. Firstly to enable unit testing. Mocking a DbContext is "messy", where-as mocking a dependency that has a method that returns IQueryable<Employee> is easy. The other reason is that systems often have low level rules to enforce, such as systems that use soft-delete (i.e. IsActive=false rather than deleting rows) or multi-tenancy. (I.e. employees for multiple ClientId that access the system) Repositories can serve as excellent boundaries to ensure these low-level rules are enforced.
public static IQueryable<Employee> GetEmployees(MyDbContext context, bool includeInactive = false)
{
var clientId = ClientService.ResolveCurrentUserClient();
var query context.Employees.Where(x => x.ClientId == clientId);
if(!includeInactive)
query = query.Where(x => x.IsActive);
return query;
}
In the above example the ClientService would be a dependency that is configured to check the current user and their association to a tenancy client to filter data by, then optionally filter out only active rows.
If you're not planning on implementing unit tests and have no low level rules to enforce, then my recommendation would be to just use the DbContext than adding a layer of abstraction.
When it comes to methods that return data outside of the DbContext's scope, the best solution I can recommend is to use projection to return a data object (View Model or DTO) rather than an entity. The view model can represent only the data your consumer needs and you can either leverage Select or Automapper's ProjectTo to populate it directly from an IQueryable resulting in a minimum sized payload and maximum performance.
[Serializable]
public class EmployeeViewModel
{
public int Id { get; set; }
public string Name { get; set; }
public int DepartmentId { get; set; }
public string DepartmentName { get; set; }
public static MapperConfigurationExpression BuildMapExpression(MapperConfigurationExpression expression = null)
{
if (expression == null)
expression = new MapperConfigurationExpression();
expression.CreateMap<Employee, EmployeeViewModel>()
.ForMember(x => x.DepartmentId, opt => opt.MapFrom(src => src.Department.Id))
.ForMember(x => x.DepartmentName, opt => opt.MapFrom(src => src.Department.Name));
return expression;
}
}
Then in the controller action or other method that would serialize our Employee & Department info:
public ViewResult List()
{
using(var context = new MyDbContext())
{
var config = new MapperConfiguration(EmployeeViewModel.BuildMapExpression());
var employees = EmployeeRepository.GetEmployees(context)
.ProjectTo<EmployeeViewModel>(config)
.ToList(); // Consider pagination /w Skip/Take
return View(employees);
}
}
So rather than serializing an Employee entity and worrying about the serializer lazy loading data or having a bunch of #null references, the method packaging up the data would project the resulting IQueryable down into a DTO or ViewModel class that is safe for serialization. This produces highly efficient queries for both performance and memory use. I cover off why to avoid sending entities beyond the scope of their DBContext and why they should always be considered complete, or complete-able in my response to this question:
What's the real difference between EntityState.Deleted and Remove() method? When to use each of them?
If I need to load only certain properties I usually go this way . Add to employee class extra property DepartmentName and add attribute [NotMaped] (or Ignore in fluent)
public class Employee
{
public int Id { get; set; }
public string Name { get; set; }
.......
[NotMapped]
public string DepartmentName { get; set; }
}
action
public static List<Employee> GetEmployees(MyDbContext context)
{
return context.Employees
.Select(e=> new Employee
{
Id=e.Id,
Name=e.Name,
DepartmentName=e.Department.Name
}).ToList();
}
I don't like to use mapper, it is very limited, so if I need to include lots of properties I usually to this
public static List<Employee> GetEmployees(MyDbContext context)
{
var employees= context.Employees
.Include(i=> Department)
.ToList();
foreach(var item in employees)
{
item.DepartmentName=item.Department.Name
.....
..... another calculated fiels if needed
item.Department=null;
}
return employees;
}
I'm using SQL Server Change Tracking and I'm trying to adapt this article from Microsoft Docs to an Entity Framework application: Work with Change Tracking.
I want to run this SQL query using Entity Framework:
SELECT
P.*, CT.*
FROM
dbo.Product AS P
RIGHT OUTER JOIN
CHANGETABLE(CHANGES dbo.Product, #last_synchronization_version) AS CT
ON
P.ProductID = CT.ProductID
This is what I've got so far:
public class Product
{
public int ProductID { get; set; }
// omitted dozens of other properties
}
public class ProductChange
{
public int ProductID { get; set; }
public Product? Product { get; set; }
public long SYS_CHANGE_VERSION { get; set; }
public long? SYS_CHANGE_CREATION_VERSION { get; set; }
public char SYS_CHANGE_OPERATION { get; set; }
public byte[]? SYS_CHANGE_COLUMNS { get; set; }
public byte[]? SYS_CHANGE_CONTEXT { get; set; }
}
protected override void OnModelCreating(ModelBuilder modelBuilder)
{
modelBuilder.Entity<ProductChange>()
.HasNoKey()
.ToView(null);
base.OnModelCreating(modelBuilder);
}
long lastSynchronizationVersion = ...; // obtained as described in "Work with Change Tracking"
const string sql = #"
SELECT
P.*, CT.*
FROM
dbo.Product AS P
RIGHT OUTER JOIN
CHANGETABLE(CHANGES dbo.Product, {0}) AS CT
ON
P.ProductID = CT.ProductID";
var changes = await dbContext.Set<ProductChange>.FromSqlRaw(sql, lastSynchronizationVersion);
It does not work, because EF does not understand how P.* maps to public Product? Product { get; set; }. When I remove the Product property and remove P.* from the query, things work as expected. However, I need all of the properties, not just the ID.
Copying all of Product's properties into ProductChange and making them all nullable works, but I really don't want to resort to doing that.
In practice I will be using Change Tracking not just for products, but for dozens of entity types, which all have many properties. Having to specify each property in two places just to make Entity Framework play nice with Change Tracking is not a good idea.
Is there a way to get Keyless Entity Types to do what I want? Or do I have to use ADO.NET's ExecuteReader and manually map the result?
It turns out you can use relationships with navigation properties on keyless entity types, just like you can with entity types.
Configure the relationship in OnModelCreating:
modelBuilder.Entity<ProductChange>()
.HasNoKey()
.HasOne(x => x.Entity).WithMany().HasForeignKey(x => x.ProductID) // I added this line.
.ToView(null);
Now you can use Include instead of manually joining tables:
const string sql = "SELECT * FROM CHANGETABLE(CHANGES dbo.Product, {0}) AS CT";
var changes = await dbContext
.Set<ProductChange>
.FromSqlRaw(sql, lastSynchronizationVersion)
.Include(x => x.Entity)
.DefaultIfEmpty() // https://stackoverflow.com/a/63006304/1185136
.ToArrayAsync();
// it works!
Additionally (this is optional), I created base types Change and Change<TEntity> that can be inherited from easily:
public abstract class Change
{
public long Version { get; set; }
public long? CreationVersion { get; set; }
public char Operation { get; set; }
public byte[]? Columns { get; set; }
public byte[]? Context { get; set; }
}
public abstract class Change<TEntity> : Change
where TEntity : class
{
public TEntity? Entity { get; set; }
}
public ProductChange : Change<Product>
{
public int ProductID { get; set; }
}
public OrderChange : Change<Order>
{
public int OrderID { get; set; }
}
// etc...
You'll have to configure the relationship for each derived type in OnModelCreating.
modelBuilder.Entity<ProductChange>()
.HasOne(x => x.Entity).WithMany().HasForeignKey(x => x.ProductID);
modelBuilder.Entity<OrderChange>()
.HasOne(x => x.Entity).WithMany().HasForeignKey(x => x.OrderID);
// etc...
You won't have to repeat HasNoKey() and ToView(null) for every enitity though, add this loop instead:
foreach (var changeType in modelBuilder.Model.FindLeastDerivedEntityTypes(typeof(Change)))
{
var builder = modelBuilder.Entity(changeType.ClrType).HasNoKey().ToView(null);
builder.Property(nameof(Change.Version)).HasColumnName("SYS_CHANGE_VERSION");
builder.Property(nameof(Change.CreationVersion)).HasColumnName("SYS_CHANGE_CREATION_VERSION");
builder.Property(nameof(Change.Operation)).HasColumnName("SYS_CHANGE_OPERATION");
builder.Property(nameof(Change.Columns)).HasColumnName("SYS_CHANGE_COLUMNS");
builder.Property(nameof(Change.Context)).HasColumnName("SYS_CHANGE_CONTEXT");
}
If you want to, you can move the ID property to Change<TEntity>. By doing this, you can remove the ProductChange, OrderChange etc. classes. But, you'll have to specify the column name so Entity Framework Core understands the ID property from Change<Product> maps to ProductID, and the ID property from Change<Order> maps to OrderID, etc. I opted not to do this because this approach won't work if you have composite keys.
I am building sort of multi tenant application with shared tables using .NET Core 2.0 and EF Core.
I am also using generic repository together with Unit of Work if it matters.
I want to make it properly secured and also avoid repeating the logic, so I think if it's possible to somehow modify the DbContext which I am using to for every find operation add something like: entity => entity.tenantId == userContext.tenantId.
I also have to ensure that while creating the correct tenantId is applied and do not authorize update of other tenant property, but so far this logic is included in Service Layer - correct me if I am wrong with this approach?
The IUserContext is defined in Domain abstractions and the application layer implements it differently (API or Web App), but I am not sure if it is not code smell/anti pattern when data layer is doing this kind of logic? ( I am afraid it is).
Should this logic go to the Services (it will then have to be repeated many times which is not good idea I think), DbContext or should I adjust the repository in some way?
So what you want is that if someone would write some Linq statement like this
var result = myDbcontext.myDbSet.SomeLinq(...)
It would internally be like
var result = myDbContext.myDbSet
.Where(entity => entity.tenantId == userContext.tenantId)
.SomeLinq(...)
So what you should do, is that when users think that they access myDbContext.myDbSet, they actually get the subset where tenantId == userContext.tenantId
I think the neat solution would be to create a class that exposes an IQueryable for every DbSet in your DbContext and hides the actual DbContext.
Something like this:
class MyOriginalDbContext : DbContext
{
public DbSet<Student> Students {get; set;}
public DbSet<Teacher> Teachers {get; set;}
public DbSet<ClassRoom> ClassRooms {get; set;}
...
}
public MyLimitedContext : IDisposable
{
// to be filled in constructor
private readonly MyOriginalDbcontext dbContext = ...
private readonly int tenantId = ...
IQueryable<Student> Students
{
get
{
return this.dbContext.Students
.Where(student => student.tenantId == tenantId);
}
}
IQueryable<Student> Teachers
{
get
{
return this.dbContext.Teachers
.Where(teacher => teacher.tenantId == tenantId);
}
}
...
Users won't notice the difference:
using (var dbContext = new MyLimitedContext(...))
{
var TeachersWithTheirStudents = dbContext.Teachers
.Join(dbContext.Student)
.GroupBy(teacher => teacher.Id,
...
}
You can use Global Query Filters. Read more here.
Such filters are automatically applied to any LINQ queries involving
those Entity Types, including Entity Types referenced indirectly, such
as through the use of Include or direct navigation property references
An example:
public class Blog
{
private string _tenantId;
public int BlogId { get; set; }
public string Name { get; set; }
public string Url { get; set; }
public List<Post> Posts { get; set; }
}
public class Post
{
public int PostId { get; set; }
public string Title { get; set; }
public string Content { get; set; }
public bool IsDeleted { get; set; }
public int BlogId { get; set; }
public Blog Blog { get; set; }
}
protected override void OnModelCreating(ModelBuilder modelBuilder)
{
modelBuilder.Entity<Blog>().Property<string>("TenantId").HasField("_tenantId");
// Configure entity filters
modelBuilder.Entity<Blog>().HasQueryFilter(b => EF.Property<string>(b, "TenantId") == _tenantId);
modelBuilder.Entity<Post>().HasQueryFilter(p => !p.IsDeleted);
}
You can even disable filter for individual LINQ query if you need:
blogs = db.Blogs
.Include(b => b.Posts)
.IgnoreQueryFilters()
.ToList();
I'm building a Blog Comment and Reply section and I have these three classes mapped to my DB. The first class holds a collection of related comments to an article, the second class holds a collection of related remarks to the comments:
public class Article
{
public int ArticleID { get; set; }
public byte[] Image { get; set; }
public string Title { get; set; }
public string Body { get; set; }
public DateTime DatePublished { get; set; }
public string Author { get; set; }
public CategoryTyp Category { get; set; }
public virtual ICollection<Comment> Comments { get; set; }
}
public class Comment
{
public int CommentID { get; set; }
public int ArticleID { get; set; }
public int CategoryID { get; set; }
public int UserID { get; set; }
public string Description { get; set; }
public DateTime CommentDate { get; set; }
public virtual ICollection<Remark> Remarks { get; set; }
}
public class Remark
{
public int RemarkID { get; set; }
public int CommentID { get; set; }
public int ArticleID { get; set; }
public string RemarkDetail { get; set; }
public DateTime RemarkTime { get; set; }
}
And inside my Controller:
public ActionResult GetArticle(int id)
{
var article = db.Articles.Include("Comments").Where(a => a.ArticleID == id).SingleOrDefault();
return View(article);
}
I understand the basis of eager loading but my questions are:
How do you implement it when you're pulling data from multiple related tables?
What is the best practice of populating it to the View? Once I create a View Model how do I stuff the related collections?
1) With multiple related tables you can have two scenarios:
a) Multiple top level relations: you simply add multiple Include statements (I would suggest using lambda expressions instead of strings for this, to avoid typos).
db.Articles
.Include(a=>a.Comments)
.Include(a=>a.SomethingElse)
.FirstOrDefault(a=>ArticleID==id); // Side note: I would suggest this instead of your Where plus SingleOrDefault
For these scenarios I always use a helper method like this one.
b) Multiple nested related entities:
db.Articles
.Include(a=>a.Comments.Select(c=>c.Remarks)
.FirstOrDefault(a=>ArticleID==id);
2) It's a bit up to you how you pass the data to the views. One best practice I can tell you is that you shouldn't let views lazy load any dependant entities or collections. So your use of Include is correct, but I would even suggest to remove the virtual (deactivate lazy loading) to avoid missing an Include by accident.
Regarding the ViewModels you mention, you are actually not using view models, but your data models. This is OK in most cases, unless you need to format the data somehow or add extra information. Then you would need to create a View Model and map it from the data coming from EF.
Another scenario would be if you used WebAPI or an Ajax Action. In that case, I would suggest to use a DTO (equivalent to a ViewModel) to be able to better control the data returned and its serialization.
One last comment about ViewModels is that if you have heavy entities but you only need a few properties, a good choice is to use Projections, to instruct EF to only load the required properties, instead of the full object.
db.Articles
.Include(a=>a.Comments)
.Select(a=>new ArticleDto { Id = a.ArticleID, Title = a.Title })
.ToListAsync();
This will translate to a "SELECT ArticleID, Title FROM Articles", avoiding returning the article bodies and other stuff that you might not need.
You can chain the relationships with Include. For example:
var article = db.Articles.Include("Comments.Remarks").Where(a => a.ArticleID == id).SingleOrDefault();
I'm not sure what you mean by your second question, though. By issuing this query you already have all the comments and all the remarks for those comments. Therefore, you can access them off of the article instance out of the box:
foreach (var comment in article.Comments)
{
...
foreach (var remark in comment.Remarks)
{
...
}
}
How you handle that with your view model is entirely up to you. You could map the comments/remarks to view models of their own, set them directly on the view model, etc. That's all down to what the needs of your application are, and no one but you can speak to that.
I am encountered an error that I am not familier with. I tried to google with no success.
I wrote the following query where I am having this error.
The entity or complex type 'MyWebProject.Models.UserDetail' cannot be constructed in a LINQ to Entities query.
The query:
UsersContext db = new UsersContext();
var userdata = (from k in db.UserDetails
where k.UserId == WebSecurity.CurrentUserId
select new UserDetail()
{
FullName = k.FullName,
Email = k.Email,
About = k.About,
Link = k.Link,
UserSchool = new School()
{
SchoolId = k.UserSchool.SchoolId,
SchoolName = k.UserSchool.SchoolName
},
UserCourse = new Course()
{
CourseId=k.UserCourse.CourseId,
CourseName=k.UserCourse.CourseName
},
Country=k.Country
}).FirstOrDefault();
Class:
public class UserDetail
{
public int Id { get; set; }
public int UserId { get; set; }
public string FullName { get; set; }
public string Link { get; set; }
public bool? Verified { get; set; }
public string Email { get; set; }
public string About { get; set; }
public School UserSchool { get; set; }
public Course UserCourse { get; set; }
public string Country { get; set; }
}
public class School
{
public int SchoolId { get; set; }
public string SchoolName { get; set; }
public string Country { get; set; }
}
public class Course
{
public int CourseId { get; set; }
public string CourseName { get; set; }
public School School { get; set; }
}
Any idea what went wrong??
It looks like it is due to how you are creating the complex properties School and Course in the middle of the query. It would be better to select the User (remove the select transformation), then use navigation properties to access those objects instead of building them manually. The navigation are meant for this as long as you have the proper relations built with foreign keys.
UsersContext db = new UsersContext();
var userdata = (from k in db.UserDetails
where k.UserId == WebSecurity.CurrentUserId})
.FirstOrDefault();
// access navigation properties which will perform the joins on your behalf
// this also provides for lazy loading which would make it more effecient. (it wont load the school object until you need to access it)
userdata.School
userdata.Course
MSDN article about navigation properties: http://msdn.microsoft.com/en-us/library/vstudio/bb738520(v=vs.100).aspx
This should give you what you want. It will load your objects as part of the query (and not rely on lazy loading).
UsersContext db = new UsersContext();
var userdata = db.UserDetails.Include(x => x.UserSchool)
.Include(x => x.UserCourse)
.Include(x => x.Country)
.Where(x => x.UserId == WebSecurity.CurrentUserId)
.FirstOrDefault();
I think it's because your entity has the same name of the object you're trying to create. Try renaming the object you want to return back. If you want to return the same type as your entity try the eager loading with .Include("relationshipname") feature.
A great answer from #Yakimych is given below.
You cannot (and should not be able to) project onto a mapped entity. You can, however, project onto an annonymous type or onto a DTO:
public class ProductDTO
{
public string Name { get; set; }
// Other field you may need from the Product entity
}
And your method will return a List of DTO's.
public List<ProductDTO> GetProducts(int categoryID)
{
return (from p in db.Products
where p.CategoryID == categoryID
select new ProductDTO { Name = p.Name }).ToList();
}
Mapped entities in EF basically represent database tables. If you project onto a mapped entity, what you basically do is partially load an entity, which is not a valid state. EF won't have any clue how to e.g. handle an update of such an entity in the future (the default behaviour would be probably overwriting the non-loaded fields with nulls or whatever you'll have in your object). This would be a dangerous operation, since you would risk losing some of your data in the DB, therefore it is not allowed to partially load entities (or project onto mapped entities) in EF.
For more details please go to the following link:
The entity cannot be constructed in a LINQ to Entities query