I've a query like this
public IEnumerable<ContractInquiry> ContractInquiry(Guid itemSoldID)
{
IEnumerable<ContractInquiry> result;
using (var context = new ContractDbContext(_ctxOptions))
{
var qry = from con in context.Contracts
join product in context.ContractProducts on con.ID equals product.ContractID
join service in context.ServiceDetails on con.ID equals service.ContractID into tmpService
from service in tmpService.DefaultIfEmpty()
where product.ItemSoldID == itemSoldID
&& product.ProductStatus != ProductStatus.Deleted.ToString()
&& con.Status != Status.Deleted.ToString()
select new ContractInquiry
{
ServiceID = con.ID,
ServiceType = con.ServiceType,
ServiceDate = service.ServiceDate,
ServiceNumber = service.ServiceNumber,
ServiceManager = con.Contacts.Where(q => q.Role.Contains(ContractRole.ServiceManager.ToString()))
.OrderBy(o => o.ID).FirstOrDefault()
};
result = qry.ToList();
}
return result;
}
This query was working fine. But when we upgraded to .NET Core 3.1.5 and Entity Framework Core 3.1.5, it started throwing a client-side evaluation error:
"could not be translated. Either rewrite the query in a form that can be translated, or switch to client evaluation explicitly by inserting a call to either AsEnumerable(), AsAsyncEnumerable(), ToList(), or ToListAsync()."
So I had to take the following line out from the query:
ServiceManager = con.Contacts.Where(q => q.Role.Contains(ContractRole.ServiceManager.ToString()))
.OrderBy(o => o.ID).FirstOrDefault()
So re-wrote the query like this:
public IEnumerable<ContractInquiry> ContractInquiry(Guid itemSoldID)
{
List<ContractInquiry> result;
using (var context = new ContractDbContext(_ctxOptions))
{
var result = (from con in context.Contracts
join product in context.ContractProducts on con.ID equals product.ContractID
join service in context.ServiceDetails on con.ID equals service.ContractID into tmpService
from service in tmpService.DefaultIfEmpty()
where product.ItemSoldID == itemSoldID
&& product.ProductStatus != ProductStatus.Deleted.ToString()
&& con.Status != Status.Deleted.ToString()
select new ContractInquiry
{
ServiceID = con.ID,
ServiceType = con.ServiceType,
ServiceDate = service.ServiceDate,
ServiceNumber = service.ServiceNumber,
Contacts = con.Contacts
}).ToList();
}
result.ForEach(con => con.Contacts.Where(q => q.Role.Contains(ContractRole.ServiceManager.ToString()))
.OrderBy(o => o.ID).FirstOrDefault();
return result;
}
Here
con.Contacts
is a table collection in Contract.cs class
I've added a property like this in ContractInquiry.cs class:
[JsonIgnore]
public IEnumerable<Contact> Contacts { set; get; }
This is working fine as well.
Question:
Doing like this will work fine but at run time, the table collection "con.Contacts" will be in memory right? And that will impact the performance of the query right if the table is a huge collection? So is there a work around for this instead of using a memory table? How can I take out the "ServiceManager = .." from the select clause in my first query?
UPDATE: Can someone answer my question?
To answer your question:
No the whole Contacts table won't be loaded into memory.
It will be slower than using a database query but unless you have a crazy amount of records you won't be able to 'humanly' measure it (obv. a stress test will point out that this may be slower by 150ms on 10000 records).
Why this is:
EF Core only loads related data and when it is needed. For example you have 1000 of these ContractInquiry records when calling .ToList(). Every one of these records contain ten contacts. Then EF Core will load only 1000*10 contacts. Due to references if any of these overlap they will share memory location and only a reference to it will be saved.
Some changes you can do to make this even faster:
Change .ToList() to .AsEnumerable(). You can do this because you only iterate over that list once, so you save a whole iteration using .AsEnumerable(). (to create a list the program must iterate over it and then you iterate over it again). Also you are returning an IEnumerable so creating a list is pointless (if you are iterating over it once, which is the case here) unless you later cast it back, which I do not recommend.
Add .AsNoTracking() in the query. I don't know how you can achieve the same thing with this type of querying (I only use Linq). This will save a lot of time because EF Core will not have to create tracking (and will also save memory).
If you would change the query to a Linq query I would be happy to have a look at it and help you optimise it.
I'm playing with Entity Framework for the first time. I'm writing the following simply query:
public List<Request> GetResult(string code)
{
List<Request> requests = new List<Request>();
using (var db = new Context())
{
requests = (from c in db.Requests where c.Code == code select c).ToList();
}
return requests;
}
There are a few related entities to the Request object such as a Results table that has a 1-to-1 relation with the Request table. However they are all coming back null.
How can I query with Entity Framework and return a entity and ALL its related entities?
TIA
Single query using eager loading
db.Requests.Where(req => req.Code == code)
.Include(req => req.Results) // Joining is performed here
.Include(req => req.SomeOtherProperty)
.ToList()
Multiple queries using explicit loading
// Hits the database once.
var requests = db.Requests.Where(req => req.Code == code).ToList();
var requestIDs = requests.Select(req => req.ID);
// Hits the database another time to load your Results property.
db.Results.Where(res => requestIDs.Contains(res.RequestID)).Load();
If lazy loading is enabled, everytime you access the Results property of Request list, a query is executed to on the database to load it for you, which could result in the N+1 problem. Lazy loading is not yet available on EntityFramework Core though.
If you want to choose objects to be loaded use
db.Entry(Requests).Reference(p => p.Code).Load();
If you want to load all automatically on your datacontext constructor set
this.Configuration.LazyLoadingEnabled = true;
Can the select new MobileTeamModel be refactored into a reusable method and still be read by Entity Framework? I have quite a bit of requests that need this same data and would like to reuse it but I know Entity Framework complains about this type of stuff.
var teams = new MobileListResponse<MobileTeamModel>
{
List = (from e in _divisionsRepository.DataContext.DivisionTeams.Where(#where.Expand())
orderby e.Team.Name
select new MobileTeamModel
{
Id = e.Id,
Name = e.Team.Name,
Status = e.Status,
Paid = e.Paid,
Division = e.Division.Name,
City = e.Team.TeamAddress.Address.City,
StateRegion =
e.Team.TeamAddress.Address.StateRegionId.HasValue
? e.Team.TeamAddress.Address.StateRegion.Name
: null
}).ToList()
};
EDIT
The idea is to implement the select new MobileTeamModel { ... } in a reusable way, while having EF only query the required columns.
Probably a nice extension method:
public static class MobileTeamModelExtensions
{
public static IEnumerable<MobileTeamModel> ToMobileTeamModels
(this IQueryable<DivisionTeam> instance)
{
var result = instance.Select(e =>
select new MobileTeamModel
{
Id = e.Id,
Name = e.Team.Name,
Status = e.Status,
Paid = e.Paid,
Division = e.Division.Name,
City = e.Team.TeamAddress.Address.City,
StateRegion =
e.Team.TeamAddress.Address.StateRegionId.HasValue
? e.Team.TeamAddress.Address.StateRegion.Name
: null
}).ToList()
return result;
}
}
So you could:
var query = _divisionsRepository.DataContext.DivisionTeams
.Where(#where.Expand());
var list = query.ToMobileTeamModels();
var query = query.Where(<more where>);
var list2 = query.ToMobileTeamModels();
You simply have to add a constructor or a factory (static method that return a MobileTeamModel) and receives a Team.
Then you can make a simpler query like this:
select new MobileTeamModel(e.Team) // parametreized constructor
or this
select MobileTeamModel.FromTeam(e.Team) // factory
Finallyyou can use something like AutoMapper or ValueInjecter to project the data returned by the query to another class by using conventions or mapping.
NOTE: I can't see how your clasees look like, but this is the basic idea.
EDIT: getting EF to query only the necessary columns
As for your comment, you want to reuse the construction part, and have EF only query the required columns, and not he whole entities.
Another good explanation here: Los Techies. Efficient querying with LINQ, AutoMapper and Future queries
To do so you must use Automapper, with Queryable Extensions.
WHen you use this extensions, you can use a mapping from a query result to the destination class, and have EF only query the mapped columns.
I have a MembershipGroups table that is associated with a child Members table. The Members table has a Status column which can be set to Active or Inactive.
I want to select all MembershipGroups and only their active Members
As an example,
MembershipGroups
ID----Title
1-----Group #1
2-----Group #2
Members
MembershipGroupID-Name--Status
1-------------------------John----Active
1-------------------------Sally----Inactive
1-------------------------David---Inactive
I'm trying to create a query that looks something like the following (which doesn't currently work):
var query = from mg in db.MembershipGroups
where mg.Members.Status = "Active"
select mg
The result for this example should return a MembershipGroup of ID#1 with only one child Member entity
How can use LINQ-to-SQL to select a parent object that filters on child objects? If I were using straight T-SQL then this would be a simple join with a where clause but it seems to be much more difficult to do using LINQ-to-SQL.
Edit - Updated answer to return the MemberShipGroup object
var query = (from mg in db.MembershipGroups
join m in db.Members.Where(mem => mem.Status == "Active")
on mg.ID equals m.MembershipGroups into members
select new
{
MembershipGroup = mg,
Members = members
}).AsEnumerable()
.Select(m => new MembershipGroup
{
ID = m.MembershipGroup.ID,
Title = m.MembershipGroup.Title,
Members = m.Members
});
In LINQ to SQL, you can use the AssociateWith method on the DataLoadOptions to set your child filter at the context level.
DataLoadOptions opt = new DataLoadOptions();
opt.AssociateWith<Member>(m => m.Status == "Active");
db.LoadOptions = opt;
With this in place, you can simply return your member groups (or filter them for the active ones using where mg.Any(group => group.Members.Status == "Active"). Then when you try to drill into the Members of that group, only the Active ones will be returned due to the LoadOptions.
See also http://msdn.microsoft.com/en-us/library/system.data.linq.dataloadoptions.associatewith.aspx .
One word of warning, once you set the LoadOptions on a context instance, you can not change it. You may want to use a customized context to use this option.
As an alternative, you could use LINQ to SQL's inheritance model to create an ActiveMember type using the Status column as your discriminator and then create an association between the MemberGroups and ActiveMembers types. This would be the approach you would need to use to model this with the Entity Framework if you though about going that route as well as EF doesn't support the concept of the LoadOptions.
Make sure you are including the child objects you are trying to filter on, inside the query.
E.g.
var query = db.MembershipGroups
.Include("Members")
.Where(m => m.Members.Status == "Active");
This is easy for me to perform in TSQL, but I'm just sitting here banging my head against the desk trying to get it to work in EF4!
I have a table, lets call it TestData. It has fields, say: DataTypeID, Name, DataValue.
DataTypeID, Name, DataValue
1,"Data 1","Value1"
1,"Data 1","Value2"
2,"Data 1","Value3"
3,"Data 1","Value4"
I want to group on DataID/Name, and concatenate DataValue into a CSV string. The desired result should contain -
DataTypeID, Name, DataValues
1,"Data 1","Value1,Value2"
2,"Data 1","Value3"
3,"Data 1","Value4"
Now, here's how I'm trying to do it -
var query = (from t in context.TestData
group h by new { DataTypeID = h.DataTypeID, Name = h.Name } into g
select new
{
DataTypeID = g.Key.DataTypeID,
Name = g.Key.Name,
DataValues = (string)g.Aggregate("", (a, b) => (a != "" ? "," : "") + b.DataValue),
}).ToList()
The problem is that LINQ to Entities does not know how to convert this into SQL. This is part of a union of 3 LINQ queries, and I'd really like it to keep it that way. I imagine that I could retrieve the data and then perform the aggregate later. For performance reasons, that wouldn't work for my app. I also considered using a SQL server function. But that just doesn't seem "right" in the EF4 world.
Anyone care to take a crack at this?
If the ToList() is part of your original query and not just added for this example, then use LINQ to Objects on the resulting list to do the aggregation:
var query = (from t in context.TestData
group t by new { DataTypeID = t.DataTypeID, Name = t.Name } into g
select new { DataTypeID = g.Key.DataTypeID, Name = g.Key.Name, Data = g.AsEnumerable()})
.ToList()
.Select (q => new { DataTypeID = q.DataTypeID, Name = q.Name, DataValues = q.Data.Aggregate ("", (acc, t) => (acc == "" ? "" : acc + ",") + t.DataValue) });
Tested in LINQPad and it produces this result:
Some of the Answers suggest calling ToList() and then perform the calculation as LINQ to OBJECT. That's fine for a little amount of data, but what if I have a huge amount of data that I do not want to load into memory too early, then, ToList() may not be an option.
So, the better idea would be to process/format the data in the presentation layer and let the Data Access layer do only loading or saving raw data that SQL likes.
Moreover, in your presentation layer, most probably you are filtering the data by paging, or maybe you are showing one row in the details page, so, the data you will load into the memory is likely smaller than the data you load from the database. (Your situation/architecture may be different,.. but I am saying, most likely).
I had a similar requirement. My problem was to get the list of items from the Entity Framework object and create a formatted string (comma separated value)
I created a property in my View Model which will hold the raw data from the repository and when populating that property, the LINQ query won't be a problem because you are simply querying what SQL understands.
Then, I created a get-only property in my ViewModel which reads that Raw entity property and formats the data before displaying.
public class MyViewModel
{
public IEnumerable<Entity> RawChildItems { get; set; }
public string FormattedData
{
get
{
if (this.RawChildItems == null)
return string.Empty;
string[] theItems = this.RawChildItems.ToArray();
return theItems.Length > 0
? string.Format("{0} ( {1} )", this.AnotherRegularProperty, String.Join(", ", theItems.Select(z => z.Substring(0, 1))))
: string.Empty;
}
}
}
Ok, in that way, I loaded the Data from LINQ to Entity to this View Model easily without calling.ToList().
Example:
IQueryable<MyEntity> myEntities = _myRepository.GetData();
IQueryable<MyViewModel> viewModels = myEntities.Select(x => new MyViewModel() { RawChildItems = x.MyChildren })
Now, I can call the FormattedData property of MyViewModel anytime when I need and the Getter will be executed only when the property is called, which is another benefit of this pattern (lazy processing).
An architecture recommendation: I strongly recommend to keep the data access layer away from all formatting or view logic or anything that SQL does not understand.
Your Entity Framework classes should be simple POCO that can directly map to a database column without any special mapper. And your Data Access layer (say a Repository that fetches data from your DbContext using LINQ to SQL) should get only the data that is directly stored in your database. No extra logic.
Then, you should have a dedicated set of classes for your Presentation Layer (say ViewModels) which will contain all logic for formatting data that your user likes to see. In that way, you won't have to struggle with the limitation of Entity Framework LINQ. I will never pass my Entity Framework model directly to the View. Nor, I will let my Data Access layer creates the ViewModel for me. Creating ViewModel can be delegated to your domain service layer or application layer, which is an upper layer than your Data Access Layer.
Thanks to moi_meme for the answer. What I was hoping to do is NOT POSSIBLE with LINQ to Entities. As others have suggested, you have to use LINQ to Objects to get access to string manipulation methods.
See the link posted by moi_meme for more info.
Update 8/27/2018 - Updated Link (again) - https://web.archive.org/web/20141106094131/http://www.mythos-rini.com/blog/archives/4510
And since I'm taking flack for a link-only answer from 8 years ago, I'll clarify just in case the archived copy disappears some day. The basic gist of it is that you cannot access string.join in EF queries. You must create the LINQ query, then call ToList() in order to execute the query against the db. Then you have the data in memory (aka LINQ to Objects), so you can access string.join.
The suggested code from the referenced link above is as follows -
var result1 = (from a in users
b in roles
where (a.RoleCollection.Any(x => x.RoleId = b.RoleId))
select new
{
UserName = a.UserName,
RoleNames = b.RoleName)
});
var result2 = (from a in result1.ToList()
group a by a.UserName into userGroup
select new
{
UserName = userGroup.FirstOrDefault().UserName,
RoleNames = String.Join(", ", (userGroup.Select(x => x.RoleNames)).ToArray())
});
The author further suggests replacing string.join with aggregate for better performance, like so -
RoleNames = (userGroup.Select(x => x.RoleNames)).Aggregate((a,b) => (a + ", " + b))
You are so very close already. Try this:
var query = (from t in context.TestData
group h by new { DataTypeID = h.DataTypeID, Name = h.Name } into g
select new
{
DataTypeID = g.Key.DataTypeID,
Name = g.Key.Name,
DataValues = String.Join(",", g),
}).ToList()
Alternatively, you could do this, if EF doesn't allow the String.Join (which Linq-to-SQL does):
var qs = (from t in context.TestData
group h by new { DataTypeID = h.DataTypeID, Name = h.Name } into g
select new
{
DataTypeID = g.Key.DataTypeID,
Name = g.Key.Name,
DataValues = g
}).ToArray();
var query = (from q in qs
select new
{
q.DataTypeID,
q.Name,
DataValues = String.Join(",", q.DataValues),
}).ToList();
Maybe it's a good idea to create a view for this on the database (which concatenates the fields for you) and then make EF use this view instead of the original table?
I'm quite sure it's not possible in a LINQ statement or in the Mapping Details.