Develop Reference

Entity Framework Include performance problems

Good Morning,
I have a problem with the performance if i use include.
Have a look at the following query, if I load the data from the include (x.Server_Item.itemType) the query needs like 980ms.
If i remove them, the query needs 51ms.
980ms:
using (var db = new dbContext())
{
var items = db.Characters_Inventory
.Where(x => x.charId == charId && x.itemLocation == ItemLocationInventory)
.Include(p => p.Server_Items)
.AsSplitQuery()
.Select(x => new
{
itemId = x.id,
x.itemName,
x.nameTag,
x.itemAmount,
x.itemLocation,
x.isItemEquipped,
x.itemCategory,
x.metaId,
x.metaTag0,
x.metaTag1,
itemType = x.Server_Items.itemType,
itemSubType = x.Server_Items.itemSubType,
itemClothesType = x.Server_Items.Server_Clothes.type,
itemPicName = x.Server_Items.itemPicSRC,
itemWeight = x.Server_Items.itemWeight,
itemGender = x.Server_Items.Server_Clothes.gender,
isItemUseable = x.Server_Items.isItemUseable,
isItemDroppable = x.Server_Items.isItemDroppable,
isItemGiveable = x.Server_Items.isItemGiveable,
isItemStorable = x.Server_Items.isItemStorable,
isItemStackable = x.Server_Items.isItemStackable,
isItemShowable = x.Server_Items.isItemShowable,
isItemNameable = x.Server_Items.isItemNameable,
isItemPlaceable = x.Server_Items.isItemPlaceable,
//isClothesTypeEquipped = IsClothesTypeEquipped(charId, x.id),
isClothesSwitchable = x.Server_Items.Server_Clothes.altClothesName != "" ? true : false
//isWeaponTypeEquipped = IsWeaponTypeEquipped(charId, x.id),
//charName = Characters.GetCharacterName(x.metaId)
}).AsNoTracking().ToList();
return JsonConvert.SerializeObject(items);
}
50ms:
using (var db = new dbContext())
{
var items = db.Characters_Inventory
.Where(x => x.charId == charId && x.itemLocation == ItemLocationInventory)
.Include(p => p.Server_Items)
.AsSplitQuery()
.Select(x => new
{
itemId = x.id,
x.itemName,
x.nameTag,
x.itemAmount,
x.itemLocation,
x.isItemEquipped,
x.itemCategory,
x.metaId,
x.metaTag0,
x.metaTag1
}).AsNoTracking().ToList();
return JsonConvert.SerializeObject(items);
}
I tried with AsSplitQuery and without AsSplitQuery.
Maybe nice to know: On the included table (Server_Items) is alrdy a table included.
Thanks in advance.
Sincerely

LINQ Method - Optimization

I'm reading a CSV file splitting it into cols, then grouping into a new class.
It looks clunky just wondering is there is a more simple method for instance like not selecting them into the class first:
EDIT: so to clarify I'm trying to get the TimesheetHours grouped by all the other columns.
var rowList = csvFile.Rows.Select(row => row.Split(','))
.Select(cols => new UtilisationRow {
UploadId = savedUpload.Id,
FullName = cols[0],
TimesheetWorkDateMonthYear = Convert.ToDateTime(cols[1]),
TimesheetTaskJobnumber = cols[2],
TimesheetWorktype = cols[3],
TimesheetHours = Convert.ToDouble(cols[4]),
TimesheetOverhead = cols[5]
})
.GroupBy(d => new {
d.FullName,
d.TimesheetWorkDateMonthYear,
d.TimesheetTaskJobnumber,
d.TimesheetWorktype,
d.TimesheetOverhead
})
.Select(g => new UtilisationRow {
FullName = g.First().FullName,
TimesheetWorkDateMonthYear = g.First().TimesheetWorkDateMonthYear,
TimesheetTaskJobnumber = g.First().TimesheetTaskJobnumber,
TimesheetWorktype = g.First().TimesheetWorktype,
TimesheetHours = g.Sum(s => s.TimesheetHours),
TimesheetOverhead = g.First().TimesheetOverhead
})
.ToList();
Many thanks,
Lee.

The two problems in your code are that you call First() repeatedly on a group, while you should retrieve that same data from group's key, and that you are using UtilisationRow in the first Select, which should use an anonymous type instead:
var rowList = csvFile.Rows.Select(row => row.Split(','))
.Select(cols => new {
UploadId = savedUpload.Id,
FullName = cols[0],
TimesheetWorkDateMonthYear = Convert.ToDateTime(cols[1]),
TimesheetTaskJobnumber = cols[2],
TimesheetWorktype = cols[3],
TimesheetHours = Convert.ToDouble(cols[4]),
TimesheetOverhead = cols[5]
})
.GroupBy(d => new {
d.FullName,
d.TimesheetWorkDateMonthYear,
d.TimesheetTaskJobnumber,
d.TimesheetWorktype,
d.TimesheetOverhead
})
.Select(g => new UtilisationRow {
FullName = g.Key.FullName,
TimesheetWorkDateMonthYear = g.Key.TimesheetWorkDateMonthYear,
TimesheetTaskJobnumber = g.Key.TimesheetTaskJobnumber,
TimesheetWorktype = g.Key.TimesheetWorktype,
TimesheetHours = g.Sum(s => s.TimesheetHours),
TimesheetOverhead = g.Key.TimesheetOverhead
})
.ToList();
Now the "pipeline" of your method looks pretty clean:
The first Select does the initial parsing into a temporary record
GroupBy bundles matching records into a group
The final Select produces records of the required type.

LINQ to Entities does not recognize the method String.Format

I'm trying to format a double value (by showing only 2 decimals). I tried to use AsEnumerable but I keep getting this error
LINQ to Entities does not recognize the method
String.Format
var tw = workers.Select(x => new
{
Id = x.Id,
JobOpportunityFeedbacks = x.JobOpportunityFeedbacks.AsEnumerable().
Select(y => new
{
Rating = String.Format("0.00",y.Rating),
Feedback = y.Feedback
});

You have to do the AsEnumerable outside of your initial Select
var tw = workers.Select(x => new
{
Id = x.Id,
JobOpportunityFeedbacks = x.JobOpportunityFeedbacks
.Select(y => new
{
y.Rating,
y.Feedback
})
})
.AsEnumerable()
.Select(x => new
{
x.Id,
JopOpertunityFeedbacks = x.JobOpportunityFeedbacks
.Select(y => new
{
Rating = String.Format("0.00",y.Rating),
y.Feedback
})
});

Use SqlFunctions class - I didn't try this but should work.
var tw = workers.Select(x => new
{
Id = x.Id,
JobOpportunityFeedbacks = x.JobOpportunityFeedbacks.AsEnumerable().
Select(y => new
{
Rating = SqlFunctions.StringConvert(y.Rating, 4, 2)
Feedback = y.Feedback
});
https://msdn.microsoft.com/en-us/library/dd487158(v=vs.110).aspx

Why this linq query doesn't return distinct code?

I want list of all unique Scheme_Codes but I am unable to write query. I tried this one but I am confused what's problem with this query.
var queryresult = db.MFData.GroupBy(x => new { Scheme_Name = x.Scheme_Name, Scheme_Code = x.Scheme_Code, FundFamily = x.FundFamily, Date = x.Date })
.Select(group => new
{
Scheme_name = group.Key.Scheme_Name,
Scheme_Code = group.Key.Scheme_Code,
FundFamily = group.Key.FundFamily,
Date = group.Max(x => x.Date),
count = group.Select( x => x.Scheme_Code).Distinct().Count()
}
).OrderBy(x => x.Scheme_Code);
I have this query but I am not sure how to convert this to linq
SELECT [Scheme_Code],[FundFamily],[Scheme_Name],
MAX([Date]) as LastDate
FROM [MFD].[dbo].[MFDatas]
GROUP BY [Scheme_Code],[Scheme_Name], [FundFamily]
ORDER BY [Scheme_Code]

All you have to do is omit the date from your groupby-clause:
var queryresult = db.MFData.GroupBy(x => new
{
Scheme_Name = x.Scheme_Name,
Scheme_Code = x.Scheme_Code,
FundFamily = x.FundFamily
}).Select(group => new
{
Scheme_name = group.Key.Scheme_Name,
Scheme_Code = group.Key.Scheme_Code,
FundFamily = group.Key.FundFamily,
Date = group.Max(x => x.Date),
count = group.Select(x => x.Scheme_Code).Distinct().Count()
}).OrderBy(x => x.Scheme_Code);

linq compiled query error "parameteres cannot be sequences"

I've got the code below:
var catRoots = CatalogContext.CatalogRoots.Where(cr => cr.Visible);
var catChapter = CatalogContext.CatalogChapters.Where(cch => cch.Visible);
var catThemes = CatalogContext.CatalogThemes.Where(cth => cth.Visible);
var catCompanies = CatalogContext.CatalogCompanies.Where(cc => cc.Visible);
var catRelations = CatalogContext.CatalogCompanyThemeRelations.Where(cctr => cctr.Visible && cctr.OwnerVisible);
var regions = CatalogContext.Regions.AsQueryable();
var compChapters = catRelations.Where(cctr => cctr.Location == CatalogCompanyLocations.Chapter)
.Join(catChapter, cctr => cctr.ParentID, cch => cch.ID, (cctr, cch) => new { Relation = cctr, Chapter = cch })
.Join(catRoots, cch => cch.Chapter.CatalogRootID, cr => cr.ID, (cch, cr) => new { CatalogRoot = cr, CatalogChapter = cch.Chapter, CatalogRelation = cch.Relation })
.Join(catCompanies, cr => cr.CatalogRelation.CompanyID, cc => cc.ID, (cr, cc) => new { Root = cr.CatalogRoot, Chapter = cr.CatalogChapter, Theme = default(CatalogTheme), Company = cc })
.Join(regions, cc => cc.Company.RegionID, r => r.ID, (cc, r) => new { Root = cc.Root, Chapter = cc.Chapter, Theme = cc.Theme, Company = cc })
.GroupBy(gr => new { Chapter = gr.Chapter, Name = gr.Root.Name, ID = gr.Root.ID, Icon = gr.Root.Icon, Rewrite = gr.Root.Rewrite, Sort = gr.Root.Sort })
.Select(gr => new { Chapter = gr.Key.Chapter, ID = gr.Key.ID, Name = gr.Key.Name, Icon = gr.Key.Icon, Rewrite = gr.Key.Rewrite, Sort = gr.Key.Sort, Count = gr.Count() });
var compThemes = catRelations.Where(cctr => cctr.Location == CatalogCompanyLocations.Theme)
.Join(catThemes, cctr => cctr.ParentID, cth => cth.ID, (cctr, cth) => new { Relation = cctr, Theme = cth })
.Join(catChapter, cth => cth.Theme.CatalogChapterID, cch => cch.ID, (cth, cch) => new { Relation = cth.Relation, Theme = cth.Theme, Chapter = cch })
.Join(catRoots, cch => cch.Chapter.CatalogRootID, cr => cr.ID, (cch, cr) => new { Relation = cch.Relation, Theme = cch.Theme, Chapter = cch.Chapter, Root = cr })
.Join(catCompanies, cr => cr.Relation.CompanyID, cc => cc.ID, (cr, cc) => new { Root = cr.Root, Chapter = cr.Chapter, Theme = cr.Theme, Company = cc })
.Join(regions, cc => cc.Company.RegionID, r => r.ID, (cc, r) => new { Root = cc.Root, Chapter = cc.Chapter, Theme = cc.Theme, Company = cc.Company })
.GroupBy(gr => new { Chapter = gr.Chapter, Name = gr.Root.Name, ID = gr.Root.ID, Icon = gr.Root.Icon, Rewrite = gr.Root.Rewrite, Sort = gr.Root.Sort })
.Select(gr => new { Chapter = gr.Key.Chapter, ID = gr.Key.ID, Name = gr.Key.Name, Icon = gr.Key.Icon, Rewrite = gr.Key.Rewrite, Sort = gr.Key.Sort, Count = gr.Count() });
var source = compChapters.Union(compThemes);
var chapters = source.Select(r => new { Chapter = r.Chapter, Count = r.Count }).Cast<object>().Distinct();
public static Func<DataContext, IQueryable<object>, IEnumerable<object>> filteredFunc =
CompiledQuery.Compile<DataContext, IQueryable<object>, IEnumerable<object>>
(
(DataContext db, IQueryable<object> q) => q.Distinct().ToList()
);
filtredChapters = filteredFunc(CatalogContext, chapters);
I'm getting an error "parameteres cannot be sequences" when I run filteredFunc, which is weird, because "chapters" object is IQueryable, not IEnumerable, so why am I getting the error?
The code below works fine, but it is not good for me.
filtredChapters = chapters.Distinct().Cast<object>().ToList();

You cannot use compiled queries with an IEnumerable like this. The number of items in the enumeration can vary and so the query plan for the query will vary based on its size. Just remove the compiled query and use the function as is.