Develop Reference

Improving speed of method after adding value from different table aspn.net

I've got two tables: Index and Codes
when one condition is true, I need to check whether the Index is still valid and for that I need to get EndDate of the code which is in Codes table (as I've got only id of code in Index table)
This is how I do that:
1) First I get all Codes that have ended already (approx 3k+ items)
var goods = _context.Codes.Select(a =>
new Codes
{
Loid = a.Loid,
Pid = a.Pid,
Code = a.Code,
Startdate = a.Startdate,
Enddate = a.Enddate
})
.Where(x => x.Enddate < DateTime.Now)
.ToList();
then I'm taking my Index and adding values there as well as CodeEnd from Codes list above:
query = _context.VpAbcIndex
.AsNoTracking()
.Select(e => new VpAbcIndex
{
Id = e.Id,
ParentName = e.ParentName ?? "!",
ParentEndDate = e.ParentEndDate,
ParentStartDate = e.ParentStartDate,
ParentCode = e.ParentCode,
ParentNote = e.ParentNote ?? "",
ParentStatus = e.ParentStatus,
ChildName = e.ChildName ?? "!",
ChildId = e.ChildId,
ChildEndDate = e.ChildEndDate,
ChildStartDate = e.ChildStartDate,
CodeEnd = (filter.Level == 0) ? goods
.FirstOrDefault( x => x.Code == e.ParentCode).Enddate :
(filter.Level == 1) ? goods
.FirstOrDefault(x => x.Code == e.ChildCode).Enddate :
(filter.Level == 2) ?
goods
.FirstOrDefault(x => x.Code == e.GrandChildCode).Enddate : null
})
;
}
That's approx 10k items. It doesn't seem that much however it takes quite a long time for them to appear in my browser. Am I doing something wrong? Is there a faster way to join these values?

LINQ Where instead of Select

I need to do this:
if(...)
{
var query2 = query.Select(i => new { Balance = i.Balance * i.RateValue, Date = i.Date });
}
Of course I may do it, but then I'll have a problem:
var list = query2.ToList();
How I can implement this? Maybe something like this:
if(accountName == "Safe" && currency == "Br")
{
query = query.Where(i => new { Balance = i.Balance * i.RateValue, Date = i.Date });
}
But this doesn't work.

Your question is lacking context. What is the type of i and which properties does it have more? Do you want to filter on account name and currency? Also Where and Select are completely different functions.
Or did you mean something like this?
var query = query.Where(i => i.AccountName == "Safe" && i.Currency == "Br")
.Select(i => new { Balance = i.Balance * i.RateValue, Date = i.Date });

LINQ to Entities does not recognize the method

This is my Code where I am fetching data.
var list = (from u in _dbContext.Users
where u.IsActive
&& u.IsVisible
&& u.IsPuller.HasValue
&& u.IsPuller.Value
select new PartsPullerUsers
{
AvatarCroppedAbsolutePath = u.AvatarCroppedAbsolutePath,
Bio = u.Bio,
CreateDateTime = u.CreationDate,
Id = u.Id,
ModifieDateTime = u.LastModificationDate,
ReviewCount = u.ReviewsReceived.Count(review => review.IsActive && review.IsVisible),
UserName = u.UserName,
Locations = (from ul in _dbContext.UserLocationRelationships
join l in _dbContext.Locations on ul.LocationId equals l.Id
where ul.IsActive && ul.UserId == u.Id
select new PartsPullerLocation
{
LocationId = ul.LocationId,
Name = ul.Location.Name
}),
Rating = u.GetPullerRating()
});
Now Here is my Extension.
public static int GetPullerRating(this User source)
{
var reviewCount = source.ReviewsReceived.Count(r => r.IsActive && r.IsVisible);
if (reviewCount == 0)
return 0;
var totalSum = source.ReviewsReceived.Where(r => r.IsActive && r.IsVisible).Sum(r => r.Rating);
var averageRating = totalSum / reviewCount;
return averageRating;
}
I have check this Post LINQ to Entities does not recognize the method
And I come to know I need to use
public System.Linq.Expressions.Expression<Func<Row52.Data.Entities.User, int>> GetPullerRatingtest
But how ?
Thanks

You can use conditionals inside LINQ to Entity queries:
AverageRating = u.ReviewsReceived.Count(r => r.IsActive && r.IsVisible) > 0 ?
u.ReviewsReceived.Where(r => r.IsActive && r.IsVisible).Sum(r => r.Rating) /
u.ReviewsReceived.Count(r => r.IsActive && r.IsVisible)
: 0
This will be calculated by the server, and returned as part of your list. Although with 10 million rows like you said, I would do some serious filtering before executing this.

Code within LINQ (to Entities) query is executed within database, so you can't put random C# code there. So you should either use user.GetPullerRating() after it is retrieved or create a property if you don't want to do the calculation every time.
You can also do:
foreach (var u in list)
u.Rating = u.GetPullerRating()
By the way, why is it extension method.

Multiple counts on single table in LINQ query

I am having trouble doing multiple counts on a single table in a LINQ query. I am using NHibernate, LINQ to NHibernate and C#.
query is a populated list. I have a table that has a boolean called FullRef. I want to do a LINQ query to give a count of occurances of FullRef = false and FullRef = true on each TrackId. TrackId gets a new row for each time he gets a track.Source == "UserRef".
In the following query I get the correct number count (from the FullRefTrueCount) of FullRef = true, but it gives an unknown wrong number on the FullRefFalseCount.
var query2 = from track in query
where track.Source == "UserRef"
group track by new { TrackId = track.TrackId, FullRef = track.FullRef } into d
select new FullReferrer
{
Customer = d.Key.TrackId,
FullRefFalseCount = d.Where(x => x.FullRef == false).Count(),
FullRefTrueCount = d.Where(x => x.FullRef == true).Count()
};
I also tried to modify it to not contain the FullRef in the group by. This was done by removing FullRef = track.FullRef on the by like this
var query2 = from track in query
where track.Source == "UserRef"
group track by new { TrackId = track.TrackId } into d
select new FullReferrer
{
Customer = d.Key.TrackId,
FullRefFalseCount = d.Where(x => x.FullRef == false).Count(),
FullRefTrueCount = d.Where(x => x.FullRef == true).Count()
};
Now it gives me the total count of TrackId, ignoring my .Where(x => x.FullRef == true/false) statement.
Anyone have any idea on how to fix it?
I guess it is the "group by" that is the problem. Can I somehow avoid doing a group by?
Do I maybe need a join?

For NHibernate I don't know, but using Linq With Entity Framework this should get what you want:
var query2 = (
from track in someDbSet
select new FullReferrer
{
Customer = track.trackId
, FullRefFalseCount = (from fullRefFalse in someDbSet.tracks
where fullRefFalse.IsSale == false
&& fullRefFalse.trackId == track.trackId
select fullRefFalse).Count()
, FullRefTrueCount = (from fullRefTrue in someDbSet.tracks
where fullRefTrue.IsSale == true
&& fullRefTrue.trackId == track.trackId
select fullRefTrue).Count()
}
).Distinct();

FullRefFalseCount = d.Where(x => x.FullRef == false).ToList().Count
FullRefTrueCount = d.Where(x => x.FullRef == true).ToList().Count
try this

Try out Count(condition).
FullRefFalseCount = d.Count(x => x.FullRef == false),
FullRefTrueCount = d.Count(x => x.FullRef == true)

try this. This takes the expected data from data table.
string source = "UserRef";
var result = from row in dt.AsEnumerable()
where row["source"].Equals(source)
group row by row["TrackId"]
into g
select new
{
TrackId = g.Key,
FullRefTrueCount = ((from track in g where track["FullRef"].Equals("true") select track).Count()),
FullRefFalseCount = ((from track in g where track["FullRef"].Equals("false") select track).Count())
};

To anyone else having a similar problem I solved it by making it ".AsEnumerable()"
var query2 = from track in query.AsEnumerable() // <--- the solution
where track.Source == "UserRef"
group track by new { TrackId = track.TrackId } into d
select new FullReferrer
{
Customer = d.Key.TrackId,
FullRefFalseCount = d.Count(x => !x.FullRef),
FullRefTrueCount = d.Count(x => !x.FullRef)
};

Improving performance of linq query

I'm optimizing a method with a number of Linq queries. So far the execution time is around 3 seconds and I'm trying to reduce it. There is quite a lot of operations and calculations happening in the method, but nothing too complex.
I will appreciate any suggections and ideas how the performance can be improved and code optimized.
The whole code of the method(Below I'll point where I have the biggest delay):
public ActionResult DataRead([DataSourceRequest] DataSourceRequest request)
{
CTX.Configuration.AutoDetectChangesEnabled = false;
var repoKomfortaktion = new KomfortaktionRepository();
var komfortaktionen = CTX.Komfortaktionen.ToList();
var result = new List<AqGeplantViewModel>();
var gruppen = new HashSet<Guid?>(komfortaktionen.Select(c => c.KomfortaktionsGruppeId).ToList());
var hochgeladeneKomplettabzuege = CTX.Komplettabzug.Where(c => gruppen.Contains(c.KomfortaktionsGruppeId)).GroupBy(c => new { c.BetriebId, c.KomfortaktionsGruppeId }).Select(x => new { data = x.Key }).ToList();
var teilnehmendeBetriebe = repoKomfortaktion.GetTeilnehmendeBetriebe(CTX, gruppen);
var hochgeladeneSperrlistenPlz = CTX.SperrlistePlz.Where(c => gruppen.Contains(c.KomfortaktionsGruppeId) && c.AktionsKuerzel != null)
.GroupBy(c => new { c.AktionsKuerzel, c.BetriebId, c.KomfortaktionsGruppeId }).Select(x => new { data = x.Key }).ToList();
var hochgeladeneSperrlistenKdnr = CTX.SperrlisteKdnr.Where(c => gruppen.Contains(c.KomfortaktionsGruppeId) && c.AktionsKuerzel != null)
.GroupBy(c => new { c.AktionsKuerzel, c.BetriebId, c.KomfortaktionsGruppeId }).Select(x => new { data = x.Key }).ToList();
var konfigsProAktion = CTX.Order.GroupBy(c => new { c.Vfnr, c.AktionsId }).Select(c => new { count = c.Count(), c.Key.AktionsId, data = c.Key }).ToList();
foreach (var komfortaktion in komfortaktionen)
{
var item = new AqGeplantViewModel();
var zentraleTeilnehmer = teilnehmendeBetriebe.Where(c => c.TeilnahmeStatus.Any(x => x.KomfortaktionId == komfortaktion.Id && x.AktionsTypeId == 1)).ToList();
var lokaleTeilnehmer = teilnehmendeBetriebe.Where(c => c.TeilnahmeStatus.Any(x => x.KomfortaktionId == komfortaktion.Id && x.AktionsTypeId == 2)).ToList();
var hochgeladeneSperrlistenGesamt =
hochgeladeneSperrlistenPlz.Count(c => c.data.AktionsKuerzel == komfortaktion.Kuerzel && c.data.KomfortaktionsGruppeId == komfortaktion.KomfortaktionsGruppeId) +
hochgeladeneSperrlistenKdnr.Count(c => c.data.AktionsKuerzel == komfortaktion.Kuerzel && c.data.KomfortaktionsGruppeId == komfortaktion.KomfortaktionsGruppeId);
item.KomfortaktionId = komfortaktion.KomfortaktionId;
item.KomfortaktionName = komfortaktion.Aktionsname;
item.Start = komfortaktion.KomfortaktionsGruppe.StartAdressQualifizierung.HasValue ? komfortaktion.KomfortaktionsGruppe.StartAdressQualifizierung.Value.ToString("dd.MM.yyyy") : string.Empty;
item.LokalAngemeldet = lokaleTeilnehmer.Count();
item.ZentralAngemeldet = zentraleTeilnehmer.Count();
var anzHochgelandenerKomplettabzuege = hochgeladeneKomplettabzuege.Count(c => zentraleTeilnehmer.Count(x => x.BetriebId == c.data.BetriebId) == 1) +
hochgeladeneKomplettabzuege.Count(c => lokaleTeilnehmer.Count(x => x.BetriebId == c.data.BetriebId) == 1);
item.KomplettabzugOffen = (zentraleTeilnehmer.Count() + lokaleTeilnehmer.Count()) - anzHochgelandenerKomplettabzuege;
item.SperrlisteOffen = (zentraleTeilnehmer.Count() + lokaleTeilnehmer.Count()) - hochgeladeneSperrlistenGesamt;
item.KonfigurationOffen = zentraleTeilnehmer.Count() - konfigsProAktion.Count(c => c.AktionsId == komfortaktion.KomfortaktionId && zentraleTeilnehmer.Any(x => x.Betrieb.Vfnr == c.data.Vfnr));
item.KomfortaktionsGruppeId = komfortaktion.KomfortaktionsGruppeId;
result.Add(item);
}
return Json(result.ToDataSourceResult(request));
}
The first half (before foreach) takes half a second which is okay. The biggest delay is inside foreach statement in the first iteration and in particular in these lines, execution of zentraleTeilnehmer takes 1.5 second for the first time.
var zentraleTeilnehmer = teilnehmendeBetriebe.Where(c => c.TeilnahmeStatus.Any(x => x.KomfortaktionId == komfortaktion.Id && x.AktionsTypeId == 1)).ToList();
var lokaleTeilnehmer = teilnehmendeBetriebe.Where(c => c.TeilnahmeStatus.Any(x => x.KomfortaktionId == komfortaktion.Id && x.AktionsTypeId == 2)).ToList();
TeilnehmendeBetriebe has over 800 lines, where TeilnahmeStatus property has normally around 4 items. So, maximum 800*4 iterations, which is not a huge number afterall...
Thus, I'm mostly interected in optimizing these lines, hoping to reduce execution time to half a second or so.
What I tried:
Rewrite Linq to foreach: didn't help, same time... probably not surprising, but was worth a try.
foreach (var tb in teilnehmendeBetriebe) //836 items
{
foreach (var ts in tb.TeilnahmeStatus) //3377 items
{
if (ts.KomfortaktionId == komfortaktion.Id && ts.AktionsTypeId == 1)
{
testResult.Add(tb);
break;
}
}
}
Selecting particular columns for teilnehmendeBetriebe with .Select(). Didn't help either.
Neither helped other small manipulations I tried.
What is interesting - while the first iteration of foreach can take up to 2 seconds, the second and further take just milisecons, so .net is capable of optimizing or reusing calculation data.
Any advice on what can be changed in order to improve performance is very welcome!
Edit:
TeilnahmeBetriebKomfortaktion.TeilnahmeStatus is loaded eagerly in the method GetTeilnehmendeBetriebe:
public List<TeilnahmeBetriebKomfortaktion> GetTeilnehmendeBetriebe(Connection ctx, HashSet<Guid?> gruppen)
{
return ctx.TeilnahmeBetriebKomfortaktion.Include(
c => c.TeilnahmeStatus).ToList();
}
Edit2:
The query which is sent when executing GetTeilnehmendeBetriebe:
SELECT
[Extent1].[Id] AS [Id],
[Extent1].[BetriebId] AS [BetriebId],
[Extent1].[MandantenId] AS [MandantenId],
[Extent1].[CreatedUser] AS [CreatedUser],
[Extent1].[UpdatedUser] AS [UpdatedUser],
[Extent1].[CreatedDate] AS [CreatedDate],
[Extent1].[UpdatedDate] AS [UpdatedDate],
[Extent1].[IsDeleted] AS [IsDeleted]
FROM [Semas].[TeilnahmeBetriebKomfortaktion] AS [Extent1]
WHERE [Extent1].[IsDeleted] <> cast(1 as bit)

My assumption is that TeilnahmeBetriebKomfortaktion.TeilnahmeStatus is a lazy loaded collection, resulting in the N + 1 problem. You should eagerly fetch that collection to improve your performance.
The following iterations of the foreach loop are fast, because after the first iteration those objects are no longer requested from the database server but are server from memory.