Query with large WHERE clause causes timeout exception in EF6 - c#

clients = (IQueryable<client>)opts.ApplyTo(_dbContext.sp_client.FromSqlRaw(
SqlQuerys.GetVisibleClients(),
new NpgsqlParameter("providerId", providerId),
new NpgsqlParameter("userId", userId))
.Where(predicate)
.Include(q => q.NameDataQualityValue)
.Include(q => q.SsnDataQualityValue)
.Include(q => q.VeteranStatus)
.Include(q => q.Answers).ThenInclude(q => q.AnswerValue)
.Include(q => q.Answers).ThenInclude(q => q.Question)
.Include(q => q.HouseHoldsRelationship).AsQueryable());
}
List<client> visibleClientsList = client.ToList();
In client.ToList(), the code throws a timeout error - the issue in predicate it have the condition a switch statement
"gt" => predicate.And(q => q.Answers.Any(q =>
(_dbContext.da_answer.FromSqlRaw(
SqlQuerys.GetTopVisibleAnswers(),
new NpgsqlParameter("providerId", providerId),
new NpgsqlParameter("userId", userId),
new NpgsqlParameter("effectiveTimeSpan", dateEffective.Value)).Any(o => o.answer_id == q.answer_id) &&
q.Question.virt_field_name == answerCondition.virtualFieldName && q.val_date != null &&
q.val_date.Value.Date > dateValue.Date))),

Related

Linq variable inside select

I have a following query
return _context.Table1
.Where(x => x.ContactId == contactKey.Id)
.Include(x => x.Table2)
.Include(x => x.Table3.Table4)
.Select(a =>
new MyReadModel
{
PriorityAssignment = true,
LastContactedDate = (a.Table3.Table4 != null && a.Table3.Table4.FirstOrDefault(h =>
h.Id == a.Table2.FkId
) != null ?
a.Table3.Table4.FirstOrDefault(h => && h.Id == a.Table2.FkId
).LastContactedDatetime : default
)
}
)
.ToListAsync();
What i wants is to simplify LastContactedDate assignment with in select. I think we can assign
a.Table3.Table4.FirstOrDefault(h =>
h.Id == a.Table2.FkId
)
to some variable but can't able to do it
can someone identify what is needed
With EF Core you don't have to check for null, LINQ Translator do not execute your code, but uses it for translation to the SQL. Also Includes is not needed if you do not retrieve whole objects.
return await _context.Table1
.Where(x => x.ContactId == contactKey.Id)
.Select(a => new MyReadModel
{
PriorityAssignment = true,
LastContactedDate = (DateTime?)a.Table3.Table4.Where(h => h.Id == a.Table2.FkId)
.OrderByDescending(h => LastContactedDatetime)
.Select(h => LastContactedDatetime)
.FirstOrDefault()
}).ToListAsync();
you can use it like this example
List<string> someList= new List<string>();
someList= someList.Select(x =>
{
var newVariable= "newVariable";
return newVariable;
}).ToList();
in your case
return _context.Table1
.Where(x => x.ContactId == contactKey.Id)
.Include(x => x.Table2)
.Include(x => x.Table3.Table4)
.Select(a => {
Table4 newVariable = null;
if(a.Table3.Table4 != null)
newVariable = a.Table3.Table4.FirstOrDefault(h => h.Id == a.Table2.FkId;
var result = new MyReadModel
{
PriorityAssignment = true,
LastContactedDate = (newVariable != null ? newVariable.LastContactedDatetime : default
)
};
}
) .ToListAsync();

c# Lambda referencing class object in Select method

I've written this code which works:
var uniqueCustomerIdList = services
.GroupBy(x => x.CustomerId)
.Select(cl => new Customer
{
CustomerId = cl.First().CustomerId,
CustomerName = cl.First().CompanyName,
PdfServices = services.Where(x => x.CustomerId == cl.First().CustomerId).ToList(),
PdfServiceLines = services.Where(x => x.CustomerId == cl.First().CustomerId).ToList()
.GroupBy(l => l.ServiceDescription)
.Select(cy => new PdfServiceLine
{
ServiceName = cy.First().ServiceDescription,
Quantity = cy.Count(),
UnitPrice = cy.First().PlanCharge,
ServiceCharges = services.Where(x => x.CustomerId == cl.First().CustomerId && x.ServiceDescription == cy.First().ServiceDescription).Sum(y => y.TotalBill),
UsageCharges = usage.Where(x => x.CustomerId == cl.First().CustomerId && x.ServiceDescription == cy.First().ServiceDescription).Sum(y => y.Charge),
Total = cy.Sum(c => c.PlanCharge),
}).ToList(),
PdfUsages = usage.Where(x => x.CustomerId == cl.First().CustomerId).ToList()
})
.ToList();
I wanted to know if it's possible to reference values from the outer Select statement in the inner statement? As it looks rather clunky at the moment.
For instance in the outer Customer select I use PdfServices - can I use that in the inner select where I have ServiceCharges?
ServiceCharges = PdfServices.Where(s => s.ServiceDescription == cy.First().ServiceDescription).Sum(y => y.TotalBill)
instead of
ServiceCharges = services.Where(x => x.CustomerId == cl.First().CustomerId && x.ServiceDescription == cy.First().ServiceDescription).Sum(y => y.TotalBill),
Thanks,
Lee.
The results that you want to re-use are members of an anonymous object, not variables. As such, you cannot expect them to be available, like a variable would be, to the inner lambda. If you re-wrote your statement, you could store them in intermediate variables:
var uniqueCustomerIdList = services
.GroupBy(x => x.CustomerId)
.Select(cl =>
{
var pdfServices = services.Where(x => x.CustomerId == cl.First().CustomerId).ToList();
return new Customer
{
CustomerId = cl.First().CustomerId,
CustomerName = cl.First().CompanyName,
PdfServices = pdfServices,
PdfServiceLines = pdfServices
.GroupBy(l => l.ServiceDescription)
.Select(cy => new PdfServiceLine
{
ServiceName = cy.First().ServiceDescription,
Quantity = cy.Count(),
UnitPrice = cy.First().PlanCharge,
ServiceCharges = services.Where(x => x.CustomerId == cl.First().CustomerId && x.ServiceDescription == cy.First().ServiceDescription).Sum(y => y.TotalBill),
UsageCharges = usage.Where(x => x.CustomerId == cl.First().CustomerId && x.ServiceDescription == cy.First().ServiceDescription).Sum(y => y.Charge),
Total = cy.Sum(c => c.PlanCharge),
}).ToList(),
PdfUsages = usage.Where(x => x.CustomerId == cl.First().CustomerId).ToList()
};
})
.ToList();
Note that the lamda body now uses curly braces and a return statement. This is not convertible to an expression tree, and some ORM frameworks, like Entity Framework, will not be able to translate the lamda into SQL.

Linq Query with a Where clause in an Include statement [duplicate]

This question already has answers here:
EF: Include with where clause [duplicate]
(5 answers)
Closed 2 years ago.
I am trying to replace my big, ugly query; although ugly it works as desired:-
using (var ctx = new Data.Model.xxxTrackingEntities())
{
var result = ctx.Offenders
.Join(ctx.Fees, o => o.OffenderId, f => f.OffenderId,
(o, f) => new { Offenders = o, Fees = f })
.Join(ctx.ViolationOffenders, o => o.Fees.ViolationId, vo => vo.ViolationId,
(o, vo) => new { Offenders = o, ViolationOffenders = vo })
.Join(ctx.Violations, v => v.ViolationOffenders.ViolationId, vo => vo.ViolationId,
(v, vo) => new { Violations = v, ViolationOffenders = vo })
.Where(o => o.Violations.Offenders.Offenders.YouthNumber != "")
.ToList();
gvwData.DataSource = result;
}
with the following linq Query:-
var result = ctx.Offenders
.Include(o => o.Fees.Where(f => f.Amount != null))
.Include(o => o.ViolationOffenders)
.Include(o => o.ViolationOffenders.Select(of => of.Violation))
.Where(o => o.YouthNumber != "" && o.FirstName != "")
.ToList();
I am blowing up on the 2nd line of the query... once I add the Where clause... o => o.Fees.Where(f=> f.Amount != null)
The error message I get...
The Include path expression must refer to a navigation property defined on the type. Use dotted paths for reference navigation properties and the Select operator for collection navigation properties.
In addition, I tried writing my query as:-
var result = ctx.Offenders
.Include(o => o.Fees)
.Include(o => o.ViolationOffenders)
.Include(o => o.ViolationOffenders.Select(of => of.Violation))
.Where(o => o.YouthNumber != "" && o.FirstName != "" && o.Fees.Where(f=> f.Amount != null))
.ToList();
But then I get the following error:-
Operator '&&' cannot be applied to operands of type 'bool' and 'System.Collections.Generic.IEnumerable
I know the concept is right, but I need help with the syntax.
You cant have a Where inside the Where, but you can use Any which will return a boolean
var result = ctx.Offenders
.Include(o => o.Fees)
.Include(o => o.ViolationOffenders)
.Include(o => o.ViolationOffenders.Select(of => of.Violation))
.Where(o => o.YouthNumber != "" && o.FirstName != ""
&& o.Fees.Any(f=> f.Amount != null)) // here
.ToList();
In .Net 5 Filtered include feature is added (EF Core 5.0).
Supported operations are: Where, OrderBy, OrderByDescending, ThenBy, ThenByDescending, Skip, and Take
using (var context = new BloggingContext())
{
var filteredBlogs = context.Blogs
.Include(blog => blog.Posts
.Where(post => post.BlogId == 1)
.OrderByDescending(post => post.Title)
.Take(5))
.ToList();
}
MSDN Reference : https://learn.microsoft.com/en-us/ef/core/querying/related-data/eager

Linq query fine tuning

Query logic is Grouping the items by Id and Ordering it by Id. Then inside grouped items Ordering by Item1 then by Item2.
Linq query below,
var group1Items = MyList.GroupBy(g => g.Id)
.Where(w => w.Any(a => a.Code = 1)
.Select(s => new
{ key = s.Key,
items = s.OrderBy(o => o.Item1)
.ThenBy(t => t.Item2)
}
)
.OrderBy(o => o.Id)
.SelectMany(sm => sm.Items).ToList();
var group2Items = MyList.GroupBy(g => g.Id)
.Where(w => w.Any(a => a.Code = 2)
.Select(s => new
{ key = s.Key,
items = s.OrderBy(o => o.Item1)
.ThenBy(t => t.Item2)
}
)
.OrderBy(o => o.Id)
.SelectMany(sm => sm.Items).ToList();
MyList.Clear();
MyList.InsertRange(Mylist.Count, group1Items);
MyList.InsertRange(Mylist.Count, group2Items);
In the above two queries, only difference is Where condition. Is it possible to rewrite into single query?
Single query:
var groupItems = MyList.GroupBy(g => g.Id)
.Where(w => w.Any(a => a.Code == 1 || a.Code == 2)
.Select(s => new
{ key = s.Key,
items = s.OrderBy(o => o.Item1)
.ThenBy(t => t.Item2)
}
)
.OrderBy(o => o.Id)
.SelectMany(sm => sm.Items).ToList();
If Code 1 must come before Code 2:
var groupItems = MyList.GroupBy(g => g.Id)
.Where(w => w.Any(a => a.Code == 1)
.Select(s => new
{ key = s.Key,
items = s.OrderBy(o => o.Item1)
.ThenBy(t => t.Item2)
}
)
.OrderBy(o => o.Id)
.SelectMany(sm => sm.Items)
.Union(MyList.GroupBy(g => g.Id)
.Where(w => w.Any(a => a.Code == 2)
.Select(s => new
{ key = s.Key,
items = s.OrderBy(o => o.Item1)
.ThenBy(t => t.Item2)
}
)
.OrderBy(o => o.Id)
.SelectMany(sm => sm.Items)).ToList();
If you only want to avoid code duplication it may be easiest to capture a variable holding the code by which to filter.
int code = 0; // initialize with any value
var groupItems = MyList.GroupBy(g => g.Id)
.Where(w => w.Any(a => a.Code == code)
.Select(s => new
{
key = s.Key,
items = s.OrderBy(o => o.Item1)
.ThenBy(t => t.Item2)
})
.OrderBy(o => o.Id)
.SelectMany(sm => sm.Items); // No ToList() here!
MyList.Clear();
code = 1;
MyList.InsertRange(Mylist.Count, groupItems.ToList());
code = 2;
MyList.InsertRange(Mylist.Count, groupItems.ToList());
Removed where clause in the LINQ expression and implemented at InsertRange() method. This avoids redundant queries
var groupItems = MyList
.GroupBy(g => g.Id)
.Select(s => new
{ key = s.Key,
items = s.OrderBy(o => o.Item1)
.ThenBy(t => t.Item2)
}
)
.OrderBy(o => o.Id)
.SelectMany(sm => sm.Items).ToList();
MyList.Clear();
MyList.InsertRange(Mylist.Count, groupItems.Where(w => w.Code == 1));
MyList.InsertRange(Mylist.Count, groupItems.Where(w => w.Code == 2));

OrderByDescending doesn't work in nested linq statement

In Linqpad, i can see the correct list. But in code, after putting in the list collection, order by doesn't work for BeginDate. If i use BeginDate with Max, it works. I don't understand where i am wrong?
var templist = contentRepository
.Get(q => (q.Status == (int)StatusEnum.Active) &&
(q.CategoryId == category.GetHashCode() || q.Category.ParentId == category.GetHashCode())
&& q.MinorVersion == 0
&& q.MajorVersion > 0)
.GroupBy(q => q.VersionId)
.OrderByDescending(q => q.Key)
.Select(q => new
{
VersionId = q.Key,
Id = q.Max(x => x.Id),
MajorVersion = q.Max(x => x.MajorVersion),
UpdatedAt = q.Max(x => x.UpdatedAt),
//BeginDate = q.Max(x=>x.BeginDate),
BeginDate = (q.OrderByDescending(x => x.Id).Take(1).Select(x=>x.BeginDate)).First(),
Title = (q.OrderByDescending(x => x.Id).Take(1).Select(x => x.Title)).First(),
ShowOnHomePage = (q.OrderByDescending(x => x.Id).Take(1).Select(x=>x.ShowOnHomePage)).First()
})
.OrderByDescending(x => x.BeginDate)
.Take(maxItemCount)
.ToList();
List<ContentEntity> contents = new List<ContentEntity>();
templist.ForEach(q => contents.Add(
contentRepository
.Get(x => x.VersionId == q.VersionId && x.MajorVersion == q.MajorVersion && x.MinorVersion == 0)
.FirstOrDefault()
));
return contents.Where(q => q.ShowOnHomePage == true)
.OrderByDescending(q => q.MajorVersion)
.OrderByDescending(q => q.BeginDate)
.Take(maxItemCount)
.ToList();
You are ordering by Id, not by BeginDate. Equivalent code for
q.Max(x => x.BeginDate)
Will be
q.OrderByDescending(x => x.BeginDate).Take(1).Select(x => x.BeginDate).First()
Or simplified
q.OrderByDescending(x => x.BeginDate).First().BeginDate

Categories

Resources