Develop Reference

T-SQL to LINQ to SQL using Navigation Properties

I can’t seem to come up with the right corresponding LINQ to SQL statement to generate the following T-SQL. Essentially, I'm trying to return payment information with only one of the customer's addresses... the AR address, if it exists, then the primary address, if it exists, then any address.
SELECT < payment and address columns >
FROM Payment AS p
INNER JOIN Customer AS c ON c.CustomerID = p.CustomerID
OUTER APPLY (
SELECT TOP 1 < address columns >
FROM Address AS a
WHERE a.person_id = c.PersonID
ORDER BY CASE WHEN a.BusinessType = 'AR' THEN 0
ELSE 1
END
, a.IsPrimary DESC
END
) AS pa
WHERE p.Posted = 1
We’re using the Repository Pattern to access the DB, so inside a method of the Payment Repository, I’ve tried:
var q = GetAll()
.Where(p => p.Posted == true)
.SelectMany(p => p.Customer
.Address
.OrderBy(a => a.BusinessType != "AR")
.ThenBy(a => a.Primary != true)
.Take(1)
.DefaultIfEmpty()
.Select(a => new
{
< only the columns I need from p and a >
});
But when I execute .ToList(), it throws the NullReferenceException (Object reference not set to an instance of an object) on a record where the customer has no addresses set up. So, I tried:
var q1 = GetAll().Where(p => p.Posted == true);
var q2 = q11.SelectMany(p => p.Customer
.Address
.OrderBy(a => a.BusinessType != "AR")
.ThenBy(a => a.Primary != true));
var q3 = q1.SelectMany(p => q2.Where(a => a.PersonID == p.Customer.PersonID)
.Take(1)
.DefaultIfEmpty()
.Select(a => new
{
< only the columns I need from p and a >
});
This returns the correct results, but the T-SQL it generates puts the entire T-SQL from above into the outer apply, which is then joined again on Payment and Customer. This seems somewhat inefficient and I wondered if it could be made more efficient because the T-SQL above returns in 6ms for the test case I’m using.
Additional Info:
Q: I think the problem here is that GetAll() returns IEnumerable, not IQueryable ... it would help to see this GetAll() method. - Gert Arnold
A: Actually, GetAll(), when traced all the way back, returns Table<TEntity> System.Data.Linq.GetTable<TEntity>() and Table<TEntity> does implement IQueryable.
However, DefaultIfEmpty() does return IEnumerable<Address>, which is what is throwing the exception, if I'm not mistaken, as I mentioned in the first L2S code section.
SOLUTION UPDATE
Okay, I knew I could fall back to simply going straight to joining the tables and foregoing the use of the navigation properties, and in this case, I now know that is how it should be done. It all makes sense now. I just had become accustomed to preferring using the navigation properties, but here, it’s best to go straight to joining tables.
The reason the T-SQL generated by the second L2S code section was so inefficient was because in order to get to the Address table, it required the inclusion of the Payment/Customer data.
When I simply go straight to joining the tables, the generated T-SQL, while not ideal, is much closer to the desired script code section. That’s because it didn’t require the inclusion of the Payment/Customer data. And that’s when the “well, duh” light bulb flashed on.
Thanks for all who helped on this path to discovery!

When trying a similar query it turned out that this DefaultIfEpty() call knocks down LINQ-to-SQL. The exception's stack trace shows that things go wrong in System.Data.Linq.SqlClient.SqlBinder.Visitor.IsOuterDependent, i.e. during SQL query building.
Contrary to your conclusion it's not advisable to abandon the use of navigation properties and return to explicit joins. The question is: how to use the best parts of LINQ (which includes nav properties) without troubling LINQ-to-SQL. This, by the way, is true for each ORM with LINQ support.
In this particular case I'd switch to query syntax for the main query and use the keyword let. Something like:
from p in context.Payments
let address = p.Customer
.Addresses
.OrderBy(a => a.BusinessType != "AR")
.ThenBy(a => a.Primary != true)
.FirstOrDefault()
select new
{
p.PropertyX,
address.PropertyY
...
}
This will be translated into one SQL statement and it avoids LINQ-to-SQL's apparent issue with DefaultIfEmpty.

Entity Framework Takes Too Much Time where Adding "WHERE Clause"

I'm working on Schools System, and the System have many Users(teachers And Managers).
(Tools)
Asp.net 5 (MVC With C#) VS 2015
SQL SERVER (Azure)
Code-First Approach
Each user Assigned to One Or More Of Classes.
(The User Should only See His Classes And Students inside of Classes)
in the other side
to catch Each User Classes
I've write this Method
public static IEnumerable<Class> GetClasses()
{
string UserId = HttpContext.Current.User.Identity.GetUserId();
ApplicationDbContext db = new ApplicationDbContext();
var Job = db.Jobs.Where(j => j.UserId == UserId).FirstOrDefault();
if (Job.EntityLevelID == 1)
{
var x = db.Classes.Where(a => a.Levels.SupervisionCenter.Organization.Id == Job.EntityID) as IEnumerable<Class>;
return x;
}
return null;
}
in The Students List Controller I just want the User to get his Students which are Only in his Classes
to get all the Students i write this Code
var items = db.Students.AsEnumerable().Where(o => o.Name.ToLower().Contains(search)).AsQueryable();
This Code will Give all the Students, it takes around 10 Sec to load and finish all the data (But that's is not needed Bcs, i need to Filter on ClassID Field).
when i need to filter on user Permissions based-classes
I've edit the code to be
var items = db.Students.AsEnumerable().Where(o => o.Name.ToLower().Contains(search))
.Where(s => UserDB.GetClasses().Select(c => c.Id).Contains(s.ClassID.Value))
.AsQueryable();
in the Previous case, when i add the where query it takes more than 80 Sec
EDIT (1)
The Organization Chart Will Be
Organizations
SuperVisionCenter
Schools
Classes
So I don't know what is the Problem I've made it here, can you please advise me for this
Thanks And Regards.

The very first thing that is slowing you down is AsEnumerable()
See here the effects of using AsEnumerable()
Basically you are getting all students in memory then running your where clause in app memory instead of just getting selected students.
Edit 1:
o.Name.ToLower().Contains(search)
I dont think you need to do ToLower() as MSSQL is case insensitive by default. This should remove the need for db.Students.AsEnumerable()
Edit 2:
Why don't you use join ? That would be a good optimisation to begin with.
something like
List<Student> data = (from student in db.Students
join clss in db.Classes on student.Class equals clss
where student.Name.Contains(search)).ToList()

This solved My Problem, and it takes around 3 to 4 seconds only.
int[] Classes = UserDB.GetClasses().Select(c => c.Id).ToArray();
var items = db.Students.Where(o => o.Name.ToLower().Contains(search))
.Where(r => Classes.Contains(r.ClassID.Value)).AsQueryable();

how to implement Linq to Entities IQueryable TakeWhile() functionality

Today while working with LINQ, I leanrnt that TakeWhile() is not supported for LINQ to entities, is there any efficient way to implement such a functionality? The use case I have is as below -
I have an Employee entity, and I have sorted the entity by Name, now I want to fetch the records from this IQueryable till the time (EmployeeID = 123)
Something like this -
IQueryable<Employee> employees = ObjectContext.Employees
.OrderBy(a => a.Name)
.TakeWhile(a => a.EmployeeId != 123)
However in above code the TakeWhile is not supported for Linq to Entities so it throws an error.
I am trying with below approach, Please let me know if anyone has better and efficient aproach:
Fetch first X records,
check if the required EmployeeId is part of it,
if not then fetch the next set of X records
and Concat them with previous set
and check if the EmployeeID is part of it again,
break the loop when the matching EmployeeId is found in the set of X records..

You shouldn't sort by Name, but store somewhere Id. May be, this will help:
// eager loading of employees, which name is less or equal, than stored name:
ObjectContext.Employees.Where(a => a.Name.CompareTo(storedName) < 0 || a.Name.CompareTo(storedName) == 0)
// lazy loading the rest of employees:
ObjectContext.Employees.Where(a => a.Name.CompareTo(storedName) > 0)

There is no TakeWhile() equivalent in SQL so you'll have to do that step in memory. You can force the SQL to be executed by casting the IQueryable<> to an IEnumerable<> prior to adding the filters that don't have SQL equivalents.
var employees = ObjectContext.Employees
.OrderBy(a => a.Name)
.AsEnumerable() // filters after this point will be done in memory on each record
.TakeWhile(a => a.EmployeeId != 123)

very slow IOrderedQueryable ToList()

I have the query that returns parent with filtered child's:
Context.ContextOptions.LazyLoadingEnabled = false;
var query1 = (from p in Context.Partners
where p.PartnerCategory.Category == "03"
|| p.PartnerCategory.Category == "02"
select new
{
p,
m = from m in p.Milk
where m.Date >= beginDate
&& m.Date <= endDate
&& m.MilkStorageId == milkStorageId
select m,
e = p.ExtraCodes,
ms = from ms in p.ExtraCodes
select ms.MilkStorage,
mp = from mp in p.MilkPeriods
where mp.Date >= beginDate
&& mp.Date <= endDate
select mp
}).Where(
p =>
p.p.ExtraCodes.Select(ex => ex.MilkStorageId).Contains(
milkStorageId) ).OrderBy(p => p.p.Name);
var partners = query1.AsEnumerable().ToList();
Query return 200 records and converting from IOrderedQueryable ToList() is very slow. Why?
After profiling query in sql server management studio i've noticed that query execute's 1 second and returns 2035 records.

There could be a number of reasons for this and without any profiler information it's just guess work and even highly educated guess work by some one that knows the code and domain well is often wrong.
You should profile the code and since it's likely that the bottleneck is in the DB get the command text as #Likurg suggests and profile that in the DB. It's likely that you are missing one or more indexes.
There's a few things you could do to the query it self as well if for nothing else to make it easier to understand and potentially faster
E.g.
p.p.ExtraCodes.Select(ex => ex.MilkStorageId).Contains(milkStorageId)
is really
p.p.ExtraCodes.Any(ex => ex.MilkStorageId == milkStorageId)
and could be moved to the first where clause potentially lowering the number of anonymously typed objects you create. That said the most likely case is that one of the many fields you use in your comparisons are with out an index potentially resulting in a lot of table scans for each element in the result set.
Some of the fields where an index might speed things up are
p.p.Name
m.Date
m.MilkStorageId
mp.Date
PartnerCategory.Category

The reason it is slow is because when you do ToList that is the time when the actual query execution takes place. This is called deferred execution.
You may see: LINQ and Deferred Execution
I don't think you need to do AsEnumerable when converting it to a list, you can do it directly like:
var partners = query1.ToList();

At first, look at the generated query by using this
Context.GetCommand(query1).CommandText;
then invoke this command in db. And check how many records reads by profiler.

Getting Entity Framework to eager load on Group By

I know that changing the shape of a query causes Entity Framework to ignore the include calls but is there a way I can get it to load the sub properties when I do a select many and a group by. In the following example I want to notify all the employees who have a job booked in a certain time period. Calling .ToArray() after the where only hits the database once but I am doing the SelectMany and GroupBy in memory. Is there a way I can get the SelectMany and the GroupBy to happen on the SQL server and still include the ServiceType and Ship and the Employee details?
I am looking for a way to make one SQL call to the database and end up with a list of Employees who have a job in the time period and the jobs they are assigned to.
var employeeJobs = DataContext.Jobs.
Include("ServiceType").
Include("Ship").
Include("JobEmployees.Employee").
Where(j => j.Start >= now && j.Start <= finish).
OrderBy(j => j.Start).
ToArray().
SelectMany(j => j.JobEmployees, (j, je) => new {
Job = j,
Employee = je.Employee
}).GroupBy(j => j.Employee);

The following should work:
var q = from e in DataContext.Employees
where e.Job.Start > ....
order by e.Job.Start
select new {Employee = e, Job = e.Job, Ship = e.Job.Ship, ServiceType = e.Job.ServiceType}; // No db hit yet.
var l = q.GroupBy(item=>item.Employee) // no db hit yet.
.ToList(); // This one causes a db hit.

why don't you create a view and then reference this from the EF?
, this also has the added benefit of the database server doing the work, rather than the app server.

Trying move the Include()'s to the very end after your groupby.