I can’t seem to come up with the right corresponding LINQ to SQL statement to generate the following T-SQL. Essentially, I'm trying to return payment information with only one of the customer's addresses... the AR address, if it exists, then the primary address, if it exists, then any address.
SELECT < payment and address columns >
FROM Payment AS p
INNER JOIN Customer AS c ON c.CustomerID = p.CustomerID
OUTER APPLY (
SELECT TOP 1 < address columns >
FROM Address AS a
WHERE a.person_id = c.PersonID
ORDER BY CASE WHEN a.BusinessType = 'AR' THEN 0
ELSE 1
END
, a.IsPrimary DESC
END
) AS pa
WHERE p.Posted = 1
We’re using the Repository Pattern to access the DB, so inside a method of the Payment Repository, I’ve tried:
var q = GetAll()
.Where(p => p.Posted == true)
.SelectMany(p => p.Customer
.Address
.OrderBy(a => a.BusinessType != "AR")
.ThenBy(a => a.Primary != true)
.Take(1)
.DefaultIfEmpty()
.Select(a => new
{
< only the columns I need from p and a >
});
But when I execute .ToList(), it throws the NullReferenceException (Object reference not set to an instance of an object) on a record where the customer has no addresses set up. So, I tried:
var q1 = GetAll().Where(p => p.Posted == true);
var q2 = q11.SelectMany(p => p.Customer
.Address
.OrderBy(a => a.BusinessType != "AR")
.ThenBy(a => a.Primary != true));
var q3 = q1.SelectMany(p => q2.Where(a => a.PersonID == p.Customer.PersonID)
.Take(1)
.DefaultIfEmpty()
.Select(a => new
{
< only the columns I need from p and a >
});
This returns the correct results, but the T-SQL it generates puts the entire T-SQL from above into the outer apply, which is then joined again on Payment and Customer. This seems somewhat inefficient and I wondered if it could be made more efficient because the T-SQL above returns in 6ms for the test case I’m using.
Additional Info:
Q: I think the problem here is that GetAll() returns IEnumerable, not IQueryable ... it would help to see this GetAll() method. - Gert Arnold
A: Actually, GetAll(), when traced all the way back, returns Table<TEntity> System.Data.Linq.GetTable<TEntity>() and Table<TEntity> does implement IQueryable.
However, DefaultIfEmpty() does return IEnumerable<Address>, which is what is throwing the exception, if I'm not mistaken, as I mentioned in the first L2S code section.
SOLUTION UPDATE
Okay, I knew I could fall back to simply going straight to joining the tables and foregoing the use of the navigation properties, and in this case, I now know that is how it should be done. It all makes sense now. I just had become accustomed to preferring using the navigation properties, but here, it’s best to go straight to joining tables.
The reason the T-SQL generated by the second L2S code section was so inefficient was because in order to get to the Address table, it required the inclusion of the Payment/Customer data.
When I simply go straight to joining the tables, the generated T-SQL, while not ideal, is much closer to the desired script code section. That’s because it didn’t require the inclusion of the Payment/Customer data. And that’s when the “well, duh” light bulb flashed on.
Thanks for all who helped on this path to discovery!
When trying a similar query it turned out that this DefaultIfEpty() call knocks down LINQ-to-SQL. The exception's stack trace shows that things go wrong in System.Data.Linq.SqlClient.SqlBinder.Visitor.IsOuterDependent, i.e. during SQL query building.
Contrary to your conclusion it's not advisable to abandon the use of navigation properties and return to explicit joins. The question is: how to use the best parts of LINQ (which includes nav properties) without troubling LINQ-to-SQL. This, by the way, is true for each ORM with LINQ support.
In this particular case I'd switch to query syntax for the main query and use the keyword let. Something like:
from p in context.Payments
let address = p.Customer
.Addresses
.OrderBy(a => a.BusinessType != "AR")
.ThenBy(a => a.Primary != true)
.FirstOrDefault()
select new
{
p.PropertyX,
address.PropertyY
...
}
This will be translated into one SQL statement and it avoids LINQ-to-SQL's apparent issue with DefaultIfEmpty.
Related
Because of a poor design on our database I have to write a complex query to get my data.
I need to get all valid data from a table_1. In other works I need to get each valid row of my table_1. I don't have a simple valid or invalid column in my table_1. This information is stored in a table_2. Table_2 contains all invalid row with the error message.
Let say the data I need to retrieve are orders. For my example notice that OrderNo is the number of an order. So I can have multiple line, version, of this order in table_1 and I can also have multiple line of error on this order in table_2. So I will also have to use a version number.
I already tried this:
table_1.Where(y => (y.OrderNo == "1234"));
table_2.Where(y => (y.OrderNo == "1234")).Select(y => y.Version).Distinct();
And I think I need to do something like this:
var errorList = table_2.Where(y => (y.OrderNo == "1234")).Select(y => y.Version).Distinct();
table_1.Where(y => (y.OrderNo == "1234" && y.Version.NOT_IN(erriList)));
Could you help me?
I suppose you are searching for Contains function with ! symbol (logical negation operator). Like this:
var errorList = table_2.Where(y => y.OrderNo == "1234")
.Select(y => y.Version);
var res = table_1.Where(y => y.OrderNo == "1234"
//here you get only rows that doesn't represent in errorList
&& !errorList.Contains(y.Version));
to get data from a table but not if in another table
This is called antijoin. While you can use Contains and Any based approaches presented in the other answers, usually you'll get the best performance by using the classic SQL approach - LEFT OUTER JOIN combined with checking the right side for NULL.
Which in LINQ looks like this:
var query =
from t1 in table_1
//where t1.OrderNo == "1234"
join t2 in table_2 on t1.OrderNo equals t2.OrderNo into t2group
from t2 in t2group.DefaultIfEmpty()
where t2 == null
select t1;
Actually when you use OrderNo filter, most probably there will not be a noticeable speed difference between this and other queries. The main benefit of the above would be if you remove that filter, although many nowadays SQL query optimizers are able to derive one and the same execution plan regardless of whether the query uses JOIN / IN / EXISTS constructs.
How about this:
var validRows = table1
.Where(t1 => !table2
.Any(t2 => t1.OrderNo == t2.OrderNo &&
t1.Version == t2.Version));
Note that this is far more efficient in SQL unless you're using something fancy that translates the expression to SQL.
I'm querying my local list against database and would like to know what's the best way to do so.
Currently I'm loading my database to memory where "options" in local List
tmp.AddRange(from course in cs.studhists.Where(x => x.year == year).AsEnumerable()
from option in options.Where(x => x.type.Equals("course"))
join stud in cs.sstudents on course.studid equals stud.studid
where
course.csid.Contains(option.identifier) && course.crsinst.Contains(option.extra_identifier)
select stud);
"studhists" table has quite a few rows and if I didn't have option to select only the ones with current year or if year was inside option object it would take a while to load to memory.
Alternatively I could just loop through every option (which worked a lot faster before I figured out to load only data with current year. I haven't timed it, but I think it still does.
foreach (OptionListItem opt in options.Where(x => x.type.Equals("course")))
{
tmp.AddRange(from course in cs.studhists
join stud in cs.sstudents on course.studid equals stud.studid
where course.year == year
&& course.csid.Contains(opt.identifier) && course.crsinst.Contains(opt.extra_identifier)
select stud);
}
Is there a way to maybe create temp table to hold "options" and query it?
Or am I completely missing some other way to do this?
Because you're using "Contains", it's difficult, and you'd probably have build a query dynamically, using PredicateBuilder:
var predicate = PredicateBuilder.False<Course>();
foreach(var opt in options.Where(x => x.type.Equals("course"))
{
predicate = predicate.Or(course.csid.Contains(opt.identifier) && course.crsinst.Contains(opt.extra_identifier));
}
(the first example on the Predicate builder page shows a similar example)
I have two tables from which I want to select data from:
Document_Data
Document_info
I want to execute the following query :
SELECT DISTINCT Document_Data.DOC_CLASS, TITLE FROM Document_info,Document_Data WHERE (((DOC_STATUS = '1') AND (PORTAL = 'First Page'))) AND (Document_info.DOC_NUMBER = Document_Data.DOC_NUMBER AND Document_info.REVISION = Document_Data.REVISION AND STATUS = 'CURRENT' AND Document_Data.DOC_CLASS = 'MESSAGE')
Can anyone give me info on how to execute the following query using Linq?
I have made a few assumptions since your query did leave off a few table names. I assumed that STATUS was on the Document_data table and DOC_STATUS was on the Document_info table. If its any different, it shouldn't be hard to modify this query to work.
DbContext is your entity framework context or wherever your store your db collections.
dbContext.Document_info.Where(i => i.DOC_STATUS == "1" && i.PORTAL == "First Page")
.Join(dbContext.Document_data.Where(d => d.DOC_CLASS == "MESSAGE" && d.STATUS == "CURRENT"),
i => new { i.REVISION, i.DOC_NUMBER }, //Document_info
d => new { d.REVISION, d.DOC_NUMBER }, //Document_data
(i, d) => new { d.DOC_CLASS, i.TITLE }) //(Document_info, Document_data)
.Distinct()
.ToList();
The way this works is that it first filters the document_info table to what you wanted from there. It then joins it with a filtered Document_data table on a composite "key" made up of REVISION and DOC_NUMBER. After that, it runs the Distinct and executes the whole query with a ToList.
The above should compile to valid SQL (at least it would using the MySQL connector...I haven't tried anything like that with MSSQL, but I assume that since the MSSQL one works better than MySQL so it would make sense that it would work there too). This particular query would come out to be a little convoluted, however, and might not work very optimally unless you have some foreign keys defined on REVISION and DOC_NUMBER.
I would note that your query will only return things where d.DOC_CLASS == "MESSAGE" and so your results will be quite repetitious.
Today while working with LINQ, I leanrnt that TakeWhile() is not supported for LINQ to entities, is there any efficient way to implement such a functionality? The use case I have is as below -
I have an Employee entity, and I have sorted the entity by Name, now I want to fetch the records from this IQueryable till the time (EmployeeID = 123)
Something like this -
IQueryable<Employee> employees = ObjectContext.Employees
.OrderBy(a => a.Name)
.TakeWhile(a => a.EmployeeId != 123)
However in above code the TakeWhile is not supported for Linq to Entities so it throws an error.
I am trying with below approach, Please let me know if anyone has better and efficient aproach:
Fetch first X records,
check if the required EmployeeId is part of it,
if not then fetch the next set of X records
and Concat them with previous set
and check if the EmployeeID is part of it again,
break the loop when the matching EmployeeId is found in the set of X records..
You shouldn't sort by Name, but store somewhere Id. May be, this will help:
// eager loading of employees, which name is less or equal, than stored name:
ObjectContext.Employees.Where(a => a.Name.CompareTo(storedName) < 0 || a.Name.CompareTo(storedName) == 0)
// lazy loading the rest of employees:
ObjectContext.Employees.Where(a => a.Name.CompareTo(storedName) > 0)
There is no TakeWhile() equivalent in SQL so you'll have to do that step in memory. You can force the SQL to be executed by casting the IQueryable<> to an IEnumerable<> prior to adding the filters that don't have SQL equivalents.
var employees = ObjectContext.Employees
.OrderBy(a => a.Name)
.AsEnumerable() // filters after this point will be done in memory on each record
.TakeWhile(a => a.EmployeeId != 123)
I have the query that returns parent with filtered child's:
Context.ContextOptions.LazyLoadingEnabled = false;
var query1 = (from p in Context.Partners
where p.PartnerCategory.Category == "03"
|| p.PartnerCategory.Category == "02"
select new
{
p,
m = from m in p.Milk
where m.Date >= beginDate
&& m.Date <= endDate
&& m.MilkStorageId == milkStorageId
select m,
e = p.ExtraCodes,
ms = from ms in p.ExtraCodes
select ms.MilkStorage,
mp = from mp in p.MilkPeriods
where mp.Date >= beginDate
&& mp.Date <= endDate
select mp
}).Where(
p =>
p.p.ExtraCodes.Select(ex => ex.MilkStorageId).Contains(
milkStorageId) ).OrderBy(p => p.p.Name);
var partners = query1.AsEnumerable().ToList();
Query return 200 records and converting from IOrderedQueryable ToList() is very slow. Why?
After profiling query in sql server management studio i've noticed that query execute's 1 second and returns 2035 records.
There could be a number of reasons for this and without any profiler information it's just guess work and even highly educated guess work by some one that knows the code and domain well is often wrong.
You should profile the code and since it's likely that the bottleneck is in the DB get the command text as #Likurg suggests and profile that in the DB. It's likely that you are missing one or more indexes.
There's a few things you could do to the query it self as well if for nothing else to make it easier to understand and potentially faster
E.g.
p.p.ExtraCodes.Select(ex => ex.MilkStorageId).Contains(milkStorageId)
is really
p.p.ExtraCodes.Any(ex => ex.MilkStorageId == milkStorageId)
and could be moved to the first where clause potentially lowering the number of anonymously typed objects you create. That said the most likely case is that one of the many fields you use in your comparisons are with out an index potentially resulting in a lot of table scans for each element in the result set.
Some of the fields where an index might speed things up are
p.p.Name
m.Date
m.MilkStorageId
mp.Date
PartnerCategory.Category
The reason it is slow is because when you do ToList that is the time when the actual query execution takes place. This is called deferred execution.
You may see: LINQ and Deferred Execution
I don't think you need to do AsEnumerable when converting it to a list, you can do it directly like:
var partners = query1.ToList();
At first, look at the generated query by using this
Context.GetCommand(query1).CommandText;
then invoke this command in db. And check how many records reads by profiler.