Develop Reference

Reuse LINQ's SELECT (When querying to tables with the same signature) [duplicate]

I have an Expression that converts one type of object to another type. The expression is as follows:
public Expression<Func<SQLRepository.ActionType, Model.ActionType>> DBActionTypeToActionType =
(SQLRepository.ActionType at) => new Model.ActionType()
{
ID = at.OID,
DisplayName = at.DisplayName
};
I can use the Expression like this:
var linq = (from at in dc.SQLRepositoryDC.ActionTypes select at).Select(DBActionTypeToActionType);
But I'd like to use it like this:
var linq = (from at in dc.SQLRepositoryDC.ActionTypes select DBActionTypeToActionType.Compile().Invoke(at));
I've been looking for a couple days now and I can only find references to doing this in the Where clause. It seems that if I'm able to use the function calls to do this, it should be possible using the query syntax.
The reason it is important to do use the query syntax is that some of the objects that are being selected are composed of many sub-objects and trying to chain them all of the conversions together with the function notation will be much harder to write and maintain.

It seems that if I'm able to use the function calls to do this, it should be possible using the query syntax.
That's not true. Query notation always goes via a lambda expression. For example
from x in y select z
ends up as
y.Select(x => z)
That means if you've already got an expression tree that you want to pass directly as the argument to Select, you can't use query expressions because there's this extra level of indirection.
Now the options available depend on where you need to apply the predefined expression. You can always use it in the source, and then continue with the query:
var query = from foo in dc.ActionTypes.Select(DBActionTypeToActionType)
where foo.Stuff
select foo.Other;
Or using it at the end is easy:
var query = (from bar in dc.ActionTypes
where bar.Stuff
select bar).Select(DBActionTypeToActionType);
Does that help at all?

How is OrderBy Applied?

I have the following query:
var vendors = (from pp in this.ProductPricings
join pic in this.ProductItemCompanies
on pp.CompanyId equals pic.CompanyId into left
from pic in left.DefaultIfEmpty()
orderby pp.EffectiveDate descending
group pp by new { pp.Company, SortOrder = (pic != null) ? pic.SortOrder : short.MinValue } into v
select v).OrderBy(z => z.Key.SortOrder);
Does anyone know how the last OrderBy() is applied? Does that become part of the SQL query, or are all the results loaded in to memory and then passed to OrderBy()?
And if it's the second case, is there any way to make it all one query? I only need the first item and it would be very inefficent to return all the results.

Well it will try to apply the OrderBy to the original query since you are still using an IQueryable - meaning it hasn't been converted to an IEnumerable or hydrated to a collection using ToList or an equivalent.
Whether it can or not depends on the complexity of the resulting query. You'd have to try it to find out. My guess is it will turn the main query into a subquery and layer on a "SELECT * FROM (...) ORDER BY SortOrder" outer query.

Given your specific example the order by in this situation most, likely be appliead as part of the expression tree when it getting build, there for it will be applied to sql generated by the LINQ query, if you would convert it to Enumarable like ToList as mentioned in another answer then Order by would be applied as an extension to Enumerable.

Might use readable code, because as you write it is not understandable.
You will have a problem in the future with the linq statement. The problem is that if your statement does not return any value the value will be null and whenever you make cause a exception.
You must be careful.
I recommend you to do everything separately to understand the code friend.

Concatenating three lists into one with LINQ throws an exception

Ok, I must be doing something dumb, but shouldn't this work? I have following three lists:
var commonViews = (from v in context.TPM_VIEWS where v.VIEWID < 0 select v); // IQueryable<TPM_VIEWS>
var ownedViews = (from v in context.TPM_VIEWS where v.OWNERID == userId && v.VIEWID > 0 select v); // IQueryable<TPM_VIEWS>
var sharedViews = (from v in context.TPM_USER.Include("TPM_VIEWS2") where v.USERID == userId select v).First().TPM_VIEWS2; // EntityCollection<TPM_VIEWS>
Each list has the proper values and count. I can return any one of these lists:
return commonViews.ToList();
And I can return a any two of these lists:
return commonViews.Concat(ownedViews).ToList();
However, when I try to return all three:
return commonViews.Concat(ownedViews).Concat(sharedViews).ToList();
I get the exception:
Unable to create a constant value of type 'Entity.TPM_VIEWS'. Only
primitive types or enumeration types are supported in this context.
What am I doing wrong? All three values are indeed enumerable. Mostly, I'm asking this question because it's the best possible way to guarantee I'll notice the problem 30 seconds after posting.
UPDATE:
I'm 93% sure the problem is here:
var sharedViews = (from v in context.TPM_USER.Include("TPM_VIEWS2") where v.USERID == userId select v).First().TPM_VIEWS2;
This looks like an enumerable list of TPM_VIEWS object, and I can call ToList() on it and get the correct data, but it doesn't play well with the other lists.
UPDATE 2:
This actually works. Points to the person who can tell me why!
commonViews.ToList().Concat(ownedViews.ToList()).Concat(sharedViews.ToList()).ToList();

The problem is that Concat() on an EF IQueryable<T> will turn the entire concatenation into a single query.
When you call .Concat(sharedViews), you're passing a scalar (pre-loaded) collection of your nested entity class.
EF doesn't know how to convert that into a query, so it complains.
You can make it faster by calling AsEnumerable() instead of ToList().

This actually works. Points to the person who can tell me why!
commonViews.ToList().Concat(ownedViews.ToList()).Concat(sharedViews.ToList()).ToList();
That's because each of the original queries is executed separately; you're only concatenating the results in memory. There seems to be a bug in the Entity Framework query translator when you combine the 3 queries, but when you call ToList on each of them, they're no longer EF queries, they're just lists, so they're concatenated using Linq to Objects.

SqlException about UNION, INTERSECT and EXCEPT

Could someone help me with this exception? I don't understand what it means or how to fix it... It is an SqlException with the following message:
All queries combined using a UNION, INTERSECT or EXCEPT operator must have an equal number of expressions in their target lists.
I get it when running a query in pseudo code looking like this:
// Some filtering of data
var query = data.Subjects
.Where(has value)
.Where(has other value among some set of values);
// More filtering, where I need to have two different options
var a = query
.Where(some foreign key is null);
var b = query
.Where(some foreign key is not null)
.Where(and that foreign key has a property which is what I want);
query = a.Union(b);
// Final filter and then get result as a list
var list = query
.Where(last requirement)
.ToList();
If I remove the a.Union(b) parts, it runs without the exception. So I know the error is there. But why do I get it? And how can I fix it? Am I doing something too crazy here? Have I misunderstood how to use the Union thing?
Basically what I have is some entities which have a foreign key to some other entity. And I need to get all the entities which either have that foreign key set to null or where that foreign entity fulfills some requirements.

Judging from the SQL error you listed you may be experiencing the same issue I was. Basically when Linq to SQL queries that use the Concat or Union extension method on two different queries it appears that there is a bug in Linq to SQL which optimizes each projection separately without regard to the fact that the projection must stay the same in order to accomplish the SQL Union.
References:
LINQ to SQL produces incorrect TSQL when using UNION or CONCAT
Linq to SQL Union Same Fieldname generating Error
If this happens to be your problem as well I've found a solution that is working for me as shown below.
var queryA =
from a in context.TableA
select new
{
id,
name,
onlyInTableA,
}
var queryB =
from b in context.TableB
let onlyInTableA = default(string)
select new
{
id,
name,
onlyInTableA,
}
var results = queryA.Union(queryB).ToList();

Since this looks like a problem with the generated SQL, you should try to use either an SQL Profiler, or use this code for DebuggerWritter class to write the SQL to your Output Window in Visual Studio.
The SQL error is normally caused by the fields retrieved for UNION is not the same for the 2 queries. For example, if the first query might have 3 fields, but the second query has 4 fields, this error will occur. So, seeing the generated SQL will definitely help in this case.

Can you perhaps write it in a single query?
.Where(row => row.ForeignKey == null || row.ForeignKey.SomeCondition);
There are also ways of merging expressions (OrElse), but that isn't trivial.
Not sure where the error comes from, though!
edit: haven't tested it, but this should be logically equivalent to a UNION:
public static IQueryable<T> WhereAnyOf<T>(
this IQueryable<T> source,
params Expression<Func<T, bool>>[] predicates)
{
if (source == null) throw new ArgumentNullException("source");
if (predicates == null) throw new ArgumentNullException("predicates");
if (predicates.Length == 0) return source.Where(row => false);
if (predicates.Length == 1) return source.Where(predicates[0]);
var param = Expression.Parameter(typeof(T), "row");
Expression body = Expression.Invoke(predicates[0], param);
for (int i = 1; i < predicates.Length; i++)
{
body = Expression.OrElse(body,
Expression.Invoke(predicates[i], param));
}
return source.Where(Expression.Lambda<Func<T, bool>>(body, param));
}

query = a.Union(b);
Not a good idea to mutate captured variables... Likely the cause of the error.
UPDATE: ok not
Here is another idea. The hint is in the error message.
var a = query
.Where(some foreign key is null)
.Select(x => x);
Or play by adding another 'fake' Where till they do become equal :)

I would call data.GetCommand(query) and analyze the resulting DbCommand (especially the generated SQL string). That should give you a clue to what goes wrong.
There is no projection going on anywhere so I would expect both target lists to be the same.
You could try to reduce your query to a smaller one that still doesn't work. Start with query.Union(query) (this should at least work). Than add your Where calls one by one to see when it stops working.
It must be one of your Where calls that adds extra columns to your select list.

Are you by any chance passing in a value to the 'select' side in a variable, or are you returning the same field more than once? SP1 introduced a bug where it tries to 'optimize' out such things and that can cause union queries to break (due to the query parts 'optimizing' out different passed-in params).
If you post your actual query rather than pseudo code it makes it easier to identify if this is the case.
(And a workaround if this is the case is to materialize the individual parts first and then do a client-side (L2O) union).

jpierson has the problem summarised correctly.
I also had the problem, this time caused by some literals in the select statement:
Dim results = (From t in TestDataContext.Table1 _
Where t.ID = WantedID _
Select t.name, SpecialField = 0, AnotherSpecialField = 0, t.Address).Union _
From t in TestDataContext.Table1 _
Where t.SecondID = WantedSecondID _
Select t.name, SpecialField = 1, AnotherSpecialField = 0, t.Address)
The first sub-query of "SpecialField = 0" and the "AnotherSpecialField = 0" were optimised, resulting in one field instead of two being used in the union, which will obviously fail.
I had to change the first query so that the SpecialField & AnotherSpecialField had different values, much like in the second sub-query.

Well I had an issue with this. Using Sql 08 i had two table functions that returned an int and a string in both cases. I created a complex object and used linq to attempt a UNION. Had an IEqualityComparer to do the comparision. All compiled fine, but crashed with a unsupported overload. Ok, i realised the problem discussed seemed to smack of defered execution. So i get the collections, and place ToList(), then do the UNION and it is all good. Not sure if this is helpful, but it works for me

How to deal with datatypes returned by LINQ

I'm new to LINQ, I've used LINQ to SQL to link to two tables, it does return data, which is cool. What I'm trying to understand is what datatype is being returned and how do I work with this datatype?
I'm used to dealing with datatables. Are we throwing out datatables (and all the other ADO.Net object like rows, datasets etc.) now if using LINQ? If so, what are we replacing that with and how can I use it to do everything I did before with datatables? Also--does it make sense to replace datables, was there a deficiency with them?
Here is some code:
protected IEnumerable<string> GetMarketCodes()
{
LINQOmniDataContext db = new LINQOmniDataContext();
var mcodes = from p in db.lkpMarketCodes
orderby 0
select p;
return (IEnumerable<string>) mcodes;
}
This code does currently return data (I can see it in debug), but errors at the "return" line, because apparently my datatype is not IEnumerables, which was my best guess. So, one thing I'd like to understand as well is what datatype is my data being put into and how to return it to the calling function.

It is returning an IQueryable<lkpMarketCode>, assuming that that lkpMarketCode is the type of data in db.lkpMarketCodes. If you want the strings, you need to select p.SomeProperty;, not just select p;.
You shouldn't need to cast (since IQueryable<T> implements IEnumerable<T>); it should also tell you this if you hover on mcodes.

I find it more convenient to return List<>'s so I know what I'm dealing with. So your code would be:
protected List<string> GetMarketCodes()
{
LINQOmniDataContext db = new LINQOmniDataContext();
var mcodes = from p in db.lkpMarketCodes
orderby 0
select p.SomeProperty;
return mcodes.ToList();
}
Having said that, I've hardly used LINQ-to-SQL so there are probably better ways around..

It's returning an IQueryable object.
How does your table look like? I'm guessing the error is because your lkpMarketCodes table is not just one string column. It's returning the whole table.
If you want to return just an IEnumerable of strings, you'll have to return something that looks like this (I'm sure the syntax is a bit off):
var mcodes = from p in db.lkpMarketCodes
orderby 0
select new { p.StringColumnName };

LINQ returns IQueryable<type>'s. This is a superset of IEnumerable. The reason you are getting an error is that your query is not returning an IQueryable<string> it's returning an IQueryable<lkpMarketCodes>. lkpMarketCodes is most likely an object, which can be thought of as similar to a row of records.
LINQ is a Object-Relational mapper, it maps Columns and Rows to Fields and Objects.
You can do pretty much all the same things that you could in ADO, but it works with objects rather than generic rows, so it's more type safe.
In your example, i'm going to assume that lkpMarketCodes is a table, and that table consists of at least two fields, mcode and description.
If you want to return an IEnumerable<string> of mcode's, you would do something like this:
protected IEnumerable<string> GetMarketCodes()
{
LINQOmniDataContext db = new LINQOmniDataContext();
var mcodes = from p in db.lkpMarketCodes
orderby 0
select p.mcode;
return mcodes;
}
This will return your IEnumerable<string> of codes. One trick you can use to find out types is to simply use the variable after its declaration, then hover your mouse over the variable name and a popup will tell you it's type.
That is, if you hover over the return mcodes, it will tell you the type, but it will not tell you the type if you hover over the var mcodes.