Develop Reference

How is OrderBy Applied?

I have the following query:
var vendors = (from pp in this.ProductPricings
join pic in this.ProductItemCompanies
on pp.CompanyId equals pic.CompanyId into left
from pic in left.DefaultIfEmpty()
orderby pp.EffectiveDate descending
group pp by new { pp.Company, SortOrder = (pic != null) ? pic.SortOrder : short.MinValue } into v
select v).OrderBy(z => z.Key.SortOrder);
Does anyone know how the last OrderBy() is applied? Does that become part of the SQL query, or are all the results loaded in to memory and then passed to OrderBy()?
And if it's the second case, is there any way to make it all one query? I only need the first item and it would be very inefficent to return all the results.

Well it will try to apply the OrderBy to the original query since you are still using an IQueryable - meaning it hasn't been converted to an IEnumerable or hydrated to a collection using ToList or an equivalent.
Whether it can or not depends on the complexity of the resulting query. You'd have to try it to find out. My guess is it will turn the main query into a subquery and layer on a "SELECT * FROM (...) ORDER BY SortOrder" outer query.

Given your specific example the order by in this situation most, likely be appliead as part of the expression tree when it getting build, there for it will be applied to sql generated by the LINQ query, if you would convert it to Enumarable like ToList as mentioned in another answer then Order by would be applied as an extension to Enumerable.

Might use readable code, because as you write it is not understandable.
You will have a problem in the future with the linq statement. The problem is that if your statement does not return any value the value will be null and whenever you make cause a exception.
You must be careful.
I recommend you to do everything separately to understand the code friend.

Concatenating three lists into one with LINQ throws an exception

Ok, I must be doing something dumb, but shouldn't this work? I have following three lists:
var commonViews = (from v in context.TPM_VIEWS where v.VIEWID < 0 select v); // IQueryable<TPM_VIEWS>
var ownedViews = (from v in context.TPM_VIEWS where v.OWNERID == userId && v.VIEWID > 0 select v); // IQueryable<TPM_VIEWS>
var sharedViews = (from v in context.TPM_USER.Include("TPM_VIEWS2") where v.USERID == userId select v).First().TPM_VIEWS2; // EntityCollection<TPM_VIEWS>
Each list has the proper values and count. I can return any one of these lists:
return commonViews.ToList();
And I can return a any two of these lists:
return commonViews.Concat(ownedViews).ToList();
However, when I try to return all three:
return commonViews.Concat(ownedViews).Concat(sharedViews).ToList();
I get the exception:
Unable to create a constant value of type 'Entity.TPM_VIEWS'. Only
primitive types or enumeration types are supported in this context.
What am I doing wrong? All three values are indeed enumerable. Mostly, I'm asking this question because it's the best possible way to guarantee I'll notice the problem 30 seconds after posting.
UPDATE:
I'm 93% sure the problem is here:
var sharedViews = (from v in context.TPM_USER.Include("TPM_VIEWS2") where v.USERID == userId select v).First().TPM_VIEWS2;
This looks like an enumerable list of TPM_VIEWS object, and I can call ToList() on it and get the correct data, but it doesn't play well with the other lists.
UPDATE 2:
This actually works. Points to the person who can tell me why!
commonViews.ToList().Concat(ownedViews.ToList()).Concat(sharedViews.ToList()).ToList();

The problem is that Concat() on an EF IQueryable<T> will turn the entire concatenation into a single query.
When you call .Concat(sharedViews), you're passing a scalar (pre-loaded) collection of your nested entity class.
EF doesn't know how to convert that into a query, so it complains.
You can make it faster by calling AsEnumerable() instead of ToList().

This actually works. Points to the person who can tell me why!
commonViews.ToList().Concat(ownedViews.ToList()).Concat(sharedViews.ToList()).ToList();
That's because each of the original queries is executed separately; you're only concatenating the results in memory. There seems to be a bug in the Entity Framework query translator when you combine the 3 queries, but when you call ToList on each of them, they're no longer EF queries, they're just lists, so they're concatenated using Linq to Objects.

Why doesn't nhibernate LINQ support simple outer joins (given a where clause on the outer joined table)?

i have tableA and tableB
tableA has a B_Id property.
This query works fine:
IEnumerable<A> a = Session.Query<A>().Fetch(r=>r.B);
but i want something like:
IEnumerable<A> a = Session.Query<A>().Where(r=>r.B.Active).Fetch(r=>r.B);
it seems like when i do this, it no longer does a simple outer join and if there are no active records in table B, i get no results.
I want to do a pure outerjoin that will still give me results but with the B property of the A entity as null.
Does nhibernate LINQ support this ability to put a where clause on your outerjoined table??

According to this post on the nhusers group it's apparently not supported, and you'll have to use HQL.
As to the why, I would speculate that the authors plan to add it but had higher priorities. Personally I'd see it as a key feature.

Guess: .Where(a => a.B == null || a.B.Active)
A thought: You're trying to select instances of A which will be in an unknown state. How should NHibernate differ between instances of A where B == null and B.Active == false?
Both will, if it worked, return null values for B and A would be populated with a null reference. What should happen if you save A again? Should the reference to B be set to null? That would remove references when B.Active == false. Should it be untouched? That would make NHibernate store a state which doesn't represent your domain object.

Is it possible to have auto filters in Linq to SQL?

I am working on a system that the client decided to use status for the records. One of them is X for excluded. What I want to know is if it is possible to run linq queries that adds something like
where status != 'X'
Automatically to not show "excluded" records. Thanks !

Sort of. Queries in Linq are lazily-evaluated, so you can append conditions to them as you like before actually fetching the first result and it'll still result in "optimal" SQL being used.
For example:
// an extension method on the LINQ context:
public static IQueryable<Story> FilteredStories(this DbContext db)
{
return from story in db.Stories where status != "X" select story;
}
// ...later...
var stories = from story in db.FilteredStories()
where title = "Something"
select story;
foreach(var story in stories)
{
// whatever...
}
You could also "hide" the underlying LINQ context and always go through a wrapper class that appends the status != "X" condition. Of course, then problem with that is then you'd have to jump through hoops if you didn't want a filtered list...

What you might be after is Dynamic LINQ. It's a helper library that you can use to parse LINQ expressions from strings.

LINQ is so easy that writing the question practically gives you the answer!
where status != "X"
My preference would be
.Where(record => record.status != "X")
which does exactly the same thing. The only thing you missed is double quotes because it's a string, not a char.

SqlException about UNION, INTERSECT and EXCEPT

Could someone help me with this exception? I don't understand what it means or how to fix it... It is an SqlException with the following message:
All queries combined using a UNION, INTERSECT or EXCEPT operator must have an equal number of expressions in their target lists.
I get it when running a query in pseudo code looking like this:
// Some filtering of data
var query = data.Subjects
.Where(has value)
.Where(has other value among some set of values);
// More filtering, where I need to have two different options
var a = query
.Where(some foreign key is null);
var b = query
.Where(some foreign key is not null)
.Where(and that foreign key has a property which is what I want);
query = a.Union(b);
// Final filter and then get result as a list
var list = query
.Where(last requirement)
.ToList();
If I remove the a.Union(b) parts, it runs without the exception. So I know the error is there. But why do I get it? And how can I fix it? Am I doing something too crazy here? Have I misunderstood how to use the Union thing?
Basically what I have is some entities which have a foreign key to some other entity. And I need to get all the entities which either have that foreign key set to null or where that foreign entity fulfills some requirements.

Judging from the SQL error you listed you may be experiencing the same issue I was. Basically when Linq to SQL queries that use the Concat or Union extension method on two different queries it appears that there is a bug in Linq to SQL which optimizes each projection separately without regard to the fact that the projection must stay the same in order to accomplish the SQL Union.
References:
LINQ to SQL produces incorrect TSQL when using UNION or CONCAT
Linq to SQL Union Same Fieldname generating Error
If this happens to be your problem as well I've found a solution that is working for me as shown below.
var queryA =
from a in context.TableA
select new
{
id,
name,
onlyInTableA,
}
var queryB =
from b in context.TableB
let onlyInTableA = default(string)
select new
{
id,
name,
onlyInTableA,
}
var results = queryA.Union(queryB).ToList();

Since this looks like a problem with the generated SQL, you should try to use either an SQL Profiler, or use this code for DebuggerWritter class to write the SQL to your Output Window in Visual Studio.
The SQL error is normally caused by the fields retrieved for UNION is not the same for the 2 queries. For example, if the first query might have 3 fields, but the second query has 4 fields, this error will occur. So, seeing the generated SQL will definitely help in this case.

Can you perhaps write it in a single query?
.Where(row => row.ForeignKey == null || row.ForeignKey.SomeCondition);
There are also ways of merging expressions (OrElse), but that isn't trivial.
Not sure where the error comes from, though!
edit: haven't tested it, but this should be logically equivalent to a UNION:
public static IQueryable<T> WhereAnyOf<T>(
this IQueryable<T> source,
params Expression<Func<T, bool>>[] predicates)
{
if (source == null) throw new ArgumentNullException("source");
if (predicates == null) throw new ArgumentNullException("predicates");
if (predicates.Length == 0) return source.Where(row => false);
if (predicates.Length == 1) return source.Where(predicates[0]);
var param = Expression.Parameter(typeof(T), "row");
Expression body = Expression.Invoke(predicates[0], param);
for (int i = 1; i < predicates.Length; i++)
{
body = Expression.OrElse(body,
Expression.Invoke(predicates[i], param));
}
return source.Where(Expression.Lambda<Func<T, bool>>(body, param));
}

query = a.Union(b);
Not a good idea to mutate captured variables... Likely the cause of the error.
UPDATE: ok not
Here is another idea. The hint is in the error message.
var a = query
.Where(some foreign key is null)
.Select(x => x);
Or play by adding another 'fake' Where till they do become equal :)

I would call data.GetCommand(query) and analyze the resulting DbCommand (especially the generated SQL string). That should give you a clue to what goes wrong.
There is no projection going on anywhere so I would expect both target lists to be the same.
You could try to reduce your query to a smaller one that still doesn't work. Start with query.Union(query) (this should at least work). Than add your Where calls one by one to see when it stops working.
It must be one of your Where calls that adds extra columns to your select list.

Are you by any chance passing in a value to the 'select' side in a variable, or are you returning the same field more than once? SP1 introduced a bug where it tries to 'optimize' out such things and that can cause union queries to break (due to the query parts 'optimizing' out different passed-in params).
If you post your actual query rather than pseudo code it makes it easier to identify if this is the case.
(And a workaround if this is the case is to materialize the individual parts first and then do a client-side (L2O) union).

jpierson has the problem summarised correctly.
I also had the problem, this time caused by some literals in the select statement:
Dim results = (From t in TestDataContext.Table1 _
Where t.ID = WantedID _
Select t.name, SpecialField = 0, AnotherSpecialField = 0, t.Address).Union _
From t in TestDataContext.Table1 _
Where t.SecondID = WantedSecondID _
Select t.name, SpecialField = 1, AnotherSpecialField = 0, t.Address)
The first sub-query of "SpecialField = 0" and the "AnotherSpecialField = 0" were optimised, resulting in one field instead of two being used in the union, which will obviously fail.
I had to change the first query so that the SpecialField & AnotherSpecialField had different values, much like in the second sub-query.

Well I had an issue with this. Using Sql 08 i had two table functions that returned an int and a string in both cases. I created a complex object and used linq to attempt a UNION. Had an IEqualityComparer to do the comparision. All compiled fine, but crashed with a unsupported overload. Ok, i realised the problem discussed seemed to smack of defered execution. So i get the collections, and place ToList(), then do the UNION and it is all good. Not sure if this is helpful, but it works for me