Develop Reference

EF + LINQ: How do I define and utilize a variable (maybe a Func variable) to define part of a Select statement?

Before try to use a variable (this has sql server do the calculation):
IQueryable<MyEntity> query = _dbSet; // IQueryable<TEntity>
var results = query.Select(m => new MyViewModel
{
MyCalculation = m.Column1 * m.Column2
}).ToList();
What I want to do (dynamically create part of my select statement from a Func variable or some other kind of variable to allow this):
IQueryable<MyEntity> query = _dbSet; // IQueryable<TEntity>
Func<MyEntity, decimal> funcVariableAttempt = m => m.Column1 * m.Column2;
var results = query.Select(m => new MyViewModel
{
MyCalculation = funcVariableAttempt.Invoke(m) // My foolish attempt does not work.
}).ToList();
The error I get when I try what I want (aka my foolish attempt):
LINQ to Entities does not recognize the method 'System.Decimal Invoke(MyProject.Repository.Models.MyEntity)' method, and this method cannot be translated into a store expression.
How do I define and utilize a variable (maybe a Func variable) to define part of a Select statement?

It's a completely valid question that I stumbeled across earlier and found a solution that works well for me.
The problem with your Func<MyEntity, decimal> is that it is a delegate and that the O/R mapper has to have access to the internal expression (the multiplication of two properties in your case). But this information is compiled into the delegate and hidden forever.
If you however start off with a Expression<Func<MyEntity, decimal>> customCalculation, things look more promising as you have the internal logic as an expression tree.
Problem with this is: you cannot invoke an expression. customCalculation(m) doesn't compile.
The compiler would let you write
m => new MyViewModel { MyCalculation = customCalculation.Compile()(m) }
, but this wouldn't be understood by most O/R mappers.
But you see that it at least somehow contains the customCalculation lambda expression and also how it relates to its surrounding expressions.
Getting from here to the expression tree as in your original working version involves some expression manipulation:
We have to replace customCalculation.Compile()(m) with the body of the lambda that is to be Compile()d, but with the lambda's parameter(s) replaced with the respective expression(s) of the delegate invocation. So if customCalculation were x => x.Column1 * x.Column2, customCalculation.Compile()(m) would have to be replaced with m.Column1 * m.Column2
Doing so is not trivial, since the lambda itself has to be dug out of a field inside an instance of a compiler generated closure class.
I've posted my implementation of this expression manipulator in another similar question . Hope that helps.
With that, you should be able to:
var customCalculation = (Expression<Func<MyEntity, decimal>>)(x => x.Column1 * x.Column2);
var selector = Express.Prepare((Expression<Func<MyEntity, MyViewModel>>)(m => new MyViewModel { MyCalculation = customCalculation.Compile()(m) }));
var result = query.Select(selector).ToList();

As you already know, your funcVariableAttempt makes no sense to your database, so you have to call your method in the linq-to-object context. i.e. for instance first fetch the data as an Enumerable, then call your method:
var results = query.Select(m => new {
Column1= col1,
Column2= col2
}).AsEnumerable()
.Select(m => new MyViewModel
{
MyCalculation = Foo(m.Column1, m.Column2)
});
*Note: Code is not tested.

You should call ToList() first and perform the Select() on the result in memory.
var results = query.ToList()
.Select(m => new MyViewModel {
MyCalculation = Foo(m.Column1, m.Column2)
});
You're trying to perform the Select as part of the query. You should just use a regular function for the mapping calculation. Lambda functions are useful with LINQ but in this case they're not needed.

Expression tree with linq expressions

I have been playing with expression trees this week and I am wondering why this expression produces error when ran.
var pe = Expression.Parameter(typeof(Nullable<DateTime>));
var ex = Expression.Lambda<Func<DateTime?, bool>>(
(Expression<Func<DateTime?, bool>>) (x => x.HasValue), pe);
The idea behind this is to write expression trees with a mix of expression tree api and linq expressions. It would make things easier to write for example instead of calling Expression.Property(...,..) I would just have x => x.Prop, right?
In my example instead of this Expression.Property(..hasvalue..) I would have this: x.HasValue. It would save me time on writing and it would look shorter, right?
The question is, is this possible?
I guess I might be missing something about
Expression<Func<DateTime?, bool>> foo = x => x.HasValue (this works)
and
Func<DateTime?, bool> bar = x => x.HasValue (this works too)
What is happening behind those two? Are they the same?
Can linq expression be mixed with standard expression tree api???
Please enlighten me on this, I feel lost. :)

This is a good question. Your two quotations
Expression<Func<DateTime?, bool>> foo = x => x.HasValue
and
Func<DateTime?, bool> bar = x => x.HasValue
are examples of homoiconicity: the same symbol (in your case x => x.HasValue) stands for two very different objects. In the first case, it indicates an expression tree; in the second, a function. The former can be compiled down to the latter, but they are different types with different purposes. It is the declaration in your case that tells the compiler which version to go for. In the absence of this context, the compiler cannot read your mind and decides to bail out instead. That's why this won't compile:
var bat = x => x.HasValue;
And that is why your statement won't compile.
Homoiconicity is what makes IQueryable and IEnumerable look so similar. When you invoke
var filteredCollection = myCollection.Where(e => e.IsActive);
you are actually calling methods with a different signature depending on the type of filteredCollection (It's Func<MyClass, bool> for IEnumerable and Expression<Func<MyClass, bool>> for IQueryable).
Regarding your specific situation, you can't achieve what you want to do directly, but if you write a sneaky extension method:
public static class ExpressionExtensions
{
public static Expression<Func<T, TProperty>> Lambda<T, TProperty>(this ParameterExpression pe, Expression<Func<T, TProperty>> property)
{
return Expression.Lambda<Func<T, TProperty>>(property, pe);
}
}
then you can do this:
var pe = Expression.Parameter(typeof(DateTime?));
var ex = pe.Lambda<DateTime?, bool>(x => x.HasValue);

NHibernate 3 LINQ - how to create a valid parameter for Average()

Say I have a very simple entity like this:
public class TestGuy
{
public virtual long Id {get;set;}
public virtual string City {get;set;}
public virtual int InterestingValue {get;set;}
public virtual int OtherValue {get;set;}
}
This contrived example object is mapped with NHibernate (using Fluent) and works fine.
Time to do some reporting. In this example, "testGuys" is an IQueryable with some criteria already applied.
var byCity = testGuys
.GroupBy(c => c.City)
.Select(g => new { City = g.Key, Avg = g.Average(tg => tg.InterestingValue) });
This works just fine. In NHibernate Profiler I can see the correct SQL being generated, and the results are as expected.
Inspired by my success, I want to make it more flexible. I want to make it configurable so that the user can get the average of OtherValue as well as InterestingValue. Shouldn't be too hard, the argument to Average() seems to be a Func (since the values are ints in this case). Easy peasy. Can't I just create a method that returns a Func based on some condition and use that as an argument?
var fieldToAverageBy = GetAverageField(SomeEnum.Other);
private Func<TestGuy,int> GetAverageField(SomeEnum someCondition)
{
switch(someCondition)
{
case SomeEnum.Interesting:
return tg => tg.InterestingValue;
case SomeEnum.Other:
return tg => tg.OtherValue;
}
throw new InvalidOperationException("Not in my example!");
}
And then, elsewhere, I could just do this:
var byCity = testGuys
.GroupBy(c => c.City)
.Select(g => new { City = g.Key, Avg = g.Average(fieldToAverageBy) });
Well, I thought I could do that. However, when I do enumerate this, NHibernate throws a fit:
Object of type 'System.Linq.Expressions.ConstantExpression' cannot be converted to type 'System.Linq.Expressions.LambdaExpression'.
So I am guessing that behind the scenes, some conversion or casting or some such thing is going on that in the first case accepts my lambda, but in the second case makes into something NHibernate can't convert to SQL.
My question is hopefully simple - how can my GetAverageField function return something that will work as a parameter to Average() when NHibernate 3.0 LINQ support (the .Query() method) translates this to SQL?
Any suggestions welcome, thanks!
EDIT
Based on the comments from David B in his answer, I took a closer look at this. My assumption that Func would be the right return type was based on the intellisense I got for the Average() method. It seems to be based on the Enumerable type, not the Queryable one. That's strange.. Need to look a bit closer at stuff.
The GroupBy method has the following return signature:
IQueryable<IGrouping<string,TestGuy>>
That means it should give me an IQueryable, all right. However, I then move on to the next line:
.Select(g => new { City = g.Key, Avg = g.Average(tg => tg.InterestingValue) });
If I check the intellisense for the g variable inside the new { } object definition, it is actually listed as being of type IGrouping - NOT IQueryable>. This is why the Average() method called is the Enumerable one, and why it won't accept the Expression parameter suggested by David B.
So somehow my group value has apparently lost it's status as an IQueryable somewhere.
Slightly interesting note:
I can change the Select to the following:
.Select(g => new { City = g.Key, Avg = g.AsQueryable<TestGuy>().Average(fieldToAverageBy) });
And now it compiles! Black magic! However, that doesn't solve the issue, as NHibernate now doesn't love me anymore and gives the following exception:
Could not parse expression '[-1].AsQueryable()': This overload of the method 'System.Linq.Queryable.AsQueryable' is currently not supported, but you can register your own parser if needed.
What baffles me is that this works when I give the lambda expression to the Average() method, but that I can't find a simple way to represent the same expression as an argument. I am obviously doing something wrong, but can't see what...!?
I am at my wits end. Help me, Jon Skeet, you're my only hope! ;)

You won't be able to call a "local" method within your lambda expression. If this were a simple non-nested clause, it would be relatively simple - you'd just need to change this:
private Func<TestGuy,int> GetAverageField(SomeEnum someCondition)
to this:
private Expression<Func<TestGuy,int>> GetAverageField(SomeEnum someCondition)
and then pass the result of the call into the relevant query method, e.g.
var results = query.Select(GetAverageField(fieldToAverageBy));
In this case, however, you'll need to build the whole expression tree up for the Select clause - the anonymous type creation expression, the extraction of the Key, and the extraction of the average field part. It's not going to be fun, to be honest. In particular, by the time you've built up your expression tree, that's not going to be statically typed in the same way as a normal query expression would be, due to the inability to express the anonymous type in a declaration.
If you're using .NET 4, dynamic typing may help you, although you'd pay the price of not having static typing any more, of course.
One option (horrible though it may be) would be try to use a sort of "template" of the anonymous type projection expression tree (e.g. always using a single property), and then build a copy of that expression tree, inserting the right expression instead. Again, it's not going to be fun.
Marc Gravell may be able to help more on this - it does sound like the kind of thing which should be possible, but I'm at a loss as to how to do it elegantly at the moment.

Eh? the parameter to Queryable.Average is not Func<T, U>. It's Expression<Func<T, U>>
The way to do this is:
private Expression<Func<TestGuy,int>> GetAverageExpr(SomeEnum someCondition)
{
switch(someCondition)
{
case SomeEnum.Interesting:
return tg => tg.InterestingValue;
case SomeEnum.Other:
return tg => tg.OtherValue;
}
throw new InvalidOperationException("Not in my example!");
}
Followed by:
Expression<Func<TestGuy, int>> averageExpr = GetAverageExpr(someCondition);
var byCity = testGuys
.GroupBy(c => c.City)
.Select(g => new { City = g.Key, Avg = g.Average(averageExpr) });

System.Linq Expressions evaluating true on a single entity

I'm trying to DRY out some lambda expressions for security rights. Is it possible to take a lamda expression and apply it to a single entity for true?
Like lets say I have a Person and a DocumentFolder
Expression<Func<Person, bool>> CanSeePerson()
{
return c => !c.IsPrivate;
}
And one for the folder
Expression<Func<DocumentFolder, bool>> CanSeeFolder()
{
return c => !c.IsPrivate && c.Owner.CanSeePerson(); // <- ???
}
How the heck can I use that CanSeePerson() function on a single type to return true and maintain an expression that can be used in linq queries like such
Entities.DocumentFolder.Where(CanSeeFolder());
I know how to use the where on an iqueryable but I can't see how to apply the expression tree to a single value.
This throws the error: Unable to create a constant value of type 'Person'. Only primitive types ('such as Int32, String, and Guid') are supported in this context.
Expression<Func<DocumentFolder, bool>> CanSeeFolder()
{
return c => !c.IsPrivate &&_entities.Persons.Where(x => x.Id == c.Owner.Id).Any(CanSeePerson());
}
The difference seems to be based on putting the IQueryable Directly in the statement. This also does NOT work
Expression<Func<DocumentFolder, bool>> CanSeeFolder()
{
return c => !c.IsPrivate &&_entities.Persons.Where(CanSeePerson()).Contains(c.User);
}
but this DOES work
Expression<Func<DocumentFolder, bool>> CanSeeFolder()
{
var canSeePersons = _entities.Persons.Where(CanSeePerson());
return c => !c.IsPrivate && canSeePersons.Contains(c.User);
}
p.s. I know I suck # using this stackoverflow formatting thing lol
The reason why this works is because the CanSeePerson() function cannot be converted and used in an expression. When you put the canSeePersons variable in the function instead you are placing in an iQueryable type which can be used in an expression. Using "var" convolutes it a bit.

You defined a Expression<Func<Person, bool>> that solves your problem. The trick is to simply transform your list of objects to the list you want to filter. It is not always possible, but many times it is. You just have to be a bit creative :-)
When using extension methods you can easily come up with a solution that allows you to do this:
var visibleFolders = Entities.DocumentFolder.WhereCanSeeFolder();
Here is the (completely DRY) solution:
public static class SecurityExtensions
{
public static IQueryable<DocumentFolder> WhereCanSeeFolder(
this IQueryable<DocumentFolder> folders)
{
var visibleOwners = folders.Select(f => f.Owner)
.Where(CanSeePerson);
return
from folder in folders.Where(CanSeeFolder)
where visibleOwners.Contains(folder.Owner)
select folder;
}
private static readonly Expression<Func<DocumentFolder, bool>>
CanSeeFolder = folder => !folder.IsPrivate;
private static readonly Expression<Func<Person, bool>>
CanSeePerson = person => !person.IsPrivate;
}
I hope this helps.

The expression tree is only applied to a single value at a time, logically, within the LINQ expression.
If you're saying you want to apply it in-process later to a single value, you can just use:
// This can be cached
Func<DocumentFolder, bool> canSeeFolderDelegate = CanSeeFolder().Compile();
DocumentFolder folder = ...; // Get the value from somewhere
if (canSeeFolderDelegate(folder))
{
// Yes, you can see that folder
}

Ah, you want to evaluate it for a single entity?
var func = CanSeeFolder().Compile(); // <=== store and re-use this;
// this isn't free
bool canSee = func(obj);
Another approach might be:
bool canSee = Enumerable.Repeat(obj, 1).AsQueryable().Any(CanSeeFolder());
but this is probably still going to do the Compile somewhere in the chain, so you may as well use the more direct code (at the top).
Edit re comments:
To evaluate that at the database, you would need a restriction, for example:
bool canSee = db.Folders.Where(f => f.FolderId == id)
.Where(CanSeeFolder()).Any();
which limits us to the single row, then adds your extra filter.

Linq2SQL "or/and" operators (ANDed / ORed conditions)

Let's say we need to apply several conditions to select from a table called "Things" (unknown count and nature)
if conditions are known, we can write
db.Things.Where(t=>foo1 && foo2 || foo3);
but if we have to build that Where condition programatically, I can imagine how can we apply ANDed conditions
IQuerable DesiredThings = db.Things.AsQuerable();
foreach (Condition c in AndedConditions)
DesiredThings = DesiredThings.Where(t => GenerateCondition(c,t));
What about ORed conditions ?
Note: we don't want to perform union, unique, or any other costly operations, it's desired that a query is generated as if we write it ad-hock
Thanks in advance.
Addition:
PredicateBuilder: Dynamically Composing Expression Predicates

You could use the Expression class with static methods to do it run time.
The below code is ment to create a delegate taking one argument called value of type int
. It reads from buttom to top so the line in question is:
var method = LambdaExpression.Lambda(orExp, Expression.Parameter(typeof(int), "value"));
the body of the method compares the value of the parameter to a call to method Bar of a newly created object of type foo
var exp2 = Expression.Equal(Expression.Parameter(typeof(int), "value"), Expression.Property(Expression.New(typeof(Foo).GetConstructor(new Type[] { })), "Bar"));
It then creates a similar expression and or's them
var orExp = Expression.OrElse(exp1, exp2);
final thing is the call to compile. That call generates a delegate that can be used in your where method call.
hope it helps tho Im not 100% sure on the expression to get the value from a parameter
var exp1 = Expression.Equal(Expression.Parameter(typeof(int),"value"), Expression.Property(Expression.New(typeof(Bar).GetConstructor(new Type[] { })), "Foo"));
var exp2 = Expression.Equal(Expression.Parameter(typeof(int), "value"), Expression.Property(Expression.New(typeof(Foo).GetConstructor(new Type[] { })), "Bar"));
var orExp = Expression.OrElse(exp1, exp2);
var method = LambdaExpression.Lambda(orExp, Expression.Parameter(typeof(int), "value"));
method.Compile();
You might wanna look at invoke for invokation instead of compiling the expression, if you need the LambdaExpression to be translated into something different than binary code (E.g. into an SQL statement)

For OR, you have two choices:
use Union/Concat
write the Expression in code
The second is closer to the .Where(x => {a} || {b}).
If you are using LINQ-to-SQL, you can use Expression.Invoke to combine multiple separate lambda expressions (see this answer) - however, this isn't supported in Entity Framework. In EF, you have to build the entire expression as a single block, using Expression.OrElse; for example here or here.