I've been trying to learn more about using Lamba expression trees and so I created a simple example. Here is the code, this works in LINQPad if pasted in as a C# program.
void Main()
{
IEnumerable<User> list = GetUsers().Where(NameContains("a"));
list.Dump("Users");
}
// Methods
public IEnumerable<User> GetUsers()
{
yield return new User{Name = "andrew"};
yield return new User{Name = "rob"};
yield return new User{Name = "chris"};
yield return new User{Name = "ryan"};
}
public Expression<Func<User, bool>> NameContains(string namePart)
{
return u => u.Name.Contains(namePart);
}
// Classes
public class User
{
public string Name { get; set; }
}
This results in the following error:
The type arguments for method 'System.Linq.Enumerable.Where(System.Collections.Generic.IEnumerable, System.Func)' cannot be inferred from the usage. Try specifying the type arguments explicitly.
However if I just substitute the first line in main with this:
IEnumerable<User> list = GetUsers().Where(u => u.Name.Contains("a"));
It works fine. Can tell me what I'm doing wrong, please?
The Enumerable.Where method takes a Func<T, bool>, not an Expression<Func<T, bool>>. Perhaps you're confusing with Queryable.Where, which does take an expression as a parameter... In your case you don't need an expression, you just need a delegate that can be executed against each item in the sequence. The purpose of expressions is (mostly) to be analysed and translated to something else (SQL for instance), to perform the query against an external data source
Change the return type of NameContains from Expression<Func<User, Bool>> to simply Func<User, Bool>. In this situation, there's no need to return the Expression, you actually want to return the compiled delegate. There's a difference between the expression that makes up the lambda, and the lambda (which is a delegate) itself.
If you send a lambda into a method, the method can accept the lambda either as an expression, or as a compiled delegate type, depending on what you specify in the parameters. If the incoming parameter type is an Expression, you can send in something that looks like a delegate, however, if the method is expecting a delegate, you have to give it a compiled delegate, not simply an expression. That being said, you can also do something like:
var certainUsers = GetUsers().Where(NameContains("a").Compile());
Which would compile the expression, and return a Func<User, Bool>.
Lambda expressions can be treated as either code (delegates) or as data (expression trees)
In your example your are attempting to treat the lambda expression as code.
You would use the Expression<> declaration when you want to treat the lambda expression as data.
Why would you want to do this?
Here is a quote from the Book Linq In Action,
" Expression trees can be given to tools at runtime, which use them to guide
their execution or translate them into something else, such as SQL in the case of
LINQ to SQL."
Using Expression Trees allows you to take the lambda expression and convert it to data, this is how Linq to SQL works, it takes the lambda expression or query operators or query expressions and converts them to SQL. You of course can view and modify the created expression tree once converted to sql.
There is a huge difference between Expression and Func<...>, the Func is a pure delegate you can invoke it directly, the expression is a data structure holds information about an expression like information about lambda expression or Linq Syntax (e.g. From x in list where x.Id = 1 select x). The expression cannot be invoked directly it must be compiled first, Expressions is used to convert the expression from a way to another like Link To Sql which converts an expression to Sql statements, the best way to do this to change the return type of the NameContains Method to Func insted of expression cuz you are working with Linq To Objects, but when using with Linq To Sql you can use both Expression or func.
Related
I am trying to achieve the following pattern:
Main()
{
...
GetFilteredUsersById(u => u >= 100 && u < 200).Dump();
....
}
public List<UserRecord> FilteredUsersById(Func<int, bool> filter)
{
Expression<Func<UserRecord, bool>> filterExpression = x => filter(x.Id);
return someUserRecordsDb.Where(filterExpression).ToList();
}
public class UserRecord
{
public int Id;
public string Name;
}
Of course, this fails with
NotSupportedException: Method 'System.Object DynamicInvoke(System.Object[])' has no supported translation to SQL.
I understand why the error is happening, but don't know how to fix it.
Below is a pattern that does work, but I'm interested in understanding what changes I need to make to the code to the first pattern above to make it work.
Main()
{
...
GetFilteredUsersById(u => u.Id >= 100 && u.Id < 200).Dump();
....
}
public List<UserRecord> FilteredUsers(Expression<Func<UserRecord, bool>> filter)
{
return someUserRecordsDb.Where(filter).ToList();
}
public class UserRecord
{
public int Id;
public string Name;
}
Thanks for answering my first question here at StackOverflow!
As already you know, LINQ to SQL works by taking your lambdas as expression trees instead of delegates. Then, it analyzes those trees and attempts to produce equivalent SQL code. Obviously, it can't support everything you can possibly write in C#.
In your second snippet, the LINQ provider sees an expression tree that contains this:
u => u.Id >= 100 && u.Id < 200
This is very easy to understand - it involves the && operator, the >= operator and the < operator, all of which are known.
But in your first snippet, the LINQ provider sees an expression tree that contains this instead:
x => filter(x.Id)
This is very different, because it's an invocation of an arbitrary method that's not known to the provider. So now the provider has to look inside that method and see what it contains. If that method had another level of indirection, the provider would have to look even further. And so on.
If your expression tree requires that kind of work, the provider will refuse to do it. So any invocations to arbitrary methods will understandably fail.
Thus, the answer is simple: don't put a level of indirection between the location where you create the expression tree and the actual expression you want translated to SQL, otherwise the LINQ provider will reject it altogether.
There are two things to keep in mind here
1. Expression vs Func
One of the main things you need to understand is the difference between an Expression and a Func or any type of delegate type. In short the expression is passing a tree representation of the lamda, which can be translated into Sql.
There is a great answer that covers this in depth here :
Why would you use Expression<Func<T>> rather than Func<T>?
2. Not all Expressions will Work with Linq-to-Sql
Even if a function can be represented as an expression it does not mean that there will be a proper mapping from the operations being preformed in the lambda expression for the provider that is consuming your linq expression.
This is why you are getting the exception :
NotSupportedException: Method 'System.Object DynamicInvoke(System.Object[])' has no supported translation to SQL.
Here even though it is a valid expression there is no way to convert the operation to sql which is why you get an exception, even though filter would be a valid lambda if passed as an expression and called directly the translation of the statement is that it is trying to call this func filter, which doesn't map directly to anything in sql
I have a Lazy<T> initialized with a lambda. How to see the body of the initializing lambda while debugging? I expect to have something like the DebugView of the Expression class but I've found nothing like that.
Because Lazy<T> takes a delegate, there is no Expression class involved. Your lambda is compiled like any other code in your project and there is no preview of that code during debug.
Lambda expression can be compiled either into IL or transformed into Expression Tree. Which one happens depends on the context. If your parameter is declared as delegate regular IL code will be generated. If it's Expression<TFunc> you'll get expression tree which can be previewed.
It's nicely explained on MSDN, based on Where method, which has two versions: Enumerable.Where which takes Func<T, bool> and Queryable.Where which takes Expression<Func<T, bool>>.
When you use method-based syntax to call the Where method in the
Enumerable class (as you do in LINQ to Objects and LINQ to XML) the
parameter is a delegate type System.Func<T, TResult>. A lambda
expression is the most convenient way to create that delegate. When
you call the same method in, for example, the System.Linq.Queryable
class (as you do in LINQ to SQL) then the parameter type is an
System.Linq.Expressions.Expression<Func> where Func is any of the Func
delegates with up to sixteen input parameters. Again, a lambda
expression is just a very concise way to construct that expression
tree. The lambdas allow the Where calls to look similar although in
fact the type of object created from the lambda is different.
I have the following code;
IQueryable<MyClass> query = listOfObjects.Where(x => x.SomeProp == 1);
I pass this to a method on a particular API that is expecting an IQueryable, which is fine.
However, I want to dynamically build up the predicate, so I'm using Expression.Lambda to achieve this, and I then .Compile it to turn it back into a Func<MyObject, bool>.
I would have expected that the following would have worked;
Func<MyClass, bool> predicate = x => GetPredicate();
IQueryable<MyClass> query = list.Fields.Where(predicate);
However, passing predicate to Where has changed the return type to IEnumerable<MyClass>, which isn't the type required by the API obviously.
I did (naively) try predicate.AsQueryable(), but the API in question (SharePoint Client Object model) just fails with a generic "Specified method is not supported." error message.
I don't know if this a limitation of the LINQ provider that is behind the scenes, but regardless... I'm keen to understand why pulling the Func out into its own variable and passing it in to Where affects the type inference in the way it does.
IQueryable is using an expression tree to build predicate. So, instead of
Func<MyClass, bool> predicate = x => GetPredicate();
use:
Expression<Func<MyClass, bool>> predicate = x => GetPredicate();
Keep in mind:
While using IQueryable expression tree is built (tree that represents operation (as operands and arguments) made on collection). In order to translate tree into other form (let's say sql query, depends on LINQ proider) translator must know all operands used in to tree. It looks like that translator in service where you are passing IQueryable don't know what does GetPredicate function do (and don't know how to translate it to sql query) so throws Not Supported Exception..
The same thing is with Func instead of Expression. Func is complied version of predicate (stored as delegate) - provider don't know how to translate delegates. When Expression is used, the predicate is stored as tree, so provider can "look inside" an expression and translate it correctly.
I'm wondering what exactly is the difference between wrapping a delegate inside Expression<> and not ?
I'm seeing Expression<Foo> being used a lot with LinQ, but so far I've not found any article that explains the difference between this, and just using a delegate.
E.g.
Func<int, bool> Is42 = (value) => value == 42;
vs.
Expression<Func<int, bool>> Is42 = (value) => value == 42;
tl;dr, To have an expression is like having the source code of an application, and a delegate is an executable to run the application. An expression can be thought of as the "source" (i.e., syntax tree) of the code that would run. A delegate is a specific compilation that you would run and does the thing.
By storing a lambda as a delegate, you are storing a specific instance of a delegate that does some action. It can't be modified, you just call it. Once you have your delegate, you have limited options in inspecting what it does and whatnot.
By storing a lambda as an expression, you are storing an expression tree that represents the delegate. It can be manipulated to do other things like changing its parameters, changing the body and make it do something radically different. It could even be compiled back to a delegate so you may call it if you wish. You can easily inspect the expression to see what its parameters are, what it does and how it does it. This is something that a query provider can use to understand and translate an expression to another language (such as write an SQL query for a corresponding expression tree).
It is also a whole lot easier to create a delegate dynamically using expressions than it is emitting the code. You can think of your code at a higher level as expressions that is very similar to how a compiler views code instead of going low-level and view your code as IL instructions.
So with an expression, you are capable to do much more than a simple anonymous delegate. Though it's not really free, performance will take a hit if you run compiled expressions compared to a regular method or an anonymous delegate. But that might not be an issue as the other benefits to using expressions may be important to you.
Func<> is just a delegate type. An Expression is a runtime representation of the complete tree of operations which, optionally, may be compiled at runtime into a delegate. It's this tree that is parsed by Expression parsers like Linq-to-SQL to generate SQL statements or do other clever things. When you assign a lambda to an Expression type, the compiler generates this expression tree as well as the usual IL code. More on expression trees.
To illustrate other answers, if you compile those 2 expressions and have look at the compiler generated code, this i what you will see:
Func<int, bool> Is42 = (value) => value == 42;
Func<int, bool> Is42 = new Func<int, bool>((#value) => value == 42);
Expression<Func<int, bool>> Is42 = (value) => value == 42;
ParameterExpression[] parameterExpressionArray;
ParameterExpression parameterExpression = Expression.Parameter(typeof(int), "value");
Expression<Func<int, bool>> Is42 = Expression.Lambda<Func<int, bool>>(Expression.Equal(parameterExpression, Expression.Constant(42, typeof(int))), new ParameterExpression[] { parameterExpression });
Expression Trees allow you to inspect the code inside the expression, in your code.
For example, if you passed this expression: o => o.Name, your code could find out that the Name property was being accessed inside the expression.
Provides the base class from which the classes that represent
expression tree nodes are derived.
System.Linq.Expressions.BinaryExpression
System.Linq.Expressions.BlockExpression
System.Linq.Expressions.ConditionalExpression
System.Linq.Expressions.ConstantExpression
System.Linq.Expressions.DebugInfoExpression
System.Linq.Expressions.DefaultExpression
System.Linq.Expressions.DynamicExpression
System.Linq.Expressions.GotoExpression
System.Linq.Expressions.IndexExpression
System.Linq.Expressions.InvocationExpression
System.Linq.Expressions.LabelExpression
System.Linq.Expressions.LambdaExpression
System.Linq.Expressions.ListInitExpression
System.Linq.Expressions.LoopExpression
System.Linq.Expressions.MemberExpression
System.Linq.Expressions.MemberInitExpression
System.Linq.Expressions.MethodCallExpression
System.Linq.Expressions.NewArrayExpression
System.Linq.Expressions.NewExpression
System.Linq.Expressions.ParameterExpression
System.Linq.Expressions.RuntimeVariablesExpression
System.Linq.Expressions.SwitchExpression
System.Linq.Expressions.TryExpression
System.Linq.Expressions.TypeBinaryExpression
System.Linq.Expressions.UnaryExpression
http://msdn.microsoft.com/en-us/library/system.linq.expressions.expression.aspx
Expression tree represents linq expression that can be analyzed and for example turned into SQL query.
To whatever the other wrote (that is completely correct) I'll add that through the Expression class you can create new methods at runtime. There are some limits. Not all the things you can do in C# can be done in an Expression tree (at least in .NET 3.5 . With .NET 4.0 they have added a great number of possible Expression "types"). The use of this could be (for example) to create a dynamic query and pass it to LINQ-to-SQL or do some filtering based on the input of the user... (you could always do this with CodeDom if all you wanted was a dynamic method incompatible with LINQ-to-SQL, but emitting directly IL code is quite difficult :-) )
Both snippets below product the same output.
I understand how Func encapsulates a method with a single parameter, and returns a bool value. And you can either assign it a
method, anonymous method or a lambda expression.
Func<int, bool> deleg = i => i < 5;
Console.WriteLine("deleg(4) = {0}", deleg(4));
Below is using expression trees which I don't fully understand yet. Why would I want to do it this way? Is it more flexible, what advantage does it give me?
System.Linq.Expressions.Expression<Func<int, bool>> expr = i => i < 5;
Func<int, bool> deleg2 = expr.Compile();
Console.WriteLine("deleg2(4) = {0}", deleg2(4));
Basically, the Expression tree is the body of a lambda expression, that allows you to
introspect the expression (see what's in it so to say)
manipulate the expression (simplify, extend (e.g. add new functionality or modify to work on different items).
Once you Compile() the expression, it is just another delegate, which you can only call, not inspect or modify.
Whenever you want to
create expressions dynamically (I mean: construct, not allocate)
operate on expressions dynamically
the Function<> types are not sufficient.
The point of expression trees is that you can do more with them than just compile them to a function. You can inspect them, modify them and compile them to something other than .net functions.
For example Linq2SQL compiles expression trees to SQL code. You couldn't do that with a plain .net function.
In your first example you just have "hardcoded" the body of the function and assigned it to a delegate.
In your second example the assignment constructs an expression-tree which is an object model reprensenting your code in a data structure in memory.
The advantage is that you can modify and inspect that datastructure.
LINQ2SQL for example uses that technique to translate your expressions to another language called SQL.
Expression trees are regular in-memory data structures that can be traversed programmatically and the result of such traversal can be something, like a query you'd like to send to the database. Read more on the ExpressionVisitor class to see how it is done.
On the other hand, the compiled function is nothing more than a sequence of CIL code. You still can inspect it programmatically but you are not inspecting the definition but rather - the compiler output of it.