I have occasionally come across statements like this
school.Where(s => s.name == "foo")
Now I understand this is a lambda and I think I am comfortable with lambdas. I just started C# so I do have this question. In the statement
s => s.name == "foo"
how is the evaluation return being done to determine true or false. I do not see a return statement ? All i see is a comparison operation in the lambda body and nothing being returned. Can anyone please explain this to me.
In this form the return is implicit. The lambda returns whatever the expression returns. You could also write it like this:
s => { return s.name == "foo"; }
The reason why the return isn’t needed in the short form is because that’s how the language was designed. It makes things short and nice to use instead of forcing the developer to write unnecessary code.
The return statement is implied. If you were to add brackets around s.name == "foo" then you'd have to have a return. The compiler essentially creates a function to be called there for you. This is just syntax sugar. It's equivalent to creating a function yourself and passing that into the Where
Lambda can be defined in two ways:
with curly braces: then you will have to write body of a method, just like in a regular method, i.e. with returns, etc. Example
var lambda = (string param) => { return param.Length; };
without curly braces: then the method has to contain one line, which also be the result of a method. Example:
var lambda = (string param) => param.Length;
This is by design, to make one-line methods look nice and clean (especially in LINQ extensoin methods) :)
The returned value is always boolean and you can get it like below:
s=> {retutn s.name=="foo";}
I am trying to achieve the following pattern:
Main()
{
...
GetFilteredUsersById(u => u >= 100 && u < 200).Dump();
....
}
public List<UserRecord> FilteredUsersById(Func<int, bool> filter)
{
Expression<Func<UserRecord, bool>> filterExpression = x => filter(x.Id);
return someUserRecordsDb.Where(filterExpression).ToList();
}
public class UserRecord
{
public int Id;
public string Name;
}
Of course, this fails with
NotSupportedException: Method 'System.Object DynamicInvoke(System.Object[])' has no supported translation to SQL.
I understand why the error is happening, but don't know how to fix it.
Below is a pattern that does work, but I'm interested in understanding what changes I need to make to the code to the first pattern above to make it work.
Main()
{
...
GetFilteredUsersById(u => u.Id >= 100 && u.Id < 200).Dump();
....
}
public List<UserRecord> FilteredUsers(Expression<Func<UserRecord, bool>> filter)
{
return someUserRecordsDb.Where(filter).ToList();
}
public class UserRecord
{
public int Id;
public string Name;
}
Thanks for answering my first question here at StackOverflow!
As already you know, LINQ to SQL works by taking your lambdas as expression trees instead of delegates. Then, it analyzes those trees and attempts to produce equivalent SQL code. Obviously, it can't support everything you can possibly write in C#.
In your second snippet, the LINQ provider sees an expression tree that contains this:
u => u.Id >= 100 && u.Id < 200
This is very easy to understand - it involves the && operator, the >= operator and the < operator, all of which are known.
But in your first snippet, the LINQ provider sees an expression tree that contains this instead:
x => filter(x.Id)
This is very different, because it's an invocation of an arbitrary method that's not known to the provider. So now the provider has to look inside that method and see what it contains. If that method had another level of indirection, the provider would have to look even further. And so on.
If your expression tree requires that kind of work, the provider will refuse to do it. So any invocations to arbitrary methods will understandably fail.
Thus, the answer is simple: don't put a level of indirection between the location where you create the expression tree and the actual expression you want translated to SQL, otherwise the LINQ provider will reject it altogether.
There are two things to keep in mind here
1. Expression vs Func
One of the main things you need to understand is the difference between an Expression and a Func or any type of delegate type. In short the expression is passing a tree representation of the lamda, which can be translated into Sql.
There is a great answer that covers this in depth here :
Why would you use Expression<Func<T>> rather than Func<T>?
2. Not all Expressions will Work with Linq-to-Sql
Even if a function can be represented as an expression it does not mean that there will be a proper mapping from the operations being preformed in the lambda expression for the provider that is consuming your linq expression.
This is why you are getting the exception :
NotSupportedException: Method 'System.Object DynamicInvoke(System.Object[])' has no supported translation to SQL.
Here even though it is a valid expression there is no way to convert the operation to sql which is why you get an exception, even though filter would be a valid lambda if passed as an expression and called directly the translation of the statement is that it is trying to call this func filter, which doesn't map directly to anything in sql
In a project I'm currently working on, we have many static Expressions that we have to bring in local scope with a variable when we call the Invoke method on them and pass our lambda expressions' arguments to.
Today, we declared a static method whose parameter is exactly the type that the query is expecting. So, my coworker and I were messing around to see if we could get this method to do the project in the Select statement of our query, instead of invoking it on the whole object, without bringing it into local scope.
And it worked! But we do not understand why.
Imagine code like this
// old way
public static class ManyExpressions {
public static Expression<Func<SomeDataType, bool> UsefulExpression {
get {
// TODO implement more believable lies and logic here
return (sdt) => sdt.someCondition == true && false || true;
}
}
}
public class ARealController : BaseController {
/* many declarations of important things */
public ARealClass( /* many ninjected in things */) {
/* many assignments */
}
public JsonNet<ImportantDataResult> getSomeInfo(/* many useful parameter */) {
var usefulExpression = ManyExpressions.UsefulExpression;
// the db context is all taken care of in BaseController
var result = db.SomeDataType
.Where(sdt => usefulExpression.Invoke(sdt))
.Select(sdt => new { /* grab important things*/ })
.ToList();
return JsonNet(result);
}
}
And then you get to do this!
// new way
public class SomeModelClass {
/* many properties, no constructor, and very few useful methods */
// TODO come up with better fake names
public static SomeModelClass FromDbEntity(DbEntity dbEntity) {
return new SomeModelClass { /* init all properties here*/ };
}
}
public class ARealController : BaseController {
/* many declarations of important things */
public ARealClass( /* many ninjected in things */) {
/* many assignments */
}
public JsonNet<SomeModelClass> getSomeInfo(/* many useful parameter */) {
// the db context is all taken care of in BaseController
var result = db.SomeDataType
.Select(SomeModelClass.FromDbEntity) // TODO; explain this magic
.ToList();
return JsonNet(result);
}
}
So when ReSharper prompts me to do this (which is not often, as this condition of matching the type that is expected by the Delegate isn't often satisfied), it says convert to a Method Group. I kind of vaguely understand that a Method Group is a set of methods, and the C# compiler can take care of converting the method group to an explicitly typed and appropriate overload for the LINQ provider and what not... but I'm fuzzy on why this works exactly.
What's going on here?
It's great to ask a question when you don't understand something, but the problem is that it can be hard to know which bit someone doesn't understand. I hope I help here, rather than tell you a bunch of stuff you know, and not actually answer your question.
Let's go back to the days before Linq, before expressions, before lambda, and before even anonymous delegates.
In .NET 1.0 we didn't have any of those. We didn't even have generics. We did though have delegates. And a delegate is related to a function pointer (if you know C, C++ or languages with such) or function as argument/variable (if you know Javascript or languages with such).
We could define a delegate:
public delegate int MyDelegate(double someValue, double someOtherValue);
And then use it as a type for a field, property, variable, method argument or as the basis of an event.
But at the time the only way to actually give a value for a delegate was to refer to an actual method.
public int CompareDoubles(double x, double y)
{
if (x < y) return -1;
return y < x ? 1 : 0;
}
MyDelegate dele = CompareDoubles;
We can invoke that with dele.Invoke(1.0, 2.0) or the shorthand dele(1.0, 2.0).
Now, because we have overloading in .NET, we can have more than one thing that CompareDoubles refers to. That isn't a problem, because if we also had e.g. public int CompareDoubles(double x, double y, double z){…} the compiler could know that you could only possibly have meant to assign the other CompareDoubles to dele so it's unambiguous. Still, while in the context CompareDoubles means a method that takes two double arguments and returns an int, outside of that context CompareDoubles means the group of all the methods with that name.
Hence, Method Group which is what we call that.
Now, with .NET 2.0 we got generics, which is useful with delegates, and at the same time in C#2 we got anonymous methods, which is also useful. As of 2.0 we could now do:
MyDelegate dele = delegate (double x, double y)
{
if (x < y) return -1;
return y < x ? 1 : 0;
};
This part was just syntactic sugar from C#2, and behind the scenes there's still a method there, though it has an "unspeakable name" (a name that is valid as a .NET name but not valid as a C# name, so C# names can't clash with it). It was handy if, as was often the case, one was creating methods just to have them used once with a particular delegate though.
Move forward a bit further, and at .NET 3.5 have covariance and contravariance (great with delegates) the Func and Action delegates (great for reusing the same name based on type, rather than having a bunch of different delegates which were often very similar) and along with it came C#3 which had lambda expressions.
Now, these are a bit like anonymous methods in one use, but not in another.
That's why we can't do:
var func = (int i) => i * 2;
var works out what it means from what's been assigned to it, but lamdas work out what they are from what they've been assigned to, so this is ambiguous.
It could mean:
Func<int, int> func = i => i * 2;
In which case it's shorthand for:
Func<int, int> func = delegate(int i){return i * 2;};
Which in turn is shorthand something like for:
int <>SomeNameImpossibleInC# (int i)
{
return i * 2;
}
Func<int, int> func = <>SomeNameImpossibleInC#;
But it can also be used as:
Expression<Func<int, int>> func = i => i * 2;
Which is shorthand for:
Expression<Func<int, int>> func = Expression.Lambda<Func<int, int>>(
Expression.Multiply(
param,
Expression.Constant(2)
),
param
);
And we also with .NET 3.5 have Linq which makes heavy use of both of these. Indeed, Expressions is considered part of Linq and is in the System.Linq.Expressions namespace. Note that the object we get here is a description of what we want done (take the parameter, multiply it by two, give us the result) not of how to do it.
Now, Linq operates in two main ways. On IQueryable and IQueryable<T> and on IEnumerable and IEnumerable<T>. The former defines operations to be used on "a provider" with just what "a provider does" being up to that provider, and the latter defines the same operations on in-memory sequences of values.
We can move from one to the other. We can turn an IEnumerable<T> into an IQueryable<T> with AsQueryable which will give us a wrapper on that enumerable, and we can turn the IQueryable<T> into an IEnumerable<T> just by treating it as one, because IQueryable<T> derives from IEnumerable<T>.
The enumerable form uses the delegates. A simplified version of how Select works (there are many optimisations this version leaves out, and I'm skipping error checking and in indirection to ensure that error checking happens immediately) would be:
public static IEnumerable<TResult> Select(this IEnumerable<TSource> source, Func<TSource, TResult> selector)
{
foreach(TSource item in source) yield return selector(item);
}
The queryable version on the other hand works by taking the expression tree from the Expression<TSource, TResult> making it part of an expression that includes the call to Select, and the source queryable, and returns an object wrapping that expression. So in other words a call to queryable's Select returns an object that represents a call to queryable's Select!
Just what is done with that depends on the provider. Database providers turn them into SQL, enumerables call Compile() on the expression to create a delegate and then we're back at the first version of Select above, and so on.
But that history considered, let's go backwards through the history again. A lambda can represent either an expression or a delegate (and if an expression, we can Compile() it to get the same delegate). A delegate is a way of pointing to a method through a variable, and a method is part of a method group. All of this is built on technology which in the first version could only be called by creating a method and then passing that.
Now, lets say we have a method that takes a single argument and has a result.
public string IntString(int num) { return num.ToString(); }
Now lets say we referenced it in a lambda selector:
Enumerable.Range(0, 10).Select(i => IntString(i));
We have a lambda creating an anonymous method for a delegate, and that anonymous method in turn calls a method with the same argument and return types. In a way that's a bit like if we had:
public string MyAnonymousMethod(int i){return IntString(i);}
MyAnonymousMethod is a bit pointless here; all it does is call IntString(i) and return the result, so why not just call IntString in the first place and cut out going through that method:
Enumerable.Range(0, 10).Select(IntString);
We've cut out a needless (though see note below about delegate caching) level of indirection by taking the lambda-based delegate and converting it to a method group. Hence ReSharper's advice "Convert to Method Group" or however it's worded (I don't use ReSharper myself).
There is though something to be careful of here. IQueryable<T>'s Select only takes expressions, so the provider can try to work out how to convert it to its way of doing stuff (e.g. SQL against a database). IEnumerable<T>'s Select only takes delegates so they can be executed in the .NET application itself. We can go from the former to the latter (when the queryable is really a wrapped enumerable) with Compile(), but we can't go from the latter to the former: We don't have a way of taking a delegate and turning it into an expression that means anything other than "call this delegate" which isn't something that can be turned into SQL.
Now when we use a lambda expression like i => i * 2 it will be an expression when used with IQueryable<T> and a delegate when used with IEnumerable<T> due to overload resolution rules favouring the expression with queryable (as a type it can handle both, but the expression form works with the most derived type). If though we explicitly give it a delegate, whether because we typed it somewhere as Func<> or it comes from a method group, then the overloads taking expressions aren't available and those taking delegates are used. This means it doesn't get passed to the database but rather the linq expression up to that point becomes the "database part" and it gets called and the rest of the work done in memory.
95% of the time that's best avoided. So 95% of the time if you get advice of "convert to method group" with a database-backed query you should think "uh oh! that's actually a delegate. Why is that a delegate? Can I change it to be an expression?". Only the remaining 5% of the time should you think "that'll be slightly shorter if I just pass in the method name". (Also, using a method group instead of a delegate prevents caching of delegates the compiler can do otherwise, so it might be less efficient).
There, I hope I covered the bit that you didn't understand in the course of all that, or at least there's a bit here you can point to and say "that bit there, that's the bit I don't grok".
Select(SomeModelClass.FromDbEntity)
This uses Enumerable.Select which is not what you want. This transitions out of "queryable-LINQ" into LINQ to objects. This means the database cannot execute this code.
.Where(sdt => usefulExpression.Invoke(sdt))
Here, I assume you meant .Where(usefulExpression). This passes the expression into the expression tree underlying the query. The LINQ provider can translate this expression.
When you perform experiments like this use SQL Profiler to see what SQL goes over the wire. Make sure all relevant parts of the query are translatable.
I don't want to disappoint you, but there is no magic at all. And I would suggest you to be very careful with this "new way".
Always check the result of a function by hovering it in VS. Remember that IQueryable<T> "inherits" IEnumerable<T> and also Queryable contains the extension methods with the same names as the Enumerable, and the only difference is that the former works with Expression<Func<...>> while the later just with Func<..>.
So anytime you use Func or method group over IQueryable<T>, the compiler will pick the Enumerable overload, thus silently switching from LINQ to Entities to LINQ to Objects context. But there is a huge difference between the two - this former is executed in database while the later in memory.
The key point is to stay as long as possible in the IQueryable<T> context, so the "old way" should be preferred. E.g. from your examples
.Where(sdt => sdt.someCondition == true && false || true)
or
.Where(ManyExpressions.UsefulExpression)
or
.Where(usefulExpression)
but not
.Where(sdt => usefulExpression.Invoke(sdt))
And never
.Select(SomeModelClass.FromDbEntity)
This solution threw up some red flags for me. Key among them was:
var result = db.SomeDataType
.Select(SomeModelClass.FromDbEntity) // TODO; explain this magic
.ToList(); // <<!!!!!!!!!!!!!
Whenever you're dealing with Entity Framework, you can read "ToList()" as "Copy the whole thing into memory." So "ToList()" should only be done at the last possible second.
Consider: there are lots of useful object you can pass around when dealing with EF:
The database context
The specific dataset you're targeting (e.g. context.Orders)
Queries against a context:
.
var query = context.Where(o => o.Customer.Name == "John")
.Where(o => o.TxNumber > 100000)
.OrderBy(o => o.TxDate);
//I've pulled NO data so far! "var query" is just an object I can pass around
//and even add on to! For example, I can now do this:
query = query.ThenBy(o => o.Items.Description); //and now I've appended that to my query
The real magic is that those lambdas can be thrown in to a variable too. Here's a method I use in one of my projects to do that:
/// <summary>
/// Generates the Lambda "TIn => TIn.memberName [comparison] value"
/// </summary>
static Expression<Func<TIn, bool>> MakeSimplePredicate<TIn>(string memberName, ExpressionType comparison, object value)
{
var parameter = Expression.Parameter(typeof(TIn), "t");
Expression left = Expression.PropertyOrField(parameter, memberName);
return (Expression<Func<TIn, bool>>)Expression.Lambda(Expression.MakeBinary(comparison, left, Expression.Constant(value)), parameter);
}
With this code, you can write something like the following:
public GetQuery(string field, string value)
{
var query = context.Orders;
var condition = MakeSimplePredicate<Order>(field, ExpressionType.Equal, value);
return query.Where(condition);
}
The best thing is that at this time, no data call has been. You can continue to add conditions as you wish. When you're ready to fetch the data, simply iterate through it or call ToList().
Enjoy!
Oh, and check this out if you'd like to see a more thoroughly-developed solution, albeit from a different context.
My Post on Linq Expression Trees
I have this line of code, it works but I don't understand it:
Genres.Find(delegate (Genre genre) { return genre.Id == id; });
Genres is a list of genre(music)
What exactly is happening here?
C# provides two ways of defining delegates without writing a named method for it - the old anonymous method syntax introduced in C# 2.0, and the shorter lambda syntax introduced in C# 3.0.
Your code is the old way of writing this:
Genres.Find(genre => genre.Id == id);
This article describes the evolution of anonymous functions in C#.
Your Find method takes a predicate delegate. Depending on the version of .NET targeted by your code it may or may not be the System.Predicate<T> delegate, but its functionality is equivalent. An anonymous method in parentheses provides an implementation of your predicate, allowing you to pass arbitrary conditions to your Find(...) method.
It says, find the Genre (from the list Genres) which has the Id equal to the value from the variable id.
The keyword delegate says, that this is a kind of inline function which decides whether the check is true for each item or not. The beginning (Genre genre) says "given I would call each element genre in the loop, I can check each items' Id with its named variable Id". This is: genre.Id == id.
A modern approach would be the usage of lambdas like:
var x = Genres.Find(g => g.Id == id);
In this case g is your loop variable you can check against.
An intuitive way to see it:
Genres.Find( --- The CompareGenres function is being called from here --- );
bool CompareGenres(Genre genre)
{
return genre.Id == id;
}
Find accepts a Predicate < T >, T is the type of the parameter, in this case: you're passing an instance of Genre which is being supplied by the Find method.
"The Predicate is a delegate to a method that returns true if the object passed to it matches the conditions defined in the delegate."
So you're just passing a method as a parameter in the form of a delegate
Maybe I do not use the right terms here. But form an abstract point of view: The Find method here accepts a delegate as parameter. It allows you to implement the "find" algorithm (here comparing the id). It is flexible code you could also compare any other object of "genre".
This is an example out of a book using northwind database. What is the => mean ?
Northwind db = new Northwind(#"Data Source=.\SQLEXPRESS;InitialCatalog=Northwind");
var orders = db.Customers
.Where(c => c.Country == "USA" && c.Region == "WA")
.SelectMany(c => c.Orders);
Console.WriteLine(orders.GetType());
why don't they just write
Where(c.Country == "USA" && c.Regian == "WA")
The part inside the parentheses of the Where() needs to be a function that takes a Customer and returns a boolean value (that's Func<Customer, bool> in C#'s type system).
It's most common to write that function using C#'s Lambda Expressions.
The c => part means "take the Customer that's passed in to you and call it c. Then with that value of c, compute this conditional. Without the c on the left side of the =>, the compiler would have no way of knowing what c refers to.
If they wrote this:
Where(c.Country == "USA" && c.Regian == "WA")
Where would you define what the variable c referred to? What if somewhere earlier in my code I had written:
var c = "this is a string";
This notation defines a lambda construct, where c => is bound to the delegate input expected by the Where function - in this case, c is bound to each row, and the function you run with c needs to return a boolean.
Note that you could have written a function like this:
public bool OnlyWA(Row c)
{
return c.Country == "USA" && c.Regian == "WA";
}
And then used it like this:
var orders = db.Customers
.Where(OnlyWA)
.SelectMany(c => c.Orders);
Here you don't need the c any longer because you're using a named function not a lambda variable. The only parameter to the function acts in place of the c. This is usually overkill for small functions, and further makes it harder to read as the small predicate function is moved away from its usage (and therefore contextual) location.
It's a lambda expression. It describes a function; the variables to the left of the arrow are parameters to the function. In this case, the function has one parameter, so there is a single variable to the left of the arrow.
Consider the "why don't they..." example you gave. Where is c declared? How does the compiler know what it represents?
You may wonder how the compiler knows the type of the c parameter, which it obviously does, because it allows you to call the Country and Region properties. This is accomplished with type inference. The Where method requires a delegate argument with a parameter whose type depends on the type of the sequence (Customers, in this case).
Since Customers is an IEnumerable<Customer>, the compiler expects a delegate with a single Customer parameter, and it therefore treats the single parameter of the lambda expression as a Customer variable.
=> is the lambda operator
See here: msdn
It is used in lambda expressions to separate the input variables on the left side from the lambda body on the right side. Lambda expressions are inline expressions similar to anonymous methods but more flexible; they are used extensively in LINQ queries that are expressed in method syntax.