Develop Reference

How to filter records from a db using linq expressions?

I am trying to achieve the following pattern:
Main()
{
...
GetFilteredUsersById(u => u >= 100 && u < 200).Dump();
....
}
public List<UserRecord> FilteredUsersById(Func<int, bool> filter)
{
Expression<Func<UserRecord, bool>> filterExpression = x => filter(x.Id);
return someUserRecordsDb.Where(filterExpression).ToList();
}
public class UserRecord
{
public int Id;
public string Name;
}
Of course, this fails with
NotSupportedException: Method 'System.Object DynamicInvoke(System.Object[])' has no supported translation to SQL.
I understand why the error is happening, but don't know how to fix it.
Below is a pattern that does work, but I'm interested in understanding what changes I need to make to the code to the first pattern above to make it work.
Main()
{
...
GetFilteredUsersById(u => u.Id >= 100 && u.Id < 200).Dump();
....
}
public List<UserRecord> FilteredUsers(Expression<Func<UserRecord, bool>> filter)
{
return someUserRecordsDb.Where(filter).ToList();
}
public class UserRecord
{
public int Id;
public string Name;
}
Thanks for answering my first question here at StackOverflow!

As already you know, LINQ to SQL works by taking your lambdas as expression trees instead of delegates. Then, it analyzes those trees and attempts to produce equivalent SQL code. Obviously, it can't support everything you can possibly write in C#.
In your second snippet, the LINQ provider sees an expression tree that contains this:
u => u.Id >= 100 && u.Id < 200
This is very easy to understand - it involves the && operator, the >= operator and the < operator, all of which are known.
But in your first snippet, the LINQ provider sees an expression tree that contains this instead:
x => filter(x.Id)
This is very different, because it's an invocation of an arbitrary method that's not known to the provider. So now the provider has to look inside that method and see what it contains. If that method had another level of indirection, the provider would have to look even further. And so on.
If your expression tree requires that kind of work, the provider will refuse to do it. So any invocations to arbitrary methods will understandably fail.
Thus, the answer is simple: don't put a level of indirection between the location where you create the expression tree and the actual expression you want translated to SQL, otherwise the LINQ provider will reject it altogether.

There are two things to keep in mind here
1. Expression vs Func
One of the main things you need to understand is the difference between an Expression and a Func or any type of delegate type. In short the expression is passing a tree representation of the lamda, which can be translated into Sql.
There is a great answer that covers this in depth here :
Why would you use Expression<Func<T>> rather than Func<T>?
2. Not all Expressions will Work with Linq-to-Sql
Even if a function can be represented as an expression it does not mean that there will be a proper mapping from the operations being preformed in the lambda expression for the provider that is consuming your linq expression.
This is why you are getting the exception :
NotSupportedException: Method 'System.Object DynamicInvoke(System.Object[])' has no supported translation to SQL.
Here even though it is a valid expression there is no way to convert the operation to sql which is why you get an exception, even though filter would be a valid lambda if passed as an expression and called directly the translation of the statement is that it is trying to call this func filter, which doesn't map directly to anything in sql

using Extension Method or Func in EF lambda expression

I have an extension method that is fairly simple.
When trying to use it in an Entity framework I get this
LINQ to Entities does not recognize the method 'Boolean Between[DateTime](System.DateTime, System.DateTime, System.DateTime, Boolean)'
Many others are getting the same issue and I understand the error. There is always a way to make it work.
I have been digging around trying to figure out how i can re-implement this method and have it be linq to EF friendly.
for this specific method its just checking if one IComparable is between two others. so really it would just expand to
.Where(x=> x.Date >= startDate && x.Date <= endDate)
and all i am really trying to do is make it easier on the eyes and express it like
.Where(x=> x.Date.Between(startDate, endDate))
Being that I am very new to the Func and such Im sure there is a way to approach extension methods (even if written specifically for EF) so that they will be friendly with EF linq
I have done some digging around SO and other sites and come across some interesting answers but not able to get to past the finish line with them.
Thanks in advance!

The query provider is going to be tasked with taking the information provided in the expression that you give it and translating that into SQL. If you take the code that you have and compile it into a C# method then the query provider has no way of inspecting it to see what the original source code was and using that to create corresponding SQL code. You need to use some means of creating Expression objects that it can understand, the easiest means of doing this is generally through lambdas.
What we can do here is create a new extension method for our queries that will accept a query, an Expression that represents the date in question, along with the constant date values that they should be between. Using this we can construct our own expression that represents what the expression would have looked like had you manually typed out the comparisons in the lambda itself:
public static IQueryable<T> WhereBetweenDates<T>(
this IQueryable<T> query,
Expression<Func<T, DateTime>> selector,
DateTime startDate,
DateTime endDate)
{
var predicate = selector.Compose(date => date >= startDate && date <= endDate);
return query.Where(predicate);
}
Here we're using a Compose method. This method accepts one expression that maps a value to another, along with a second expression that maps that value to something else, and it creates a new expression that represents mapping the original value from the first expression to the result of the second. It can do this by replacing all uses of the parameter in the second expression with the body of the first expression:
public static Expression<Func<TFirstParam, TResult>>
Compose<TFirstParam, TIntermediate, TResult>(
this Expression<Func<TFirstParam, TIntermediate>> first,
Expression<Func<TIntermediate, TResult>> second)
{
var param = Expression.Parameter(typeof(TFirstParam), "param");
var newFirst = first.Body.Replace(first.Parameters[0], param);
var newSecond = second.Body.Replace(second.Parameters[0], newFirst);
return Expression.Lambda<Func<TFirstParam, TResult>>(newSecond, param);
}
Here we're using a method to replace all instances of one expression with another. This can be done using the following helper method:
public static Expression Replace(this Expression expression,
Expression searchEx, Expression replaceEx)
{
return new ReplaceVisitor(searchEx, replaceEx).Visit(expression);
}
internal class ReplaceVisitor : ExpressionVisitor
{
private readonly Expression from, to;
public ReplaceVisitor(Expression from, Expression to)
{
this.from = from;
this.to = to;
}
public override Expression Visit(Expression node)
{
return node == from ? to : base.Visit(node);
}
}
Now I know that this does seem like a lot of code, but the idea here is that you're not using all of this just to write this one method. Everything after that first method is a fundamental building block that you can use to manipulate expressions in reasonably straightforward (and in particular, statically typed) ways. The Compose method can be re-used in all sorts of other contexts to apply a frequently used operation on a constantly changing sub-expression.

You can only use a small subset of functions on types inside the expression that Entity Framework already knows how to handle. Traditionally these are methods like Contains and similar (see this page for a fuller list). Your own extension methods will not be recognized by the parser if they are inside the expression.
However one thing you could do is make a extension method for IQueryable<T> that takes in your range as parameters.
public static IQueryable<T> Between(this IQueryable<T> query, Expression<Func<T, DateTime> selector, DateTime start, DateTime end)
{
//...
}
Now what goes inside the //... would take me a hour or so to figure it all out, and is a little too much to write up for a quick answer on the internet without being paid. But if you learn how to parse and write your own expressions (which is a good skill to have) you may be able to figure out how to do it on your own.
EDIT: Or Servy can figure it out and post it while I am typing up my answer :)

Nhibernate - Database side queries

See https://stackoverflow.com/questions/41810306/appointment-scheduling....

There are two problems with your code.
The first is how you are declaring your method:
Func<T, bool> ComposeFilter
When you do that, you are returning a compiled delegate, and the Where extension method overload that gets chosen is from Enumerable instead of Queryable.
You have to declare it as:
Expression<Func<T, bool>> ComposeFilter
The second problem is that the code you wrote cannot be converted to an expression tree. Since you are not using primitives but constructing the expression dynamically, you need to learn about the stuff in System.Linq.Expressions.

What's the purpose of the Expression class?

I'm wondering what exactly is the difference between wrapping a delegate inside Expression<> and not ?
I'm seeing Expression<Foo> being used a lot with LinQ, but so far I've not found any article that explains the difference between this, and just using a delegate.
E.g.
Func<int, bool> Is42 = (value) => value == 42;
vs.
Expression<Func<int, bool>> Is42 = (value) => value == 42;

tl;dr, To have an expression is like having the source code of an application, and a delegate is an executable to run the application. An expression can be thought of as the "source" (i.e., syntax tree) of the code that would run. A delegate is a specific compilation that you would run and does the thing.
By storing a lambda as a delegate, you are storing a specific instance of a delegate that does some action. It can't be modified, you just call it. Once you have your delegate, you have limited options in inspecting what it does and whatnot.
By storing a lambda as an expression, you are storing an expression tree that represents the delegate. It can be manipulated to do other things like changing its parameters, changing the body and make it do something radically different. It could even be compiled back to a delegate so you may call it if you wish. You can easily inspect the expression to see what its parameters are, what it does and how it does it. This is something that a query provider can use to understand and translate an expression to another language (such as write an SQL query for a corresponding expression tree).
It is also a whole lot easier to create a delegate dynamically using expressions than it is emitting the code. You can think of your code at a higher level as expressions that is very similar to how a compiler views code instead of going low-level and view your code as IL instructions.
So with an expression, you are capable to do much more than a simple anonymous delegate. Though it's not really free, performance will take a hit if you run compiled expressions compared to a regular method or an anonymous delegate. But that might not be an issue as the other benefits to using expressions may be important to you.

Func<> is just a delegate type. An Expression is a runtime representation of the complete tree of operations which, optionally, may be compiled at runtime into a delegate. It's this tree that is parsed by Expression parsers like Linq-to-SQL to generate SQL statements or do other clever things. When you assign a lambda to an Expression type, the compiler generates this expression tree as well as the usual IL code. More on expression trees.

To illustrate other answers, if you compile those 2 expressions and have look at the compiler generated code, this i what you will see:
Func<int, bool> Is42 = (value) => value == 42;
Func<int, bool> Is42 = new Func<int, bool>((#value) => value == 42);
Expression<Func<int, bool>> Is42 = (value) => value == 42;
ParameterExpression[] parameterExpressionArray;
ParameterExpression parameterExpression = Expression.Parameter(typeof(int), "value");
Expression<Func<int, bool>> Is42 = Expression.Lambda<Func<int, bool>>(Expression.Equal(parameterExpression, Expression.Constant(42, typeof(int))), new ParameterExpression[] { parameterExpression });

Expression Trees allow you to inspect the code inside the expression, in your code.
For example, if you passed this expression: o => o.Name, your code could find out that the Name property was being accessed inside the expression.

Provides the base class from which the classes that represent
expression tree nodes are derived.
System.Linq.Expressions.BinaryExpression
System.Linq.Expressions.BlockExpression
System.Linq.Expressions.ConditionalExpression
System.Linq.Expressions.ConstantExpression
System.Linq.Expressions.DebugInfoExpression
System.Linq.Expressions.DefaultExpression
System.Linq.Expressions.DynamicExpression
System.Linq.Expressions.GotoExpression
System.Linq.Expressions.IndexExpression
System.Linq.Expressions.InvocationExpression
System.Linq.Expressions.LabelExpression
System.Linq.Expressions.LambdaExpression
System.Linq.Expressions.ListInitExpression
System.Linq.Expressions.LoopExpression
System.Linq.Expressions.MemberExpression
System.Linq.Expressions.MemberInitExpression
System.Linq.Expressions.MethodCallExpression
System.Linq.Expressions.NewArrayExpression
System.Linq.Expressions.NewExpression
System.Linq.Expressions.ParameterExpression
System.Linq.Expressions.RuntimeVariablesExpression
System.Linq.Expressions.SwitchExpression
System.Linq.Expressions.TryExpression
System.Linq.Expressions.TypeBinaryExpression
System.Linq.Expressions.UnaryExpression
http://msdn.microsoft.com/en-us/library/system.linq.expressions.expression.aspx
Expression tree represents linq expression that can be analyzed and for example turned into SQL query.

To whatever the other wrote (that is completely correct) I'll add that through the Expression class you can create new methods at runtime. There are some limits. Not all the things you can do in C# can be done in an Expression tree (at least in .NET 3.5 . With .NET 4.0 they have added a great number of possible Expression "types"). The use of this could be (for example) to create a dynamic query and pass it to LINQ-to-SQL or do some filtering based on the input of the user... (you could always do this with CodeDom if all you wanted was a dynamic method incompatible with LINQ-to-SQL, but emitting directly IL code is quite difficult :-) )

Using Lambda Expressions trees with IEnumerable

I've been trying to learn more about using Lamba expression trees and so I created a simple example. Here is the code, this works in LINQPad if pasted in as a C# program.
void Main()
{
IEnumerable<User> list = GetUsers().Where(NameContains("a"));
list.Dump("Users");
}
// Methods
public IEnumerable<User> GetUsers()
{
yield return new User{Name = "andrew"};
yield return new User{Name = "rob"};
yield return new User{Name = "chris"};
yield return new User{Name = "ryan"};
}
public Expression<Func<User, bool>> NameContains(string namePart)
{
return u => u.Name.Contains(namePart);
}
// Classes
public class User
{
public string Name { get; set; }
}
This results in the following error:
The type arguments for method 'System.Linq.Enumerable.Where(System.Collections.Generic.IEnumerable, System.Func)' cannot be inferred from the usage. Try specifying the type arguments explicitly.
However if I just substitute the first line in main with this:
IEnumerable<User> list = GetUsers().Where(u => u.Name.Contains("a"));
It works fine. Can tell me what I'm doing wrong, please?

The Enumerable.Where method takes a Func<T, bool>, not an Expression<Func<T, bool>>. Perhaps you're confusing with Queryable.Where, which does take an expression as a parameter... In your case you don't need an expression, you just need a delegate that can be executed against each item in the sequence. The purpose of expressions is (mostly) to be analysed and translated to something else (SQL for instance), to perform the query against an external data source

Change the return type of NameContains from Expression<Func<User, Bool>> to simply Func<User, Bool>. In this situation, there's no need to return the Expression, you actually want to return the compiled delegate. There's a difference between the expression that makes up the lambda, and the lambda (which is a delegate) itself.
If you send a lambda into a method, the method can accept the lambda either as an expression, or as a compiled delegate type, depending on what you specify in the parameters. If the incoming parameter type is an Expression, you can send in something that looks like a delegate, however, if the method is expecting a delegate, you have to give it a compiled delegate, not simply an expression. That being said, you can also do something like:
var certainUsers = GetUsers().Where(NameContains("a").Compile());
Which would compile the expression, and return a Func<User, Bool>.

Lambda expressions can be treated as either code (delegates) or as data (expression trees)
In your example your are attempting to treat the lambda expression as code.
You would use the Expression<> declaration when you want to treat the lambda expression as data.
Why would you want to do this?
Here is a quote from the Book Linq In Action,
" Expression trees can be given to tools at runtime, which use them to guide
their execution or translate them into something else, such as SQL in the case of
LINQ to SQL."
Using Expression Trees allows you to take the lambda expression and convert it to data, this is how Linq to SQL works, it takes the lambda expression or query operators or query expressions and converts them to SQL. You of course can view and modify the created expression tree once converted to sql.

There is a huge difference between Expression and Func<...>, the Func is a pure delegate you can invoke it directly, the expression is a data structure holds information about an expression like information about lambda expression or Linq Syntax (e.g. From x in list where x.Id = 1 select x). The expression cannot be invoked directly it must be compiled first, Expressions is used to convert the expression from a way to another like Link To Sql which converts an expression to Sql statements, the best way to do this to change the return type of the NameContains Method to Func insted of expression cuz you are working with Linq To Objects, but when using with Linq To Sql you can use both Expression or func.