I will preface this with I'm actively searching for the solution to this problem but figured I might short cut some research and development time if someone here on stack has already figured this out. (I have found nothing online so here goes)
We have a case in an application framework we are building where we need the capability to take in a set of Predicates (List<Expression<Func<T,bool>>>) and parse it in a search framework.
Right now we have the capability to filter in this way being that:
//Assume predicates is passed as a method argument.
// of List<Expression<Func<T,bool>>>
//Assume user is passed in as a method argument.
//Assume FilterToUserAccess is a custom extension method that restricts the dataset
// to access restrictions.
var query = _dbContext.Set<EntityType>()
.FilterToUserAccess(user);
foreach(var p in predicates){
query = query.Where(p);
}
return p.ToList();
The reason we need to do this is for scale-ability of filterable objects. However for a quick search this is not possible given the built in capabilities of EF. What I need to be able to do is:
Object A (lets pretend it's a race car) and we want to search make, model, team, and driver in a quick search box. So if I enter "Earnhardt", it would search all race car entity properties being make, model, team, and driver. I would end up with all the DEI cars as well as Dale Jr. I would like to use the same approach so we can configure a searchable entity and reflect the search configuration on application start. I would ideally like to make some way of having the query look similar to this:
//Assume predicates is passed as a method argument.
// of List<Expression<Func<T,bool>>>
//Assume user is passed in as a method argument.
//Assume FilterToUserAccess is a custom extension method that restricts the dataset
// to access restrictions.
var query = _dbContext.Set<EntityType>()
.FilterToUserAccess(user);
foreach(var p in predicates){
query = query.Or(p);
}
return p.ToList();
I realize I can do:
_dbContext.Set<EntityType>().Where(predicate1 || predicate2 || predicate3)
However this will not work for the approach we want to take to solve this problem. Ideally an admin for one of our client sites would be able to go in and configure an additional search term with a single click to be included in any and all quick searches for that entity type like we can currently pull off with Filters which use the standard .Where(...) "and" chaining logic.
First solution was a bust, however with some more digging there is an incredibly simple solution, verified and works.
Step 1: install the NuGet package for LinqKit.
Step 2: Enjoy the code below
using (ISampleRepository repo = new SampleRepository())
{
var predicates = new List<Expression<Func<Customer,bool>>>(){
(x => x.FirstName.Contains(searchValue)),
(x => x.LastName.Contains(searchValue))
};
var lambda = PredicateBuilder.False<Customer>();
lambda = predicates.Aggregate(lambda, (current, p) => current.Or(p).Expand());
var query = repo.QueryCustomers().AsExpandable().Include(x => x.Phones).Where(lambda);
return query.Take(500)
.ToList()
.Select(x => x.ToDTO())
.ToList();
}
This is just the spike sample but doing the same thing with a method taking in ->
List<T> QuickSearch<T>(string input) ...
Will be able to use the same approach. You have a collection of predicates still in Expression form passed in, then you use the predicate builder tricks to pull the query off. Then using the AsExpandable() allows you to execute the combined predicate created using the predicate builder.
Hopefully this is helpful to more than just me, but this is the solution I'm going with as it's quite a bit less code. Allows you to build your predicates elsewhere... and still combine them in an "OR" statement after the fact.
As Ladislav says, you will need to dynamically generate your LINQ expressions. Here is a simple example of a program that dynamically builds a predicate for a collection of integers:
class Program {
static void Main(string[] args) {
// Retreive your data source
List<int> numbers = new List<int>() { 0, 10, 20, 30, 40, 50, 60 };
// Create a collection of predicates that you would like to chain together.
ParameterExpression parameterExpression = Expression.Parameter(typeof(int), "x");
List<Expression> predicates = new List<Expression>();
// x >= 50
predicates.Add(Expression.GreaterThanOrEqual(parameterExpression, Expression.Constant(50)));
// x <= 20
predicates.Add(Expression.LessThanOrEqual(parameterExpression, Expression.Constant(20)));
// Build a single predicate by chaining individual predicates together in an OR fashion
Expression whereFilter = Expression.Constant(false); // Use false a base expression in OR statements
foreach (var predicate in predicates) {
whereFilter = Expression.OrElse(whereFilter, predicate);
}
// Once the expressions have been chained, create a lambda to represent the whole predicate
// x => (x >= 50) || (x <= 20)
Expression<Func<int, bool>> whereLambda =
(Expression<Func<int, bool>>)Expression.Lambda(whereFilter,
new List<ParameterExpression>() { parameterExpression });
// To use an expression directly, the datasource must be an IQueryable
// Since I am using List<T> I must call AsQueryable. This is not necessary
// if your collection is already IQueryable, like in Entity Framework.
var results = numbers.AsQueryable().Where(whereLambda);
}
}
Essentially all I do here is create several boolean statments (x >= 50) and (x <= 20) and place them in a collection. Then by looping through that collection, I take each statement and OR it to the last one. The result is a series of boolean statements all linked together by OR. I then wrap that statement in a Lambda expression so that it can be consumed by IQueryable.Where and pass it to my queryable collection. The results are a filtered set of integers from my full set.
LINQ Expressions can be confusing to say the least, but they are incredibly powerful and worthwhile to know. Please let me know if there's anything I can do to help make more sense of this example.
Related
I am building predicates using LinqKit's PrediateBuilder class to dynamically setup filters and I want to combine a nested one to another.
I have read this (http://www.albahari.com/nutshell/predicatebuilder.aspx) :
Here is my code :
// The main predicate.
var mainPredicate = PredicateBuilder.True<Document>();
// ... some other conditions to the main predicate here ...
// The inner predicate (combined conditions using OR).
var innerPredicate = PredicateBuilder.False<Document>();
foreach (var period in periods)
{
var p = period;
innerPredicate =
innerPredicate.Or(
d =>
(d.Date >= p.DateFrom && d.Date <= p.DateTo));
}
mainPredicate = mainPredicate.And(innerPredicate);
documents = this.ObjectSet.AsExpandable().Where(mainPredicate).ToList();
I am combining my two predicates just like it is explained in the documentation. However, I get this exception :
The parameter 'f' was not bound in the specified LINQ to Entities
query expression
I first thought that the inner predicate has to be expanded before combining it with the main predicate, so I changed my combining code to add a call to to the inner predicate's Expand method like this :
mainPredicate = mainPredicate.And(innerPredicate.Expand());
But I get the exact same exception.
The only difference in my code versus the documentation is that I dynamically build my nested predicate using a foreach loop. I just don't know how it can negatively affect the resulting expression.
What is wrong with my code ?
How can I actually debug this ?
Where the f parameter comes from ? How is it generated ? Why is it problematic in my case ?
Is there some kind of expression tree visualizer of some kind that could help me actually see what is wrong with the resulting expression ? Because the expression's body is hard to read.
Finally, I have found a way to avoid combining multiple predicates to the main expression tree.
Given that each predicate represents a different filter and I want the final, combined filter to be a series of must-be-respected conditions, we can say that each of the predicates has to return true for the final predicate to return true.
For that to work, the predicates has to be combined with AND. So, the resulting SQL query must look like this :
predicate1 AND predicate2 AND predicate3 ...
A better way to combine these predicates with AND is to chain Where query operators to the final query, like this :
var documents = this.ObjectSet.AsExpandable()
.Where(mainPredicate)
.Where(otherPredicate)
.Where(yetAnotherPredicate)
.ToList();
The resulting SQL query will combine each of these predicates with AND. That is just what I wanted to do.
It is easier than hacking out an expression tree by myself.
I have a table which has hundreds of columns (like FirstName and LastName).
All this data is displayed as a GridView with listing. I've made use of the SortCommand event which has the e.SortExpression, but I can't fathom how to use it in an OrderBy function since after inputting a lambda expression. IntelliSense shows me all the hundreds of columns instead of the one I need - which I only have at run-time.
How do I use the Func<TSource, TKey> selector the OrderBy functions expects from me to let it know that the string e.SortExpression is the column that needs to be sorted?
Example of what I essentially mean:
private void dataGrid_SortCommand(object source, DataGridSortCommandEventArgs e)
{
var users = new UsersEntities().UsersTable.OrderBy(x => "x."+e.SortExpression);
FillDataGrid(users);
}
Update:
I've come to understand that the way in which I'm looking to accomplish this is using Expression trees, and I'm hoping someone can provide the exact solution.
Update 2:
I've found the solution here. Turns out my question was a duplicate and to add to that, badly worded. Thanks to everyone.
And I've also just noticed that ken's offered pretty damning evidence that I should use dynamic linq. I'll consider doing so in the future, but I couldn't help but notice how fun it is to experiment with Expression trees rather than rely on libraries to do them for me. At this stage, as I'm still new to Expression trees this is a perfect opportunity to utilize them.
You can use dynamic linq That would allow you to do an OrderBy based on a string.
Edit
See my comment. Using dynamic linq, your solution could be as easy as:
var users = new UsersEntities().UsersTable.OrderBy(e.SortExpression);
You can dynamically create LINQ expressions easily as long as only logical ANDs are involved in where clauses. Simply repeat the Where clause!
// Filter
var query = new UsersEntities().UsersTable.Select(x => x);
if (!String.IsNullOrEmpty(nameFilter) {
query = query.Where(x => x.Name == nameFilter);
}
if (!String.IsNullOrEmpty(zipFilter) {
query = query.Where(x => x.Zip == zipFilter);
}
// Sorting
switch (sortField)
{
case "Name":
query = query.OrderBy(x => x.Name);
break;
case "Zip":
query = query.OrderBy(x => x.Zip);
break;
}
You could also create an expression tree dynamically, but this is not very obvious.
I have users searching records of type Record. They type a search term in a textbox and then I search records by matching several fields with the search term.
My query looks like:
var results = from record in DataContext.Records
where
record.Field1.ToLower().Contains(term) ||
record.Field2.ToLower().Contains(term) ||
record.Field3.ToLower().Contains(term)
select record;
I have a number of queries that all use the same filter and thus I would like to extract the filtering so it can be reused. Something like:
var filter = new Func<Record, string, bool>(
(record, term) =>
record.Field1.ToLower().Contains(term) ||
record.Field2.ToLower().Contains(term) ||
record.Field3.ToLower().Contains(term)
);
var results = from record in DataContext.Records
where filter(record, term)
select record;
However, it does not work because:
Method 'System.Object DynamicInvoke(System.Object[])' has no supported translation to SQL.
How can I reuse my where condition across queries?
You need to build an expression instead of a function:
Expression<Func<Record, bool>> filter =
record => record.Field1.ToLower().Contains(term); // rest omitted
The lambda expression remains the same, but you need to return it into a variable of type Expression<Func<Record, bool>> -- that will make the C# compiler compile it as an expression instead of a delegate, allowing it to be passed to LINQ to SQL.
However, you won't be able to use an expression variable with a C#-syntax where clause: you'll need to use the Where extension method:
var results = DataContext.Records.Where(filter);
Edited to add: If you want to be able to create filters on different terms, you just need a method to produce an expression from a term:
private static Expression<Func<Record, bool>> Filter(string term)
{
return r => r.Field1.ToLower().Contains(term);
}
var results = DataContext.Records.Where(Filter(term));
If you prefer to keep filter as a lambda as you have at the moment, you can do so, but the generics get a bit nested:
Func<string, Expression<Func<Record, bool>>> filter =
term => (r => r.Field1.ToLower().Contains(term));
var results = DataContext.Records.Where(filter(term));
Regardless, the important thing is that what goes in the Where clause must be an Expression<Func<Record, bool>> -- but as shown above you can make the expression depend on term by building a suitable expression on the fly. Which is exactly what LINQ to SQL would be doing if you spelled out the filter longhand in the Where clause.
Use a CompiledQuery!
var filter = CompiledQuery.Compile(
(DatabaseDataContext dc, Record record, string term) =>
record.Field1.ToLower().Contains(term) ||
record.Field2.ToLower().Contains(term) ||
record.Field3.ToLower().Contains(term)
);
var results = from record in DataContext.Records
where filter(DataContext, record, term)
select record;
For more information, see How to: Store and Reuse Queries.
In addition to the Expression<Func<Record, bool>> issue that others have pointed out, I suggest looking into PredicateBuilder. It's very good for dynamically combining lambda expressions.
I think you need to make it an Expression<Func<Record, bool>>. Otherwise it's trying to translate the actual C# method call to SQL rather than the description of it. This is not a guarantee that this version will work; I'm not sure which string functions are translatable to SQL.
Please help this Linq newbie!
I'm creating a list inside my class under test, and I would like to use Moq to check the results.
I can easily put together a Predicate which checks the results of the list. How do I then make that Predicate into an Expression?
var myList = new List<int> {1, 2, 3};
Predicate<List<int>> myPredicate = (list) =>
{
return list.Count == 3; // amongst other stuff
};
// ... do my stuff
myMock.Verify(m => m.DidStuffWith(It.Is<List<int>>( ??? )));
??? needs to be an Expression<Predicate<List<int>>> if you can get your head round that many generics. I've found answers which do this the other way round and compile an Expression into a Predicate. They're not helping me understand Linq any better, though.
EDIT: I've got it working with methods; with expressions; I would just like to know if there's any way to do it with a lambda with a body - and if not, why not?
Change:
Predicate<List<int>> myPredicate = (list) => list.Count == 3;
To:
Expression<Predicate<List<int>>> myPredicate = (list) => list.Count == 3;
The compiler does the magic for you. With some caveats1, any lambda dealing only with expressions (no blocks) can be converted into an expression tree by wrapping the delegate type (in this case Predicate<List<int>>) with Expression<>. As you noted, you could then invert it yet again by calling myPredicate.Compile().
1 For example, async lambdas (i.e. async () => await Task.Delay(1);) cannot be converted to an expression tree.
Update: You simply cannot use the compiler to arrive at the expression tree you want if it includes statements. You'll have to build up the expression tree yourself (a lot more work) using the static methods in Expression. (Expression.Block, Expression.IfThen, etc.)
Kirk Woll's answer directly addresses your question, but consider the fact that you can use the Callback method on a Setup of a void method to handle the parameters which were passed in on invocation. This makes more sense to me since you're already having to build a method to validate the list anyway; it also gives you a local copy of the list.
//what is this list used for?
var myList = new List<int> {1, 2, 3};
List<int> listWithWhichStuffWasDone = null;
//other setup
myMock.Setup(m => m.DoStuffWith(It.IsAny<List<int>>()).
Callback<List<int>>(l => listWithWhichStufFWasDone = l);
objectUnderTest.MethodUnderTest();
myMock.Verify(m => m.DoStuffWith(It.IsAny<List<int>>()));
Validate(listWithWhichStuffWasDone);
Let's say we need to apply several conditions to select from a table called "Things" (unknown count and nature)
if conditions are known, we can write
db.Things.Where(t=>foo1 && foo2 || foo3);
but if we have to build that Where condition programatically, I can imagine how can we apply ANDed conditions
IQuerable DesiredThings = db.Things.AsQuerable();
foreach (Condition c in AndedConditions)
DesiredThings = DesiredThings.Where(t => GenerateCondition(c,t));
What about ORed conditions ?
Note: we don't want to perform union, unique, or any other costly operations, it's desired that a query is generated as if we write it ad-hock
Thanks in advance.
Addition:
PredicateBuilder: Dynamically Composing Expression Predicates
You could use the Expression class with static methods to do it run time.
The below code is ment to create a delegate taking one argument called value of type int
. It reads from buttom to top so the line in question is:
var method = LambdaExpression.Lambda(orExp, Expression.Parameter(typeof(int), "value"));
the body of the method compares the value of the parameter to a call to method Bar of a newly created object of type foo
var exp2 = Expression.Equal(Expression.Parameter(typeof(int), "value"), Expression.Property(Expression.New(typeof(Foo).GetConstructor(new Type[] { })), "Bar"));
It then creates a similar expression and or's them
var orExp = Expression.OrElse(exp1, exp2);
final thing is the call to compile. That call generates a delegate that can be used in your where method call.
hope it helps tho Im not 100% sure on the expression to get the value from a parameter
var exp1 = Expression.Equal(Expression.Parameter(typeof(int),"value"), Expression.Property(Expression.New(typeof(Bar).GetConstructor(new Type[] { })), "Foo"));
var exp2 = Expression.Equal(Expression.Parameter(typeof(int), "value"), Expression.Property(Expression.New(typeof(Foo).GetConstructor(new Type[] { })), "Bar"));
var orExp = Expression.OrElse(exp1, exp2);
var method = LambdaExpression.Lambda(orExp, Expression.Parameter(typeof(int), "value"));
method.Compile();
You might wanna look at invoke for invokation instead of compiling the expression, if you need the LambdaExpression to be translated into something different than binary code (E.g. into an SQL statement)
For OR, you have two choices:
use Union/Concat
write the Expression in code
The second is closer to the .Where(x => {a} || {b}).
If you are using LINQ-to-SQL, you can use Expression.Invoke to combine multiple separate lambda expressions (see this answer) - however, this isn't supported in Entity Framework. In EF, you have to build the entire expression as a single block, using Expression.OrElse; for example here or here.