Develop Reference

Create generic .Any extention method for IQueryable [duplicate]

I'm building a SQL "WHERE" clause dynamically using the System.Linq.Expressions.Expression class. It works well for simple clauses, e.g. to add "PhaseCode = X" clause, I do the following:
var equalTarget = Expression.Constant(phaseCode, typeof(int?));
var phaseEquals = Expression.Equal(Expression.PropertyOrField(projParam, "PhaseCode"), equalTarget);
However, now I'm trying to build an expression that will return the record if a project has been assigned to a particular group. Project and Group has many-to-many relationship.
Without the expression trees, I would do it as follows:
db.Projects.Where(p => .... && p.GroupsAssigned.Any(g => g.ID == groupId))
However, I can't seem to find a way to express that with the Expression class.
There are actually two things I can't figure out:
How to traverse the relationships between tables
How to do x.Any()
Any help is greatly appreciated.

Calling an extension method, like Enumerable.Any or Queryable.Any, is simply a static method call on the sequence and the lambda expression you created for the WHERE clause. You can use Expression.Call to do this:
// for Enumerable.Any<T>(IEnumerable<T>,Predicate<T>)
var overload = typeof(Enumerable).GetMethods("Any")
.Single(mi => mi.GetParameters().Count() == 2);
var call = Expression.Call(
overload,
Expression.PropertyOrField(projParam, "GroupsAssigned"),
anyLambda);
For Queryable.Any<T>, you'll need to roll this up into a method:
static Expression BuildAny<TSource>(Expression<Func<TSource, bool>> predicate)
{
var overload = typeof(Queryable).GetMethods("Any")
.Single(mi => mi.GetParameters().Count() == 2);
var call = Expression.Call(
overload,
Expression.PropertyOrField(projParam, "GroupsAssigned"),
predicate);
return call;
}
Although this seems odd that you're unable to do this through a normal query.

LINQ dynamic expression in order by clause

I have the following query:
product = product.OrderByDescending(d => d.ProductAttributeItem
.Where(ai => ai.AttributeItem.AttributeId == (int)DefaultAttributes.Name)
.SelectMany(p => p.AttributeItem.AttributeItemValue)
.Any(o => EF.Functions.Like(o.Value, "%apple%") || EF.Functions.Like(o.Value, "%samsung%"))
which in fact is working perfectly fine. product is a rather complex query built with many predicates based on input filters. Here we are interested in the Any() part of the query, specifically the predicate part - how EF.Functions.Like(o.Value, "%apple%") || EF.Functions.Like(o.Value, "%samsung%") can be generated dynamically.
I am already using some predicate builder extension methods in our project and they are working great for non nested cases like:
var condition = PredicateBuilder.True<AttributeItemValue>();
if(filters.OnlyActivated)
condition = condition.And(product => product.IsActive);
product = _context.Product.Where(condition);
So I have tried to build the predicate in loop:
var aivCond = PredicateBuilder.True<AttributeItemValue>();
foreach (var s in searchQueryArray)
{
aivCond = aivCond.Or(f =>
EF.Functions.Like(f.Value, "%" + s + "%"));
}
So now the aivCond is of type Expression<Func<AttributItemValue, bool> but this can't be used to replace the lambda in Any() because it expects Func<TSource, bool>. Tried to compile it this way aivCond.Compile() but the following error occurs:
System.ArgumentException: Expression of type 'System.Func`2[AttributeItemValue,System.Boolean]' cannot be used for parameter of type 'System.Linq.Expressions.Expression`1[System.Func`2[AttributeItemValue,System.Boolean]]' of method 'Boolean Any[AttributeItemValue](System.Linq.IQueryable`1[AttributeItemValue]
I have also tried to build the lambda expression from string:
var filter = "f => EF.Functions.Like(f.Value, \"%apple%\") || f => EF.Functions.Like(f.Value, \"%samsung%\")";
var options = ScriptOptions.Default
.AddReferences(typeof(AttributeItemValue).Assembly)
.AddReferences(typeof(Microsoft.EntityFrameworkCore.EF).Assembly)
.AddReferences(typeof(DbFunctions).Assembly)
.AddImports("Microsoft.EntityFrameworkCore");
Func<AttributeItemValue, bool> filterExpression = await CSharpScript.EvaluateAsync<Func<AttributeItemValue, bool>>(filter, options);
with no luck.
I know I missing knowledge for expression trees, compiling and invocation for delegates so any help(and explanation) would be really appreciated!
EDIT / SOLUTION
So there is a solution thanks to the help of Richard Deeming.
He was right for starting the predicate with False. I stupidly copy/pasted code from a different method without noticing.
As about his second comment, adding AsQueryable() allows to pass an Expression<Func<TSource, bool>> which is pretty obvious but not for me. This translates fine and the compiler is ok.
Anyway, there is another approach, using LINQKit's AsExpandble() method and the inbuilt Compile() method in the Expression class and as described in LINQKit:
Compile is an inbuilt method in the Expression class. It converts the Expression<Func<Purchase,bool> into a plain Func<Purchase,bool> which satisfies the compiler. Of course, if this method actually ran, we'd end up with compiled IL code instead of an expression tree, and LINQ to SQL or Entity Framework would throw an exception. But here's the clever part: Compile never actually runs; nor does LINQ to SQL or Entity Framework ever get to see it. The call to Compile gets stripped out entirely by a special wrapper that was created by calling AsExpandable, and substituted for a correct expression tree.
So the code would look like this:
product = product.AsExpandable().OrderBy(d => d.ProductAttributeItem
.Where(ai => ai.AttributeItem.AttributeId == (int)DefaultAttributes.Name)
.SelectMany(p => p.AttributeItem.AttributeItemValue)
.Any(aivCond.Compile()));
Thank you everyone for the help and I hope this question helps somebody lost in the code... !

As mentioned in the comments, you just need to use the AsQueryable method on the collection to pass in the Expression<Func<TItem, bool>> as the filter.
product = product.OrderByDescending(d => d.ProductAttributeItem
.Where(ai => ai.AttributeItem.AttributeId == (int)DefaultAttributes.Name)
.SelectMany(p => p.AttributeItem.AttributeItemValue)
.AsQueryable().Any(aivCond);

EF + LINQ: How do I define and utilize a variable (maybe a Func variable) to define part of a Select statement?

Before try to use a variable (this has sql server do the calculation):
IQueryable<MyEntity> query = _dbSet; // IQueryable<TEntity>
var results = query.Select(m => new MyViewModel
{
MyCalculation = m.Column1 * m.Column2
}).ToList();
What I want to do (dynamically create part of my select statement from a Func variable or some other kind of variable to allow this):
IQueryable<MyEntity> query = _dbSet; // IQueryable<TEntity>
Func<MyEntity, decimal> funcVariableAttempt = m => m.Column1 * m.Column2;
var results = query.Select(m => new MyViewModel
{
MyCalculation = funcVariableAttempt.Invoke(m) // My foolish attempt does not work.
}).ToList();
The error I get when I try what I want (aka my foolish attempt):
LINQ to Entities does not recognize the method 'System.Decimal Invoke(MyProject.Repository.Models.MyEntity)' method, and this method cannot be translated into a store expression.
How do I define and utilize a variable (maybe a Func variable) to define part of a Select statement?

It's a completely valid question that I stumbeled across earlier and found a solution that works well for me.
The problem with your Func<MyEntity, decimal> is that it is a delegate and that the O/R mapper has to have access to the internal expression (the multiplication of two properties in your case). But this information is compiled into the delegate and hidden forever.
If you however start off with a Expression<Func<MyEntity, decimal>> customCalculation, things look more promising as you have the internal logic as an expression tree.
Problem with this is: you cannot invoke an expression. customCalculation(m) doesn't compile.
The compiler would let you write
m => new MyViewModel { MyCalculation = customCalculation.Compile()(m) }
, but this wouldn't be understood by most O/R mappers.
But you see that it at least somehow contains the customCalculation lambda expression and also how it relates to its surrounding expressions.
Getting from here to the expression tree as in your original working version involves some expression manipulation:
We have to replace customCalculation.Compile()(m) with the body of the lambda that is to be Compile()d, but with the lambda's parameter(s) replaced with the respective expression(s) of the delegate invocation. So if customCalculation were x => x.Column1 * x.Column2, customCalculation.Compile()(m) would have to be replaced with m.Column1 * m.Column2
Doing so is not trivial, since the lambda itself has to be dug out of a field inside an instance of a compiler generated closure class.
I've posted my implementation of this expression manipulator in another similar question . Hope that helps.
With that, you should be able to:
var customCalculation = (Expression<Func<MyEntity, decimal>>)(x => x.Column1 * x.Column2);
var selector = Express.Prepare((Expression<Func<MyEntity, MyViewModel>>)(m => new MyViewModel { MyCalculation = customCalculation.Compile()(m) }));
var result = query.Select(selector).ToList();

As you already know, your funcVariableAttempt makes no sense to your database, so you have to call your method in the linq-to-object context. i.e. for instance first fetch the data as an Enumerable, then call your method:
var results = query.Select(m => new {
Column1= col1,
Column2= col2
}).AsEnumerable()
.Select(m => new MyViewModel
{
MyCalculation = Foo(m.Column1, m.Column2)
});
*Note: Code is not tested.

You should call ToList() first and perform the Select() on the result in memory.
var results = query.ToList()
.Select(m => new MyViewModel {
MyCalculation = Foo(m.Column1, m.Column2)
});
You're trying to perform the Select as part of the query. You should just use a regular function for the mapping calculation. Lambda functions are useful with LINQ but in this case they're not needed.

Nesting PredicateBuilder predicates : 'The parameter 'f' was not bound in the specified LINQ to Entities query expression'

I am building predicates using LinqKit's PrediateBuilder class to dynamically setup filters and I want to combine a nested one to another.
I have read this (http://www.albahari.com/nutshell/predicatebuilder.aspx) :
Here is my code :
// The main predicate.
var mainPredicate = PredicateBuilder.True<Document>();
// ... some other conditions to the main predicate here ...
// The inner predicate (combined conditions using OR).
var innerPredicate = PredicateBuilder.False<Document>();
foreach (var period in periods)
{
var p = period;
innerPredicate =
innerPredicate.Or(
d =>
(d.Date >= p.DateFrom && d.Date <= p.DateTo));
}
mainPredicate = mainPredicate.And(innerPredicate);
documents = this.ObjectSet.AsExpandable().Where(mainPredicate).ToList();
I am combining my two predicates just like it is explained in the documentation. However, I get this exception :
The parameter 'f' was not bound in the specified LINQ to Entities
query expression
I first thought that the inner predicate has to be expanded before combining it with the main predicate, so I changed my combining code to add a call to to the inner predicate's Expand method like this :
mainPredicate = mainPredicate.And(innerPredicate.Expand());
But I get the exact same exception.
The only difference in my code versus the documentation is that I dynamically build my nested predicate using a foreach loop. I just don't know how it can negatively affect the resulting expression.
What is wrong with my code ?
How can I actually debug this ?
Where the f parameter comes from ? How is it generated ? Why is it problematic in my case ?
Is there some kind of expression tree visualizer of some kind that could help me actually see what is wrong with the resulting expression ? Because the expression's body is hard to read.

Finally, I have found a way to avoid combining multiple predicates to the main expression tree.
Given that each predicate represents a different filter and I want the final, combined filter to be a series of must-be-respected conditions, we can say that each of the predicates has to return true for the final predicate to return true.
For that to work, the predicates has to be combined with AND. So, the resulting SQL query must look like this :
predicate1 AND predicate2 AND predicate3 ...
A better way to combine these predicates with AND is to chain Where query operators to the final query, like this :
var documents = this.ObjectSet.AsExpandable()
.Where(mainPredicate)
.Where(otherPredicate)
.Where(yetAnotherPredicate)
.ToList();
The resulting SQL query will combine each of these predicates with AND. That is just what I wanted to do.
It is easier than hacking out an expression tree by myself.

Chaining OR conditions in EF 5.0

I will preface this with I'm actively searching for the solution to this problem but figured I might short cut some research and development time if someone here on stack has already figured this out. (I have found nothing online so here goes)
We have a case in an application framework we are building where we need the capability to take in a set of Predicates (List<Expression<Func<T,bool>>>) and parse it in a search framework.
Right now we have the capability to filter in this way being that:
//Assume predicates is passed as a method argument.
// of List<Expression<Func<T,bool>>>
//Assume user is passed in as a method argument.
//Assume FilterToUserAccess is a custom extension method that restricts the dataset
// to access restrictions.
var query = _dbContext.Set<EntityType>()
.FilterToUserAccess(user);
foreach(var p in predicates){
query = query.Where(p);
}
return p.ToList();
The reason we need to do this is for scale-ability of filterable objects. However for a quick search this is not possible given the built in capabilities of EF. What I need to be able to do is:
Object A (lets pretend it's a race car) and we want to search make, model, team, and driver in a quick search box. So if I enter "Earnhardt", it would search all race car entity properties being make, model, team, and driver. I would end up with all the DEI cars as well as Dale Jr. I would like to use the same approach so we can configure a searchable entity and reflect the search configuration on application start. I would ideally like to make some way of having the query look similar to this:
//Assume predicates is passed as a method argument.
// of List<Expression<Func<T,bool>>>
//Assume user is passed in as a method argument.
//Assume FilterToUserAccess is a custom extension method that restricts the dataset
// to access restrictions.
var query = _dbContext.Set<EntityType>()
.FilterToUserAccess(user);
foreach(var p in predicates){
query = query.Or(p);
}
return p.ToList();
I realize I can do:
_dbContext.Set<EntityType>().Where(predicate1 || predicate2 || predicate3)
However this will not work for the approach we want to take to solve this problem. Ideally an admin for one of our client sites would be able to go in and configure an additional search term with a single click to be included in any and all quick searches for that entity type like we can currently pull off with Filters which use the standard .Where(...) "and" chaining logic.

First solution was a bust, however with some more digging there is an incredibly simple solution, verified and works.
Step 1: install the NuGet package for LinqKit.
Step 2: Enjoy the code below
using (ISampleRepository repo = new SampleRepository())
{
var predicates = new List<Expression<Func<Customer,bool>>>(){
(x => x.FirstName.Contains(searchValue)),
(x => x.LastName.Contains(searchValue))
};
var lambda = PredicateBuilder.False<Customer>();
lambda = predicates.Aggregate(lambda, (current, p) => current.Or(p).Expand());
var query = repo.QueryCustomers().AsExpandable().Include(x => x.Phones).Where(lambda);
return query.Take(500)
.ToList()
.Select(x => x.ToDTO())
.ToList();
}
This is just the spike sample but doing the same thing with a method taking in ->
List<T> QuickSearch<T>(string input) ...
Will be able to use the same approach. You have a collection of predicates still in Expression form passed in, then you use the predicate builder tricks to pull the query off. Then using the AsExpandable() allows you to execute the combined predicate created using the predicate builder.
Hopefully this is helpful to more than just me, but this is the solution I'm going with as it's quite a bit less code. Allows you to build your predicates elsewhere... and still combine them in an "OR" statement after the fact.

As Ladislav says, you will need to dynamically generate your LINQ expressions. Here is a simple example of a program that dynamically builds a predicate for a collection of integers:
class Program {
static void Main(string[] args) {
// Retreive your data source
List<int> numbers = new List<int>() { 0, 10, 20, 30, 40, 50, 60 };
// Create a collection of predicates that you would like to chain together.
ParameterExpression parameterExpression = Expression.Parameter(typeof(int), "x");
List<Expression> predicates = new List<Expression>();
// x >= 50
predicates.Add(Expression.GreaterThanOrEqual(parameterExpression, Expression.Constant(50)));
// x <= 20
predicates.Add(Expression.LessThanOrEqual(parameterExpression, Expression.Constant(20)));
// Build a single predicate by chaining individual predicates together in an OR fashion
Expression whereFilter = Expression.Constant(false); // Use false a base expression in OR statements
foreach (var predicate in predicates) {
whereFilter = Expression.OrElse(whereFilter, predicate);
}
// Once the expressions have been chained, create a lambda to represent the whole predicate
// x => (x >= 50) || (x <= 20)
Expression<Func<int, bool>> whereLambda =
(Expression<Func<int, bool>>)Expression.Lambda(whereFilter,
new List<ParameterExpression>() { parameterExpression });
// To use an expression directly, the datasource must be an IQueryable
// Since I am using List<T> I must call AsQueryable. This is not necessary
// if your collection is already IQueryable, like in Entity Framework.
var results = numbers.AsQueryable().Where(whereLambda);
}
}
Essentially all I do here is create several boolean statments (x >= 50) and (x <= 20) and place them in a collection. Then by looping through that collection, I take each statement and OR it to the last one. The result is a series of boolean statements all linked together by OR. I then wrap that statement in a Lambda expression so that it can be consumed by IQueryable.Where and pass it to my queryable collection. The results are a filtered set of integers from my full set.
LINQ Expressions can be confusing to say the least, but they are incredibly powerful and worthwhile to know. Please let me know if there's anything I can do to help make more sense of this example.