Develop Reference

Create generic .Any extention method for IQueryable [duplicate]

I'm building a SQL "WHERE" clause dynamically using the System.Linq.Expressions.Expression class. It works well for simple clauses, e.g. to add "PhaseCode = X" clause, I do the following:
var equalTarget = Expression.Constant(phaseCode, typeof(int?));
var phaseEquals = Expression.Equal(Expression.PropertyOrField(projParam, "PhaseCode"), equalTarget);
However, now I'm trying to build an expression that will return the record if a project has been assigned to a particular group. Project and Group has many-to-many relationship.
Without the expression trees, I would do it as follows:
db.Projects.Where(p => .... && p.GroupsAssigned.Any(g => g.ID == groupId))
However, I can't seem to find a way to express that with the Expression class.
There are actually two things I can't figure out:
How to traverse the relationships between tables
How to do x.Any()
Any help is greatly appreciated.

Calling an extension method, like Enumerable.Any or Queryable.Any, is simply a static method call on the sequence and the lambda expression you created for the WHERE clause. You can use Expression.Call to do this:
// for Enumerable.Any<T>(IEnumerable<T>,Predicate<T>)
var overload = typeof(Enumerable).GetMethods("Any")
.Single(mi => mi.GetParameters().Count() == 2);
var call = Expression.Call(
overload,
Expression.PropertyOrField(projParam, "GroupsAssigned"),
anyLambda);
For Queryable.Any<T>, you'll need to roll this up into a method:
static Expression BuildAny<TSource>(Expression<Func<TSource, bool>> predicate)
{
var overload = typeof(Queryable).GetMethods("Any")
.Single(mi => mi.GetParameters().Count() == 2);
var call = Expression.Call(
overload,
Expression.PropertyOrField(projParam, "GroupsAssigned"),
predicate);
return call;
}
Although this seems odd that you're unable to do this through a normal query.

LINQ dynamic expression in order by clause

I have the following query:
product = product.OrderByDescending(d => d.ProductAttributeItem
.Where(ai => ai.AttributeItem.AttributeId == (int)DefaultAttributes.Name)
.SelectMany(p => p.AttributeItem.AttributeItemValue)
.Any(o => EF.Functions.Like(o.Value, "%apple%") || EF.Functions.Like(o.Value, "%samsung%"))
which in fact is working perfectly fine. product is a rather complex query built with many predicates based on input filters. Here we are interested in the Any() part of the query, specifically the predicate part - how EF.Functions.Like(o.Value, "%apple%") || EF.Functions.Like(o.Value, "%samsung%") can be generated dynamically.
I am already using some predicate builder extension methods in our project and they are working great for non nested cases like:
var condition = PredicateBuilder.True<AttributeItemValue>();
if(filters.OnlyActivated)
condition = condition.And(product => product.IsActive);
product = _context.Product.Where(condition);
So I have tried to build the predicate in loop:
var aivCond = PredicateBuilder.True<AttributeItemValue>();
foreach (var s in searchQueryArray)
{
aivCond = aivCond.Or(f =>
EF.Functions.Like(f.Value, "%" + s + "%"));
}
So now the aivCond is of type Expression<Func<AttributItemValue, bool> but this can't be used to replace the lambda in Any() because it expects Func<TSource, bool>. Tried to compile it this way aivCond.Compile() but the following error occurs:
System.ArgumentException: Expression of type 'System.Func`2[AttributeItemValue,System.Boolean]' cannot be used for parameter of type 'System.Linq.Expressions.Expression`1[System.Func`2[AttributeItemValue,System.Boolean]]' of method 'Boolean Any[AttributeItemValue](System.Linq.IQueryable`1[AttributeItemValue]
I have also tried to build the lambda expression from string:
var filter = "f => EF.Functions.Like(f.Value, \"%apple%\") || f => EF.Functions.Like(f.Value, \"%samsung%\")";
var options = ScriptOptions.Default
.AddReferences(typeof(AttributeItemValue).Assembly)
.AddReferences(typeof(Microsoft.EntityFrameworkCore.EF).Assembly)
.AddReferences(typeof(DbFunctions).Assembly)
.AddImports("Microsoft.EntityFrameworkCore");
Func<AttributeItemValue, bool> filterExpression = await CSharpScript.EvaluateAsync<Func<AttributeItemValue, bool>>(filter, options);
with no luck.
I know I missing knowledge for expression trees, compiling and invocation for delegates so any help(and explanation) would be really appreciated!
EDIT / SOLUTION
So there is a solution thanks to the help of Richard Deeming.
He was right for starting the predicate with False. I stupidly copy/pasted code from a different method without noticing.
As about his second comment, adding AsQueryable() allows to pass an Expression<Func<TSource, bool>> which is pretty obvious but not for me. This translates fine and the compiler is ok.
Anyway, there is another approach, using LINQKit's AsExpandble() method and the inbuilt Compile() method in the Expression class and as described in LINQKit:
Compile is an inbuilt method in the Expression class. It converts the Expression<Func<Purchase,bool> into a plain Func<Purchase,bool> which satisfies the compiler. Of course, if this method actually ran, we'd end up with compiled IL code instead of an expression tree, and LINQ to SQL or Entity Framework would throw an exception. But here's the clever part: Compile never actually runs; nor does LINQ to SQL or Entity Framework ever get to see it. The call to Compile gets stripped out entirely by a special wrapper that was created by calling AsExpandable, and substituted for a correct expression tree.
So the code would look like this:
product = product.AsExpandable().OrderBy(d => d.ProductAttributeItem
.Where(ai => ai.AttributeItem.AttributeId == (int)DefaultAttributes.Name)
.SelectMany(p => p.AttributeItem.AttributeItemValue)
.Any(aivCond.Compile()));
Thank you everyone for the help and I hope this question helps somebody lost in the code... !

As mentioned in the comments, you just need to use the AsQueryable method on the collection to pass in the Expression<Func<TItem, bool>> as the filter.
product = product.OrderByDescending(d => d.ProductAttributeItem
.Where(ai => ai.AttributeItem.AttributeId == (int)DefaultAttributes.Name)
.SelectMany(p => p.AttributeItem.AttributeItemValue)
.AsQueryable().Any(aivCond);

Linq to entities extension method inner query (EF6)

Can someone explain to me why the EF Engine is failing in the following scenario?
It works fine with the following expression:
var data = context.Programs
.Select(d => new MyDataDto
{
ProgramId = d.ProgramId,
ProgramName = d.ProgramName,
ClientId = d.ClientId,
Protocols = d.Protocols.Where(p => p.UserProtocols.Any(u => u.UserId == userId))
.Count(pr => pr.Programs.Any(pg => pg.ProgramId == d.ProgramId))
})
.ToList();
But if I encapsulate some into an extension method:
public static IQueryable<Protocol> ForUser(this IQueryable<Protocol> protocols, int userId)
{
return protocols.Where(p => p.UserProtocols.Any(u => u.UserId == userId));
}
The resulting query:
var data = context.Programs
.Select(d => new MyDataDto
{
ProgramId = d.ProgramId,
ProgramName = d.ProgramName,
ClientId = d.ClientId,
Protocols = d.Protocols.ForUser(userId)
.Count(pr => pr.Programs.Any(pg => pg.ProgramId == d.ProgramId))
})
.ToList();
Fails with the exception: LINQ to Entities does not recognize the method 'System.Linq.IQueryable1[DAL.Protocol] ForUser(System.Linq.IQueryable1[DAL.Protocol], Int32)' method, and this method cannot be translated into a store expression.
I would expect the EF Engine to build the entire expression tree, chaining the necessary expressions and then generate the SQL. Why doesn't it do that?

This is happening because the call to ForUser() is being made inside of the expression tree that the C# compiler builds when it sees the lambda you pass into Select. Entity Framework tries to figure out how to convert that function into SQL, but it can't invoke the function for a few reasons (e.g. d.Protocols does not exist at the moment).
The simplest approach that works for a case like this is to have your helper return a criteria lambda expression, and then pass that into the .Where() method yourself:
public static Expression<Func<Protocol, true>> ProtocolIsForUser(int userId)
{
return p => p.UserProtocols.Any(u => u.UserId == userId);
}
...
var protocolCriteria = Helpers.ProtocolIsForUser(userId);
var data = context.Programs
.Select(d => new MyDataDto
{
ProgramId = d.ProgramId,
ProgramName = d.ProgramName,
ClientId = d.ClientId,
Protocols = d.Protocols.Count(protocolCriteria)
})
.ToList();
More information
When you invoke a LINQ method outside of an expression tree (like you do with context.Programs.Select(...)), the Queryable.Select() extension method actually gets invoked, and its implementation returns an IQueryable<> that represents the extension method getting called on the original IQueryable<>. Here's the implementation of Select, for instance:
public static IQueryable<TResult> Select<TSource,TResult>(this IQueryable<TSource> source, Expression<Func<TSource, TResult>> selector) {
if (source == null)
throw Error.ArgumentNull("source");
if (selector == null)
throw Error.ArgumentNull("selector");
return source.Provider.CreateQuery<TResult>(
Expression.Call(
null,
GetMethodInfo(Queryable.Select, source, selector),
new Expression[] { source.Expression, Expression.Quote(selector) }
));
}
When the queryable's Provider has to generate actual data from the IQueryable<>, it analyzes the expression tree and tries to figure out how to interpret those method calls. Entity Framework has built-in knowledge of many LINQ-related functions like .Where() and .Select(), so it knows how to translate those method calls into SQL. However, it doesn't know what to do for methods that you write.
So why does this work?
var data = context.Programs.ForUser(userId);
The answer is that your ForUser method is not implemented like the Select method above: you are not adding an expression to the queryable to represent calling ForUser. Instead, you are returning the result of a .Where() call. From the IQueryable<>'s perspective, it's as if Where() was called directly, and the call to ForUser() never happened.
You can prove this by capturing the Expression property on the IQueryable<>:
Console.WriteLine(data.Expression.ToString());
... which will produce something like this:
Programs.Where(u => (u.UserId == value(Helpers<>c__DisplayClass1_0).userId))
There's no call to ForUser() anywhere in that expression.
On the other hand, if you include the ForUser() call inside of an expression tree like this:
var data = context.Programs.Select(d => d.Protocols.ForUser(id));
... then the .ForUser() method never actually gets invoked, so it never returns an IQueryable<> that knows the .Where() method got called. Instead, the expression tree for the queryable shows .ForUser() getting invoked. Outputting its expression tree would look something like this:
Programs.Select(d => d.Protocols.ForUser(value(Repository<>c__DisplayClass1_0).userId))
Entity Framework has no idea what ForUser() is supposed to do. As far as it's concerned, you could have written ForUser() to do something that's impossible to do in SQL. So it tells you that's not a supported method.

As I mentioned in my comment above, I can't tell why the EF Engine is working the way it is. Therefore, I've tried to find a way to re-write the query so I'll be able to make use of my extension methods.
The tables are:
Program -> 1..m -> ProgramProtocol -> m..1 -> Protocol
ProgramProtocol is just a join table and is not mapped in the model by Entity Framework.
The idea is simple: select "from left", select "from right" and then join the resulted sets for proper filtering:
var data = context.Programs.ForUser(userId)
.SelectMany(pm => pm.Protocols,
(pm, pt) => new {pm.ProgramId, pm.ProgramName, pm.ClientId, pt.ProtocolId})
.Join(context.Protocols.ForUser(userId), pm => pm.ProtocolId,
pt => pt.ProtocolId, (pm, pt) => pm)
.GroupBy(pm => new {pm.ProgramId, pm.ProgramName, pm.ClientId})
.Select(d => new MyDataDto
{
ProgramName = d.Key.ProgramName,
ProgramId = d.Key.ProgramId,
ClientId = d.Key.ClientId,
Protocols = d.Count()
})
.ToList();

Combining expression trees

I have the following expression:
public Expression<Func<T, bool>> UserAccessCheckExpression<T>(int userId) where T : class
{
return x => (IsAdmin || userId == CurrentUserId || userId == 0);
}
Then I want to apply this filter to several collections (IQueryable) like this one:
return tasks
.Where(t => t.TaskUsers
.Any(x => UserAccessCheckExpression<TaskUser>(x.User) && x.SomeBool == true));
I'm getting the following error while doing so:
Error 40 Cannot implicitly convert type System.Linq.Expressions.Expression<System.Func<TaskUser,bool>> to bool
I can't use workaround with interface inheritance (like TaskUser inherits interface with int UserId property (where T : IHasUserId)) since I want to combine logic.

The problem is that your UserAccessCheckExpression() method is returning an Expression while the Any() method is expecting a boolean.
Now, you can get your code to compile by compiling the Expression and invoking the method (using UserAccessCheckExpression<TaskUser>(x.User).Compile().Invoke(x.User)) but that would obviously fail on runtime because Linq-to-Entities wouldn't be able to translate your Any() to a store query as it no longer contains an Expression.
LinqKit is aiming to solve this problem using its own Invoke extension method that while letting your code compile, will make sure your Expression will get replaced back to its original form using another extension method named AsExpandable() that is extending the entity set.
Try this:
using LinqKit.Extensions;
return tasks
.AsExpandable()
.Where(t => t.TaskUsers.Any(
x => UserAccessCheckExpression<TaskUser>(x.User).Invoke(x)
&& x.SomeBool == true));
More on LinqKit

Yeah, so, you can't do that. There's a difference between an Expression<> and a Func<>. You're trying to use the UserAccessCheckExpression as a func. I'm not sure what you're trying to do, but you can compile it to a func and then use it sorta like you are:
var expr = UserAccessCheckExpression<TaskUser>(x.User);
var func = expr.Compile();
// Later use it like ...
var result = func();
But I expect you're using this with EF or Linq2Sql? That being the case you'll need to rewrite the expression. It can be done by hand (not easy) or, better, use a tool like PredicateBuilder.

Reuse of a LINQ query

This is not about the reuse of a result but more the statement itself.
Nor is it about an error when using var as mentioned in: LINQ to SQL: Reuse lambda expression
Out of sheer curiosity I was wondering if it is possible to reuse a single LINQ statement.
Lets say I have the following LINQ statement:
.Where(x => x.Contains(""));
Is it possible to extract the statement x => x.Contains("") and use some kind of reference to this for later usage in, lets say, another class?
So I can call it like: .Where(previouslySavedStatement);

You can store it in a variable. If you are working with IQueryable then use:
System.Linq.Expressions.Expression<Func<Foo, bool>> selector = x => x.Contains("");
If you are using IEnumerable then use:
Func<Foo, bool> selector = x => x.Contains("");
And use it in your query:
query.Where(selector);

Yes, you can write a function containing the query you want to reuse, which takes and returns an IQueryable<T>
public IQueryable<T> ContainsEmpty(IQueryable<T> query)
{
return query.Where(x => x.Contains(""));
}
Now you can reuse it:
query1 = ContainsEmpty(query1);
query2 = ContainsEmpty(another);

It depends. There's two Where methods, Enumerable.Where and Queryable.Where. If you're applying the .Where to an IEnumerable than the first one is called, if you're applying it to an IQueryable the second one is called.
Since Enumerable.Where takes in a Func, it isn't reusable. Since Queryable.Where takes in an expression, it is reusable. You can do so as follows:
var x = new List<string>().AsQueryable();
var query = x.Where (n => n.Contains("some string"));
//Extract the lambda clause
var expr = query.Expression;
var methodExpr = (MethodCallExpression)expr;
var quoteExpr = (UnaryExpression)methodExpr.Arguments[1];
var funcExpr = (Expression<Func<string, bool>>)quoteExpr.Operand;
You can then later re-apply the where expression:
var query2 = x.Where(funcExpr);

I wrote a library to address exactly this concern, it's called CLinq and you can find an implementation for the EntityFramework here: https://www.nuget.org/packages/CLinq.EntityFramework
It allows to create query snippets and use them everywhere you in a linq query. Following the example of Hamid, create the following expression:
System.Linq.Expressions.Expression<Func<Foo, bool>> selector = x => x.Contains("");
You can now use this query everywhere in your linq queries like this:
query.AsComposable().Where(o => selector.Pass(o));
Additionally to this simple example you're also able to combine your query snippets:
query.AsComposable().Where(o => selector.Pass(o) || anotherSelector.Pass(o));
or even merge them together:
query.AsComposable().Where(o => anotherSelector.Pass(selector.Pass(o)));
There's some more features, but I think it's really helpful, so check it out :)