Develop Reference

Inner method in Linq query that return IQueryable

I would like to do following thing in Entity Framework code first:
Data.GroupBy(x => x.Person.LastName)
.Select(x => x.OtherGrouping(...)).Where(...).GroupBy(...)...etc
public static class Extensions
{
public static IQueryable<IGrouping<T, T>> OtherGrouping<T>(this IQueryable<T> collection /**** Here I want to pass consts and lambdas\expressions*****/)
{
// Grouping, filtration and other stuff here
return collection.GroupBy(x => x);
}
}
The problem is .net compiles OtherGrouping method, it doesn't represent like an expression, and couldn't be transformed to the SQL.
I found LinqKit library which suppose to help in such cases, but I coudn't figure out how to apply it in my specific case. It works fine for simple cases, when I just have expression like x => x+2, but I get stucked with return IQueryable. Probably it is possible to write expression tree completely by hand but I don't want to go so deep.
Any ideas how it might be done or where I can read about it?
Here is what I tried to do based on Rob's comment
void Main()
{
AddressBases.GroupBy(x => x.ParentId).Select(x => new {x.Key, Items = x.AsQueryable().OtherGrouping(a => a.Country) }).Dump();
}
public static class Extensions
{
public static IQueryable<IGrouping<TKey, TSource>> OtherGrouping<TSource, TKey>(this IQueryable<TSource> source, Expression<Func<TSource, TKey>> keySelector)
{
return source.Provider.CreateQuery<IGrouping<TKey, TSource>>(
source.GroupBy(keySelector).Expression);
}
}
And I got an exception:
NotSupportedException: LINQ to Entities does not recognize the method 'System.Linq.IQueryable`1[System.Linq.IGrouping`2[System.String,InsuranceData.EF.AddressBase]] OtherGrouping[AddressBase,String](System.Linq.IQueryable`1[InsuranceData.EF.AddressBase], System.Linq.Expressions.Expression`1[System.Func`2[InsuranceData.EF.AddressBase,System.String]])' method, and this method cannot be translated into a store expression.
Couldn't figure out yet why I have OtherGrouping method in expression tree? OtherGrouping method should just attach another grouping to the expression and pass it to the provider, but not put itself to the tree.

Your current code after the edit is correct - where you're going wrong is a common issue with the syntactic sugar we're given by C#.
If you write the following:
void Main()
{
var res = Containers.OtherGrouping(c => c.ContainerID);
res.Expression.Dump();
res.Dump();
}
public static class Extensions
{
public static IQueryable<IGrouping<TKey, TSource>> OtherGrouping<TSource, TKey>(this IQueryable<TSource> source, Expression<Func<TSource, TKey>> keySelector)
{
return source.Provider.CreateQuery<IGrouping<TKey, TSource>>(
source.GroupBy(keySelector).Expression
);
}
}
You'll see that we get the output:
Table(Container).GroupBy(c => c.ContainerID)
And the result executes with an issue, properly grouping by our predicate. However, you're invoking OtherGrouping while inside an expression, when you're passing it to Select. Let's try a simple case without OtherGrouping:
var res = Containers.OtherGrouping(c => c.ContainerID)
.Select(x => new
{
x.Key,
Items = x.GroupBy(c => c.ContainerID),
});
res.Expression.Dump();
What you're providing to Select is an expression tree. That is, the inner GroupBy's method is never actually invoked. If you change the query to x.AsQueryable().OtherGrouping and put a breakpoint in OtherGrouping, it will only be hit the first time.
In addition to that, x is IEnumerable<T>, not IQueryable<T>. Invoking AsQueryable() gives you an IQueryable, but it also gives you a new query provider. I'm not 100% sure as to the inner workings of entity framework, but I'd wager to say that this is invalid - as we no longer have the database as a target for the provider. Indeed, with linq2sql (using LINQPad), we get an error saying .AsQueryable() is not supported.
So, what can you do? Unfortunately there's no clean way to do this. Since the expression tree is already built before OtherGrouping has a chance, it doesn't matter what the body of OtherGrouping is. We'll need to change the expression tree after it's built, but before it's executed.
For that, you'll need to write an expression visitor which will look for .OtherGrouping expression calls, and replace it with Queryable.GroupBy

How to build a simple property selector expression in ef6

How can I create a property selector for entity framework like this?
public static List<T> StandardSearchAlgorithm<T>(this IQueryable<T> queryable, Func<T, string> property, string query)
{
return queryable.Where(e => property(e).ToLower().IndexOf(query) > -1).ToList();
}
I want the calling code to be able to be clean and simple like this:
var usernameResults = _db.Users.StandardSearchAlgorithm(u => u.Username, query);
I get a "The LINQ expression node type 'Invoke' is not supported in LINQ to Entities." error. I cannot work out how to get the expression built.
UPDATE:
Based on the answer by MBoros here is the code I ended up with. It works great.
The key to expression trees is to understand expression trees are all about breaking up what you normally write in code (like "e => e.Username.IndexOf(query)") into a series of objects: "e" gets its own object, "Username" its own object, "IndexOf()" its own object, the "query" constant its own object, and so on. The second key is to know that you can use a series of static methods on the Expression class to create various kinds of these objects, as shown below.
PropertyInfo pinfo = (PropertyInfo)((MemberExpression)property.Body).Member;
ParameterExpression parameter = Expression.Parameter(typeof(T), "e");
MemberExpression accessor = Expression.Property(parameter, pinfo);
ConstantExpression queryString = Expression.Constant(query, typeof(string));
ConstantExpression minusOne = Expression.Constant(-1, typeof(int));
MethodInfo indexOfInfo = typeof(string).GetMethod("IndexOf", new[] { typeof(string) }); // easiest way to do this
Expression indexOf = Expression.Call(accessor, indexOfInfo, queryString);
Expression expression = Expression.GreaterThan(indexOf, minusOne);
Expression<Func<T, bool>> predicate = Expression.Lambda<Func<T, bool>>(expression, parameter);
//return predicate.Body.ToString(); // returns "e => e.Username.IndexOf(query) > -1" which is exactly what we want.
var results = queryable.Where(predicate).ToList();
return results;
Now I have a real problem, but I will ask it in a separate question. My real query looks like this:
public static List<T> StandardSearchAlgorithm<T>(this IQueryable<T> queryable, Func<T, string> property, string query)
{
return queryable.Where(e => property(e).IndexOf(query) > -1).Select(e=> new { Priority = property(e).IndexOf(query), Entity = e } ).ToList();
}
So I need to build an expression that returns an Anonymous Type!! Or even if I create a class to help, I need to write an expression that returns a new object. But I will include this in a separate question.

You cannot invoke CLR delegates so simply in sql. But you can pass in the property selector as an Expression tree., so your signature would be:
public static List<T> StandardSearchAlgorithm<T>(this IQueryable<T> queryable, Expression<Func<T, string>> property, string query)
Calling would look the same. But now that you have an expression in your hand, you can have a look at this answer:
Pass expression parameter as argument to another expression
It gives you the tools to simply put an expression tree inside another one. In your case it would look like:
Expression<Func<T, bool>> predicate = e => property.AsQuote()(e).Contains(query);
predicate = predicate.ResolveQuotes();
return queryable.Where(predicate).ToList();
Once you are there, you still have the .ToLower().Contains() calls (use .Contains instead of the .IndexOf()> 1). This is actually tricky. Normally the db uses its default collation, so if it set to CI (case insensitive), then it will do the compare that way. If you don't have any constraints, and can adjust the db collation, I would go for that. In this case you can omit the .ToLower() call.
Otherwise check out this anser: https://stackoverflow.com/a/2433217/280562

Combine two Linq lambda expressions

Expression<Func<MyObject, string>> fn1 = x => x.PossibleSubPath.MyStringProperty;
Expression<Func<string, bool>> fn2 = x => x.Contains("some literal");
Is there a way to create a new lambda expression which basically uses the output of fn1 and uses it as input for fn2?
Expression<Func<MyObject, bool>> fnCombined = ...
I know that I can create the function at once, but the problem is that I'm making some generic code and therefore really need to be able to create these two functions separately, then combine them in such a way that Linq can use them on my database objects (Entity Framework).

So logically what we want to be able to do is create a new lambda in which it has a parameter of the input to the first function, and a body that calls the first function with that parameter and then passes the result as the parameter to the second function, and then returns that.
We can replicate that easily enough using Expression objects:
public static Expression<Func<T1, T3>> Combine<T1, T2, T3>(
Expression<Func<T1, T2>> first,
Expression<Func<T2, T3>> second)
{
var param = Expression.Parameter(typeof(T1), "param");
var body = Expression.Invoke(second, Expression.Invoke(first, param));
return Expression.Lambda<Func<T1, T3>>(body, param);
}
Sadly, EF and most other query providers won't really know what to do with that and won't function properly. Whenever they hit an Invoke expression they generally just throw an exception of some sort. Some can handle it though. In theory all the information they need is there, if they're written with the robustness to get at it.
What we can do however is, from a conceptual standpoint, replace every instance of the first lambda's parameter in that lambda's body with the parameter of a new lambda we're creating, and then replace all instances of the second lambda's parameter in the second lambda with the new body of the first lambda. Technically, if these expressions have side effects, and these parameters are used more than once, they wouldn't be the same, but as these are going to be parsed by an EF query provider they really shouldn't ever have side effects.
Thanks to David B for providing a link to this related question which provides a ReplaceVisitor implementation. We can use that ReplaceVisitor to go through the entire tree of an expression and replace one expression with another. The implementation of that type is:
class ReplaceVisitor : ExpressionVisitor
{
private readonly Expression from, to;
public ReplaceVisitor(Expression from, Expression to)
{
this.from = from;
this.to = to;
}
public override Expression Visit(Expression node)
{
return node == from ? to : base.Visit(node);
}
}
And now we can write our proper Combine method:
public static Expression<Func<T1, T3>> Combine<T1, T2, T3>(
this Expression<Func<T1, T2>> first,
Expression<Func<T2, T3>> second)
{
var param = Expression.Parameter(typeof(T1), "param");
var newFirst = new ReplaceVisitor(first.Parameters.First(), param)
.Visit(first.Body);
var newSecond = new ReplaceVisitor(second.Parameters.First(), newFirst)
.Visit(second.Body);
return Expression.Lambda<Func<T1, T3>>(newSecond, param);
}
and a simple test case, to just demonstrate what's going on:
Expression<Func<MyObject, string>> fn1 = x => x.PossibleSubPath.MyStringProperty;
Expression<Func<string, bool>> fn2 = x => x.Contains("some literal");
var composite = fn1.Combine(fn2);
Console.WriteLine(composite);
Which will print out:
param => param.PossibleSubPath.MyStringProperty.Contains("some literal")
Which is exactly what we want; a query provider will know how to parse something like that.

How does PredicateBuilder work

C# in a Nutshell has a free class called PredicateBuilder which constructs LINQ predicates piece by piece available here. Here's an extract of the method which adds a new expression to the predicate. Could someone explain it? (I have seen this question, I don't want a general answer like there. I am looking for a specific explanation of how Expression.Invoke and Expression.Lambda build the new expression).
public static Expression<Func<T, bool>> And<T> (this Expression<Func<T, bool>> expr1,
Expression<Func<T, bool>> expr2)
{
var invokedExpr = Expression.Invoke (expr2, expr1.Parameters.Cast<Expression> ());
return Expression.Lambda<Func<T, bool>>
(Expression.AndAlso (expr1.Body, invokedExpr), expr1.Parameters);
}

Let's say you have:
Expression<Func<Person, bool>> isAdult = p1 => p1.Age >= 18;
// I've given the parameter a different name to allow you to differentiate.
Expression<Func<Person, bool>> isMale = p2 => p2.Gender == "Male";
And then combine them with PredicateBuilder
var isAdultMale = isAdult.And(isMale);
What PredicateBuilder produces is an expression that looks like this:
// Invoke has no direct equivalent in C# lambda expressions.
p1 => p1.Age >= 18 && Invoke(p2 => p2.Gender == "Male", p1)
As you can see:
The resulting lambda reuses the parameters of the first expression.
Has a body that invokes the second expression by passing the parameters of the first expression as a replacement for the second expression's parameters. The resulting InvocationExpression is sort of like the expression-equivalent of a method-call (calling a routine by passing in arguments for parameters).
Ands the first expression's body and this InvocationExpression together to produce the body of the resulting lambda.
The idea is that the LINQ provider should be able to understand the semantics of this operation and take a sensible course of action (e.g. generate SQL like WHERE age >= 18 AND gender = 'Male').
Often though, providers have problems with InvocationExpressions, because of the obvious complications of processing a 'nested expression-call inside an expression.'
To get around this, LINQKit also provides the Expand helper. This essentially 'inlines' the invocation call smartly by replacing the call with the body of the nested expression, substituting uses of the nested expression's parameters appropriately (in this case, replacing p2 with p1). This should produce something like:
p1 => p1.Age >= 18 && p1.Gender == "Male"
Note that this how you would have manually combined those predicates if you'd done it yourself in a lambda. But with LINQKit around, you can get these predicates from independent sources and easily combine them:
Without writing "by hand" expression code.
Optionally, in a way that is transparent to consumers of the resulting lambda.

System.Linq Expressions evaluating true on a single entity

I'm trying to DRY out some lambda expressions for security rights. Is it possible to take a lamda expression and apply it to a single entity for true?
Like lets say I have a Person and a DocumentFolder
Expression<Func<Person, bool>> CanSeePerson()
{
return c => !c.IsPrivate;
}
And one for the folder
Expression<Func<DocumentFolder, bool>> CanSeeFolder()
{
return c => !c.IsPrivate && c.Owner.CanSeePerson(); // <- ???
}
How the heck can I use that CanSeePerson() function on a single type to return true and maintain an expression that can be used in linq queries like such
Entities.DocumentFolder.Where(CanSeeFolder());
I know how to use the where on an iqueryable but I can't see how to apply the expression tree to a single value.
This throws the error: Unable to create a constant value of type 'Person'. Only primitive types ('such as Int32, String, and Guid') are supported in this context.
Expression<Func<DocumentFolder, bool>> CanSeeFolder()
{
return c => !c.IsPrivate &&_entities.Persons.Where(x => x.Id == c.Owner.Id).Any(CanSeePerson());
}
The difference seems to be based on putting the IQueryable Directly in the statement. This also does NOT work
Expression<Func<DocumentFolder, bool>> CanSeeFolder()
{
return c => !c.IsPrivate &&_entities.Persons.Where(CanSeePerson()).Contains(c.User);
}
but this DOES work
Expression<Func<DocumentFolder, bool>> CanSeeFolder()
{
var canSeePersons = _entities.Persons.Where(CanSeePerson());
return c => !c.IsPrivate && canSeePersons.Contains(c.User);
}
p.s. I know I suck # using this stackoverflow formatting thing lol
The reason why this works is because the CanSeePerson() function cannot be converted and used in an expression. When you put the canSeePersons variable in the function instead you are placing in an iQueryable type which can be used in an expression. Using "var" convolutes it a bit.

You defined a Expression<Func<Person, bool>> that solves your problem. The trick is to simply transform your list of objects to the list you want to filter. It is not always possible, but many times it is. You just have to be a bit creative :-)
When using extension methods you can easily come up with a solution that allows you to do this:
var visibleFolders = Entities.DocumentFolder.WhereCanSeeFolder();
Here is the (completely DRY) solution:
public static class SecurityExtensions
{
public static IQueryable<DocumentFolder> WhereCanSeeFolder(
this IQueryable<DocumentFolder> folders)
{
var visibleOwners = folders.Select(f => f.Owner)
.Where(CanSeePerson);
return
from folder in folders.Where(CanSeeFolder)
where visibleOwners.Contains(folder.Owner)
select folder;
}
private static readonly Expression<Func<DocumentFolder, bool>>
CanSeeFolder = folder => !folder.IsPrivate;
private static readonly Expression<Func<Person, bool>>
CanSeePerson = person => !person.IsPrivate;
}
I hope this helps.

The expression tree is only applied to a single value at a time, logically, within the LINQ expression.
If you're saying you want to apply it in-process later to a single value, you can just use:
// This can be cached
Func<DocumentFolder, bool> canSeeFolderDelegate = CanSeeFolder().Compile();
DocumentFolder folder = ...; // Get the value from somewhere
if (canSeeFolderDelegate(folder))
{
// Yes, you can see that folder
}

Ah, you want to evaluate it for a single entity?
var func = CanSeeFolder().Compile(); // <=== store and re-use this;
// this isn't free
bool canSee = func(obj);
Another approach might be:
bool canSee = Enumerable.Repeat(obj, 1).AsQueryable().Any(CanSeeFolder());
but this is probably still going to do the Compile somewhere in the chain, so you may as well use the more direct code (at the top).
Edit re comments:
To evaluate that at the database, you would need a restriction, for example:
bool canSee = db.Folders.Where(f => f.FolderId == id)
.Where(CanSeeFolder()).Any();
which limits us to the single row, then adds your extra filter.