Develop Reference

Dynamic Linq where clause throws OutOfMemoryException

I am a novice at Linq and a true beginner with expression trees.
I have a generic expression routine that builds a simple Linq where clause that I found at:
https://www.simple-talk.com/dotnet/net-framework/dynamic-linq-queries-with-expression-trees/
public Func<TSource,bool> SimpleFilter<TSource> (string property, object value)
{
var type = typeof(TSource);
var pe = Expression.Parameter(type, "p");
var propertyReference = Expression.Property(pe,property);
var constantReference = Expression.Constant(value);
var ret = Expression.Lambda<Func<TSource, bool>>
(Expression.Equal(propertyReference, constantReference), new[] { pe });
return ret.Compile();
}
When I call the function as SimpleFilter("JobCustomerID", 449152) it
yields (p => p.JobCustomerId == 449152) which is correct.
If I manually place that criteria in my Linq statement, I get the correct return.
var jj = db.VW_Job_List.Where((p => p.JobCustomerId == 449152));
However when called via the filter function, the Linq throws an OutOfMemoryException.
It is called in my application as:
var jj = db.VW_Job_List.Where(SimpleFilter<VW_Job_List>("JobCustomerID", 449152));
If I call the function with a text criterion, it returns properly:
var jj = db.VW_Job_List.Where(SimpleFilter<VW_Job_List>("CompanyCode", "LCS"));
Is there something specific about using an integer variable that needs to be accommodated? Do I have something coded incorrectly? Any thoughts or insights will be appreciated.

The two calls
var jj = db.VW_Job_List.Where((p => p.JobCustomerId == 449152));
and
var jj = db.VW_Job_List.Where(SimpleFilter<VW_Job_List>("JobCustomerID", 449152));
are not equivalent. The first resolves to Queryable.Where, hence the filter is applied inside the database, while the second - to Enumerable.Where, thus causing loading the whole table in memory and applying the filter there.
The problem is that the return type of your SimpleFilter is Func<TSource, bool>. In order to make them equivalent, it should be Expression<Func<TSource, bool>>. Note that although they look visually the same, there is a huge difference between lambda expression and lambda delegate due to the different overload resolution when applied to IQueryable<T>.
So, change the method like this and try again:
public Expression<Func<TSource,bool>> SimpleFilter<TSource> (string property, object value)
{
var type = typeof(TSource);
var pe = Expression.Parameter(type, "p");
var propertyReference = Expression.Property(pe,property);
var constantReference = Expression.Constant(value);
var ret = Expression.Lambda<Func<TSource, bool>>
(Expression.Equal(propertyReference, constantReference), new[] { pe });
return ret; // No .Compile()
}

How to build a collection filter via expression trees in c#

I'm building a filtering system for UserProfiles based on known properties but unknown (until runtime) combination of filtering conditions.
In my previous question How do I create a generic Expression that has an expression as a parameter, I've figured out a way to have a FilterDefinition for any value property reachable from User entity via navigation properties (i.e. (User)u=> u.NavigationProperty.AnotherNavigationProperty.SomeValue)
and I have a method that can return a predicate as Expression<Func<User,bool>> for a given property, operation ( > < == etc ) and a value.
Now the time has come to filter them on collection properties as well.
Say for example User has CheckedOutBooks collection (which is a total fiction, but will do)
And I need to create a filter definition for Name property of CheckedOutBooks collection on User object.
What I have:
A collection of Users
User class has a collection of Books
now I would like to create a method
Expression<Func<User,bool>> GetPredicate(Expression<User,TProperty>, Operations operation, TProperty value)
That I can call like GetPredicate(u=>u.Books.Select(b=>b.Name), Operations.Contains, "C# in a nutshell")
and get an expression back similar to
u=>u.Books.Any(b=>b.Name == "C# in a nutshell")
I'm thinking maybe it will be easier to split first parameter in two to achieve this.
Maybe u=>u.Books and b=>b.Name will do better?
EDIT:
what I got so far:
class FilterDefinitionForCollectionPropertyValues<T>:FilterDefinition, IUserFilter
{
public Expression<Func<UserProfile, IEnumerable<T>>> CollectionSelector { get; set; }
public Expression<Func<T, string>> CollectionPropertySelector { get; set; }
public Expression<Func<Profile.UserProfile, bool>> GetFilterPredicateFor(FilterOperations operation, string value)
{
var propertyParameter = CollectionPropertySelector.Parameters[0];
var collectionParameter = CollectionSelector.Parameters[0];
// building predicate to supply to Enumerable.Any() method
var left = CollectionPropertySelector.Body;
var right = Expression.Constant(value);
var innerLambda = Expression.Equal(left, right);
Expression<Func<T, bool>> innerFunction = Expression.Lambda<Func<T, bool>>(innerLambda, propertyParameter);
var method = typeof(Enumerable).GetMethods().Where(m => m.Name == "Any" && m.GetParameters().Length == 2).Single().MakeGenericMethod(typeof(T));
var outerLambda = Expression.Call(method, Expression.Property(collectionParameter, typeof(UserProfile).GetProperty("StaticSegments")), innerFunction);
throw new NotImplementedException();
}
}
Now this one works awesomely and does exactly what's needed, now the only thing I need to figure out is how to replace typeof(UserProfile).GetProperty("StaticSegments")) somehow to use CollectionPropertySelector that is in current example would be (UserProfile)u=>u.StaticSegments

You're almost done. Now you just need to do a little trick - wrap your CollectionPropertySelector lambda expression in the CollectionSelector lambda expression.
Expression<Func<TParent,bool>> Wrap<TParent,TElement>(Expression<Func<TParent, IEnumerable<TElement>>> collection, Expression<Func<TElement, bool>> isOne, Expression<Func<IEnumerable<TElement>, Func<TElement, bool>, bool>> isAny)
{
var parent = Expression.Parameter(typeof(TParent), "parent");
return
(Expression<Func<TParent, bool>>)Expression.Lambda
(
Expression.Invoke
(
isAny,
Expression.Invoke
(
collection,
parent
),
isOne
),
parent
);
}
You may have to change this a bit to be used for your particular scenario, but the idea should be clear. My test looked basically like this:
var user = new User { Books = new List<string> { "Book 1", "Book 2" }};
var query = Wrap<User, string>(u => u.Books, b => b.Contains("Bookx"), (collection, condition) => collection.Any(condition));
So you specify the collection selector, predicate and predicate operator, and you're done.
I've written it as a generic method for clarity, but it's dynamic, not strongly typed in essence, so it should be pretty easy to change it to non-generic, if you need that.

Ok I've got it solved for myself.
And I published it to gitHub:
https://github.com/Alexander-Taran/Lambda-Magic-Filters
Given the filter definition class (not reafactored to support properties other than string so far, but will do later):
class FilterDefinitionForCollectionPropertyValues<T>:FilterDefinition, IUserFilter
{
//This guy just points to a collection property
public Expression<Func<UserProfile, IEnumerable<T>>> CollectionSelector { get; set; }
// This one points to a property of a member of that collection.
public Expression<Func<T, string>> CollectionPropertySelector { get; set; }
//This one does the heavy work of building a predicate based on a collection,
//it's member property, operation type and a valaue
public System.Linq.Expressions.Expression<Func<Profile.UserProfile, bool>> GetFilterPredicateFor(FilterOperations operation, string value)
{
var getExpressionBody = CollectionPropertySelector.Body as MemberExpression;
if (getExpressionBody == null)
{
throw new Exception("getExpressionBody is not MemberExpression: " + CollectionPropertySelector.Body);
}
var propertyParameter = CollectionPropertySelector.Parameters[0];
var collectionParameter = CollectionSelector.Parameters[0];
var left = CollectionPropertySelector.Body;
var right = Expression.Constant(value);
// this is so far hardcoded, but might be changed later based on operation type
// as well as a "method" below
var innerLambda = Expression.Equal(left, right);
Expression<Func<T, bool>> innerFunction = Expression.Lambda<Func<T, bool>>(innerLambda, propertyParameter);
// this is hadrcoded again, but maybe changed later when other type of operation will be needed
var method = typeof(Enumerable).GetMethods().Where(m => m.Name == "Any" && m.GetParameters().Length == 2).Single().MakeGenericMethod(typeof(T));
var outerLambda = Expression.Call(method, Expression.Property(collectionParameter, (CollectionSelector.Body as MemberExpression).Member as System.Reflection.PropertyInfo), innerFunction);
var result = Expression.Lambda<Func<UserProfile, bool>>(outerLambda, collectionParameter);
return result;
}
}

Dynamic linq query expression tree for sql IN clause using Entity framework

I want to create a dynamic linq expression for sql IN clause in EF 6.0 with code first approch. Note that i am new to Expressions. What i want to achive is
select * from Courses where CourseId in (1, 2, 3, 4)
//CourseId is integer
The normal linq query looks like this. But i want to query it dynamically
string[] ids = new string[]{"1", "2", "3", "4"};
var courselist = DBEntities.Courses.Where(c => ids.Contains(SqlFunctions.StringConvert((decimal?)c.CourseId)))
There are two ways to make dynamic expression.
1) one ways is to loop through ids and make expressions
The below code will create the following expression in debug view
{f => ((StringConvert(Convert(f.CourseId)).Equals("23") Or StringConvert(Convert(f.CourseId)).Equals("2")) Or StringConvert(Convert(f.CourseId)).Equals("1"))}
Dynamic Expression is
var param = Expression.Parameters(typeof(Course), "f")
MemberExpression property = Expression.PropertyOrField(param, "CourseId");
MethodInfo mi = null;
MethodCallExpression mce = null;
if (property.Type == typeof(int))
{
var castProperty = Expression.Convert(property, typeof(double?));
var t = Expression.Parameter(typeof(SqlFunctions), "SqlFunctions");
mi = typeof(SqlFunctions).GetMethod("StringConvert", new Type[] { typeof(double?) });
mce = Expression.Call(null,mi, castProperty);
}
mi = typeof(string).GetMethod("Equals", new Type[]{ typeof(string)});
BinaryExpression bex = null;
if (values.Length <= 1)
{
return Expression.Lambda<Func<T, bool>>(Expression.Call(mce, mi, Expression.Constant(values[0]), param));
}
//var exp1 = Expression.Call(mce, mi, Expression.Constant(values[0]));
for (int i = 0; i < values.Length; i++)
{
if (bex == null)
{
bex = Expression.Or(Expression.Call(mce, mi, Expression.Constant(values[i])), Expression.Call(mce, mi, Expression.Constant(values[i + 1])));
i++;
}
else
bex = Expression.Or(bex, Expression.Call(mce, mi, Expression.Constant(values[i])));
}//End of for loop
return Expression.Lambda<Func<T, bool>>(bex, param);
2) The 2nd way that i tried (debug view)
{f => val.Contains("23")} //val is parameter of values above
The dynamic expression for above that i tried is
var param = Expression.Parameters(typeof(Course), "f")
MemberExpression property = Expression.PropertyOrField(param, "CourseId");
var micontain = typeof(Enumerable).GetMethods().Where(m => m.Name == "Contains" && m.GetParameters().Length == 2).Single().MakeGenericMethod(typeof(string));
var mc = Expression.Call(micontain, Expression.Parameter(values.GetType(), "val"), Expression.Constant("2"));//NOTE: I haven't use CourseId for now as i am getting conversion error
return Expression.Lambda<Func<T, bool>>(mc, param);
I get the following errors
LINQ to Entities does not recognize the method 'System.String StringConvert(System.Nullable`1[System.Double])' method, and this
method cannot be translated into a store expression when i use the
first methodology. I know i can't use ToString with EF thats why I used SqlFunctions but it is not working for me.
The parameter 'val' was not bound in the specified LINQ to Entities query expression using 2nd methodology
I am trying this from last 4 days. I googled it but didn't find any suitable solution. Please help me.

After a lot of struggle I found solution to my question.
I want to achieve this sql query
select * from Courses where CourseId in (1, 2, 3, 4)
Using Linq to Entities, but I want to pass in(1,2,3,4) list dynamically to linq query. I created an Extension class for that purpose.
public static class LinqExtensions
{
public static Expression<Func<T, bool>> False<T>() { return f => false; }
public static Expression<Func<T, bool>> In<T, TValue>(this Expression<Func<T, bool>> predicate,string propertyName, List<TValue> values)
{
var param = predicate.Parameters.Single();
MemberExpression property = Expression.PropertyOrField(param, propertyName);
var micontain = typeof(List<TValue>).GetMethod("Contains");
var mc = Expression.Call(Expression.Constant(values), micontain, property);
return Expression.Lambda<Func<T, bool>>(mc, param);
}
}
Use of LinqExtensions
var pred = LinqExtensions.False<Course>(); //You can chain In function like LinqExtensions.False<Course>().In<Course, int>("CourseId", inList);
var inList= new List<int>(){1, 2, 3}; //Keep in mind the list must be of same type of the Property that will be compared with. In my case CourseId is integer so the in List have integer values
pred =pred.In<Course, int>("CourseId", inList); //TValue is int. As CourseId is of type int.
var data = MyEntities.Courses.Where(pred);
I hope this might be beneficial for some one

have you seen the type of
var courselist = DBEntities.Courses.Where(c => ids.Contains(c.CourseId)))
above statement would not return actual list of courses. The query is not executed yet. It just returns IQuereable. The query is executed when you actually call .ToList() method on it
so, your solution is..
Create array of IDs using for loop and then simply run the below query
var courselist = DBEntities.Courses.Where(c => ids.Contains(c.CourseId))).ToList()

Build expression tree using Expression.<Factory method> syntax dynamically

I have the following code that builds an IQueryable
Expression<Func<TParentEntity, TChildEntity, bool>> joinExpr =
ctx.GetJoinExpression<TParentEntity, TChildEntity>();
Expression<Func<TChildEntity, bool>> childSelectionExpression =
GetExpression<TChildEntity>(ctx);
return
from parentEntity in ctx.GetQueryable<TParentEntity>()
.AsExpandable()
let childEntities =
from child in ctx.GetQueryable<TChildEntity>()
where joinExpr.Invoke(parentEntity, child)
select child
where childEntities.Any(childSelectionExpression)
select parentEntity;
I've been trying (unsuccesfully) build the expression tree that would return the same result.
I would like to have an expression Expression<Func<TParentEntity,bool> parentExpression
that i could use as following:
from parentEntity in ctx.GetQueryable<TParentEntity>()
.AsExpandable()
where parentExpression.Invoke(parentEntity)
select parentEntity
Does anybody think this is possible? I believe so, but i am unable to solve it...

I know what you want and because I have tries it, I am sorry to inform you that you have arrived in a world of pain. Here is a partial workaround:
Expression<Func<TParentEntity, TChildEntity, bool>> joinExpr =
ctx.GetJoinExpression<TParentEntity, TChildEntity>();
Expression<Func<TChildEntity, bool>> childSelectionExpression =
GetExpression<TChildEntity>(ctx);
return
(from parentEntity in ctx.GetQueryable<TParentEntity>()
.AsExpandable()
let childEntities =
from child in ctx.GetQueryable<TChildEntity>()
where joinExpr.Invoke(parentEntity, child)
select child
select parentEntity).Where(childSelectionExpression);
You have to transform all query calls that use expression variable like I did it. Not always possible but I think in your case it is.

Eureaka :-)
The method at the end of this post returns an expression of type Expression<Func<TParentEntity,bool>>, let's call that one the parentExpression.
When used like this:
var data = (from parentEntity in parentEntities.AsExpandable()
where parentExpression.Invoke(parentEntity) select parentEntiy).ToList()
What happens is actually something like this:
var data = (from parentEntity in parentEntities.AsExpandable()
let childeren = childEntities.Where(c => c.parentId = parentEntity.Id)
where childeren.Any([childSelectionLambda])
select parent).ToList()
By being able to express the child criteria in one Lambda of type
Expression<Func<TParentEntity, bool>>
it is easy to combine multiple child criteria (which are expressed as parent criteria) or other (normal) parent criteria using PredicateBuilder.And().
Tests are green for EF and linq to SQL (and sql ce) :-)
override public Expression<Func<TParentEntity, bool>>
GetParentExpression<TParentEntity>( IDynamicQueryAdapter ctx )
{
// Define parameters needed in expresion tree
ParameterExpression parentParameter =
Expression.Parameter (typeof (TParentEntity), "parent");
ParameterExpression childParameter =
Expression.Parameter (typeof (TChildEntity), "child");
// Define the IQueryable<TChildEntity> as
// a constant for use in the expression tree.
IQueryable<TChildEntity> childDatasource =
ctx.GetQueryable<TChildEntity>().AsExpandable();
ConstantExpression childDatasourceConstant =
Expression.Constant (childDatasource);
// Get MethodInfo instance, needed for the MethodCallExpression
MethodInfo anyMethodInfoChildEntity =
QueryHelper.GetQueryableAnyMethod<TChildEntity> ();
// Get the lambda expression
// required to select only those child entities
// which meet the user defined criteria
Expression<Func<TChildEntity, bool>> childSelectionExpression =
GetExpression<TChildEntity> (ctx);
// Use the ExpressionParameter childParamter for the
// ChildSelectionExpression and strip Expression.Invoke using Expand()
Expression<Func<TChildEntity, bool>> lambda5 =
Expression.Lambda<Func<TChildEntity, bool>> (
Expression.Invoke (childSelectionExpression, childParameter),
childParameter).Expand ();
#region Express the parent child relation
PropertyInfo parentKeyPropertyInfo = null;
PropertyInfo childKeyPropertyInfo = null;
ctx.GetParentChildAssociationProperties (
typeof (TParentEntity), typeof (TChildEntity),
out parentKeyPropertyInfo, out childKeyPropertyInfo);
Expression parentPropertyExpression =
Expression.Property (parentParameter, parentKeyPropertyInfo.Name);
Expression childPropertyExpression =
Expression.Property (childParameter, childKeyPropertyInfo.Name);
if( childKeyPropertyInfo.PropertyType != parentKeyPropertyInfo.PropertyType )
{
// TODO: what if the property types are incomparable >> exception.
// some more code is needed!!
Type nullableParentType =
typeof (Nullable<>)
.MakeGenericType (parentKeyPropertyInfo.PropertyType);
if( childKeyPropertyInfo.PropertyType == nullableParentType )
{
childPropertyExpression =
Expression.Convert (childPropertyExpression,
parentKeyPropertyInfo.PropertyType);
}
else if( childKeyPropertyInfo.PropertyType.IsValueType )
{
Type nullableChildType =
typeof (Nullable<>).MakeGenericType (childKeyPropertyInfo.PropertyType);
if( parentKeyPropertyInfo.PropertyType == nullableChildType )
{
parentPropertyExpression =
Expression.Convert (parentPropertyExpression,
childKeyPropertyInfo.PropertyType);
}
}
}
#endregion
var lambda4 = Expression.Lambda<Func<TChildEntity, bool>> (
Expression.Equal (
parentPropertyExpression,
childPropertyExpression), childParameter );
var predicate = lambda4.And(lambda5).Expand();
Expression<Func<TParentEntity, bool>> parentSelectionExpression =
Expression.Lambda<Func<TParentEntity,bool>>(
Expression.Call (
null,
anyMethodInfoChildEntity,
new Expression[] { childDatasourceConstant, predicate }),
new[]{parentParameter});
return parentSelectionExpression;
}
note: PredicateBuilder, extension AsExpandable(), extenstion Expand() ... are part of LinqKit
For more info about those visit: Linqkit into

Querying Entity with LINQ using Dyanmic Field Name

I have created a dynamic search screen in ASP.NET MVC. I retrieved the field names from the entity through reflection so that I could allow the user to choose which fields they wanted to search on instead of displaying all fields in the view.
When the search result is Posted back to the controller, I receive a FormCollection containing the FieldName and the Value. I don't know how many fields are being searched on, and the FormCollection only contains fields that were chosen by the user.
I want to be able to now take that field name and apply that to my LINQ statement when I query the database for example:
public List<People> SearchPeople(Dictionary<string, string> fieldValueDictionary)
{
List<People> searchResults = new List<People>();
foreach (string key in fieldValueDictionary.Keys)
{
searchResults.Add(entities.People.Where(p => p.<use the key string as the fieldName> == fieldValueDictionary[key]));
}
return searchResults;
}
Where I have "use the key string as the fieldName" it would be like p => p.FirstName == fieldValueDictionary[key] where key = "FirstName". I've tried and failed to use Lambda Expression Trees, and have had a little success with Dynamic LINQ. The only other alternative is to do something like:
public List<People> SearchPeople(Dictionary<string, string> fieldValueDictionary)
{
IQueryable<People> results = entities.People;
foreach (string key in fieldValueDictionary.Keys)
{
switch (k)
{
case "FirstName": results = results.Where(entities.People.Where(p => p.FirstName == k);
case "LastName": results = results.Where(entities.People.Where(p => p.LastName == k);
// Repeat for all 26 fields in table
}
}
return results.ToList<People>();
}
UPDATE: I've done research into Lambda Expression Trees through the following posts:
dynamically create lambdas expressions + linq + OrderByDescending
Parameter problem with Expression.Lambda()
LINQ: Passing lambda expression as parameter to be executed and returned by method
I've gotten as far as getting a lambda to output the following: "p => p.FirstName", but I can't get this to work in a where. Any Suggestions? My code is below:
MemberInfo member = typeof(People).GetProperty("FirstName");
ParameterExpression cParam = Expression.Parameter(typeof(People), "p");
Expression body = Expression.MakeMemberAccess(cParam, member);
var lambda = Expression.Lambda(body, cParam);

After a lot more trial and error and searching I accidentally found another SO post that covers the same issue:
InvalidOperationException: No method 'Where' on type 'System.Linq.Queryable' is compatible with the supplied arguments
Here is my modified code that works:
IQueryable query = entities.People;
Type[] exprArgTypes = { query.ElementType };
string propToWhere = "FirstName";
ParameterExpression p = Expression.Parameter(typeof(People), "p");
MemberExpression member = Expression.PropertyOrField(p, propToWhere);
LambdaExpression lambda = Expression.Lambda<Func<People, bool>>(Expression.Equal(member, Expression.Constant("Scott")), p);
MethodCallExpression methodCall = Expression.Call(typeof(Queryable), "Where", exprArgTypes, query.Expression, lambda);
IQueryable q = query.Provider.CreateQuery(methodCall);
With some hopefully pretty easy modifications, I should be able to get this to work with any type.
Thanks again for your answers Ani & John Bowen

Have you tried getting the value from PropertyInfo?
entities.People.Where(p => (p.GetType().GetProperty(key).GetValue(p, null) as string) == fieldValueDictionary[key])

public List<People> SearchPeople(Dictionary<string, string> fieldValueDictionary)
{
return !fieldValueDictionary.Any()
? entities.People
: entities.People.Where(p => fieldValueDictionary.All(kvp => PropertyStringEquals(p, kvp.Key, kvp.Value)))
.ToList();
}
private bool PropertyStringEquals(object obj, string propertyName, string comparison)
{
var val = obj.GetType().GetProperty(propertyName).GetValue(obj, null);
return val == null ? comparison == null : val.ToString() == comparison; ;
}