I'm trying to build a dynamic LINQ query, in Entity Framework, from a user-provided set of collection criteria. Eventually, this will include more complex behaviors, but currently I just have a list of field names and values, and I want to return all the records in which the field names have those values.
My basic structure is this:
public IEnumerable<ThingViewModel> getMythings(SelectionCriteria selectionCriteria)
{
var predicate = constructPredicate<Thing>(selectionCriteria);
var things = this.dbContext.Things.Where(predicate).ToList();
return Mapper.Map<List<Thing>, List<ThingViewModel>>(things);
}
Where all the interesting work is in constructPredicate():
private static Expression<Func<T, bool>> constructPredicate<T>(SelectionCriteria selectionCriteria)
{
// using Pete Montgomery's PredicateBuilder:
// http://petemontgomery.wordpress.com/2011/02/10/a-universal-predicatebuilder/
var predicate = PredicateBuilder.True<T>();
foreach (var item in selectionCriteria.andList)
{
// Accessing foreach values in closures can result in unexpected results.
// http://stackoverflow.com/questions/14907987/access-to-foreach-variable-in-closure
var fieldName = item.fieldName;
var fieldValue = item.fieldValue;
var parameter = Expression.Parameter(typeof (T), "t");
var property = Expression.Property(parameter, fieldName);
var value = Expression.Constant(fieldValue);
var lambda = buildCompareLambda<T>(property, value, parameter);
predicate = predicate.And(lambda);
}
return predicate;
}
All of that seems like a perfectly reasonable structure, but it's in the buildCompareLambda() that I'm having difficulties. I'm not seeing a way to do this in a generic way, I'm having to create different methods for different types. I'd started by handling strings, and that was simple enough. I next tried to handle integers, but it turns out that the integer fields in the database are nullable, which introduced a whole new class of complexity.
My buildCompareLambda(), so far:
private static Expression<Func<T, bool>> buildCompareLambda<T>(
MemberExpression property,
ConstantExpression value,
ParameterExpression parameter)
{
Expression<Func<T, bool>> lambda = null;
if (property.Type == typeof (string))
lambda = buildStringCompareLambda<T>(property, value, parameter);
else if (property.Type.IsGenericType && Nullable.GetUnderlyingType(property.Type) != null)
lambda = buildNullableCompareLambda<T>(property, value, parameter);
if (lambda == null)
throw new Exception(String.Format("SelectrionCriteria cannot handle property type '{0}'", property.Type.Name));
return lambda;
}
As I said, buildStringCompareLambda is simple enough:
private static Expression<Func<T, bool>> buildStringCompareLambda<T>(
MemberExpression property,
ConstantExpression value,
ParameterExpression parameter)
{
var equalsMethod = typeof (string).GetMethod("Equals",
new[] {typeof (string), typeof (string)});
var comparison = Expression.Call(equalsMethod, property, value);
return Expression.Lambda<Func<T, bool>>(comparison, parameter);
}
But buildNullableCompareLambda() is getting ugly:
private static Expression<Func<T, bool>> buildNullableCompareLambda<T>(MemberExpression property,
ConstantExpression value,
ParameterExpression parameter)
{
var underlyingType = Nullable.GetUnderlyingType(property.Type);
if (underlyingType == typeof (int) || underlyingType == typeof (Int16) || underlyingType == typeof (Int32) ||
underlyingType == typeof (Int64) || underlyingType == typeof (UInt16) || underlyingType == typeof (UInt32) ||
underlyingType == typeof (UInt64))
{
var equalsMethod = underlyingType.GetMethod("Equals", new[] {underlyingType});
var left = Expression.Convert(property, underlyingType);
var right = Expression.Convert(value, underlyingType);
var comparison = Expression.Call(left, equalsMethod, new Expression[] {right});
return Expression.Lambda<Func<T, bool>>(comparison, parameter);
}
return null;
}
It's my intent to add support for more types, in buildNullableCompareLambda(), and to move the handling of each type into a function, so that the same code can be called from buildCompareLambda() and from buildNullableCompareLambda(). But that's for the future - currently I'm stuck on comparing ints. Currently, I'm converting both the property and the value to the underlying type, since I don't want to have separate functions for each integer type, and I don't want the user to have to care whether the EF models a field into an Int16 or an Int32. And that's working, for non-null fields.
I've been browsing around SO, and finding some answers, which is how I've gotten as far as I have, but none of the answers I've seen on handling nullable types really work for me, because they don't really handle nulls.
In my case, if the user passes me a selection criteria with an item that is supposed to equal null, I would want to return the records for which that field are null, and this bit about converting both sides to a base type doesn't seem to work. I'm getting an "Object reference not set to an instance of an object" exception.
In SQL, what I'd want is a "WHERE field IS NULL", if the value is null, or "WHERE field = 'value'", if it is not. And I'm not seeing how to build that kind of alternative into an expression tree.
Any ideas?
Added: It has been suggested that I use Expression.Equal().
With that, my loop becomes:
private static Expression<Func<T, bool>> constructPredicate<T>(SelectionCriteria selectionCriteria)
{
var predicate = PredicateBuilderEx.True<T>();
var foo = PredicateBuilder.True<T>();
foreach (var item in selectionCriteria.andList)
{
var fieldName = item.fieldName;
var fieldValue = item.fieldValue;
var parameter = Expression.Parameter(typeof (T), "t");
var property = Expression.Property(parameter, fieldName);
var value = Expression.Constant(fieldValue);
var comparison = Expression.Equal(property, value);
var lambda = Expression.Lambda<Func<T, bool>>(comparison, parameter);
predicate = PredicateBuilderEx.And(predicate, lambda);
}
return predicate;
}
And that doesn't work. I get an exception:
The binary operator Equal is not defined for the types
'System.Nullable`1[System.Int16]' and 'System.Int16'.
As is quite often the case, folks here may not quite come up with the answer, but they get most of the way, and close enough that I can work out the rest.
Expression.Equal requires both parameters to be of the same type. If one is nullable, they both need to be nullable. But that's not that hard to deal with:
private static Expression<Func<T, bool>> constructPredicate<T>(SelectionCriteria selectionCriteria)
{
var predicate = PredicateBuilderEx.True<T>();
var foo = PredicateBuilder.True<T>();
foreach (var item in selectionCriteria.andList)
{
var fieldName = item.fieldName;
var fieldValue = item.fieldValue;
var parameter = Expression.Parameter(typeof (T), "t");
var property = Expression.Property(parameter, fieldName);
var value = Expression.Constant(fieldValue);
var converted = Expression.Convert(value, property.Type);
var comparison = Expression.Equal(property, converted);
var lambda = Expression.Lambda<Func<T, bool>>(comparison, parameter);
predicate = PredicateBuilderEx.And(predicate, lambda);
}
return predicate;
}
Thanks, all.
As Lee states in his comment, you don't need to implement buildNullableCompareLambda<T> for each type. There is already a method that checks the types of the left and right expressions and calls the Equals method on them if they are a user-defined type, and does the lifting and proper comparison if they are nullable types. See here.
Your lambda is basically:
var property = Expression.Property(parameter, fieldName);
var value = Expression.Constant(fieldValue);
var lambda = Expression.Equal(property, value);
Edit:
It seems to me that this is a bug. Eric Lippert thinks so(link). The documentation states the scenario where they are both of the same type and what to do then:
If left.Type and right.Type are both non-nullable, the node is not
lifted. The type of the node is Boolean. If left.Type and right.Type
are both nullable, the node is lifted. The type of the node is
Boolean.
It doesn't exactly what would happen if one is nullable and the other isn't. In the same link referenced, Eric gives a workaround.
Related
There's the problem I made an Extension method to IQueryable and I want to get property name and value to do a select on it the thing is that I am using Expression tree and try to search rows by the use of Contain() method which is needs a parameter of type string to do the job. my code is doing well while I do the search over string properties but I want this method to do the search on other types like int, decimal, datetime and even booleans so I need to get the property cast it to string and search for the matching values.
what I tried so far is:
private static readonly MethodInfo _tostring = typeof(Object).GetMethod("ToString") ?? throw new Exception("Cannot create Method");
public static IQueryable<T> Search<T>(this IQueryable<T> items, string propertyName, string filterValue)
{
MethodInfo _compare =
(((Expression<Func<string, bool>>)(s => s.Contains("aa"))).Body as MethodCallExpression ?? throw new Exception("Cannot Create Method"))
.Method;
var property = typeof(T).GetProperty(propertyName) ?? throw new Exception("Couldn't Get the property");
var row = Expression.Parameter(typeof(T), "row");
Expression prop = Expression.Property(row, property);
// now making sure if the type is string
if (property.PropertyType != typeof(string))
{
//Here I want to cast it to string but the problem is exactly here anything I try feels like a dead end
prop = Expression.Call(prop, _tostring);
}
var func =
Expression.Lambda<Func<T, bool>>
(
Expression.Call
(
prop,
_compare,
Expression.Constant(filterValue)
),
row
);
return items.Where(func);
}
I tried and tested it with some dummy objects; it seems to work.
I refined my answer to use ToString
var prop = Expression.Property(row, property);
Expression expression = prop;
if (prop.Type != typeof(string))
{
expression = Expression.Call(prop, "ToString", Type.EmptyTypes);
}
var func = Expression.Lambda<Func<T, bool>>
(
Expression.Call
(
expression,
_compare,
Expression.Constant(filterValue)
),
row
);
The above expression tree will be compiled to,
If the property type is not a string,
.Where(s => s.property.ToString().Contains("filter")))
If it's a string type,
.Where(s => s.property.Contains("filter")))
I'm trying to write a dynamic query but having trouble when dealing with dynamic types in the OrderBy section of my query. I've seen some solutions using method call expressions, which may work for me, but I don't know how to fit that into my existing code where I am using method chaining on either side. It's possible I can just use dynamic as my type but I don't think that will work with Nullable types.
public async Task<IEnumerable<MyEntityModel>> GetEntities(QueryEntityResource request)
{
IQueryable<MyEntityModel> queryableData = ...;
Expression whereLambdaExp = ...;
queryableData = queryableData.Where(whereLambdaExp);
ParameterExpression param = Expression.Parameter(typeof(MyEntityModel));
PropertyInfo property = typeof(MyEntityModel).GetProperty(request.sortModel.ColId);
Expression propExpr = Expression.Property(param, property);
var lambdaExpression = Expression.Lambda<Func<MyEntityModel, dynamic>>(propExpr , new ParameterExpression[] { param }); // can't use dynamic
// Errors here; can't use dynamic type in lambda expression -- Need to specify types
if (request.sortModel.Sort == "asc")
{
queryableData = queryableData.OrderBy(lambdaExpression);
}
if (request.sortModel.Sort == "desc")
{
queryableData = queryableData.OrderByDescending(lambdaExpression);
}
queryableData = queryableData.Skip(request.StartRow).Take(request.EndRow - request.StartRow);
return queryableData.ToListAsync();
}
If I use dynamic it won't work with my nullable types:
System.ArgumentException: Expression of type 'System.Nullable`1[System.Int32]' cannot be used for return type 'System.Object'
This is untested; but I think it is close to what you need (I feel like I've written this code before... also, see this answer):
You can build calls to OrderBy instantiated to the types you need and then build a new IQueryable. My method removes the async as I couldn't build a quick test for that in LINQPad.
public IEnumerable<MyEntityModel> GetEntities(QueryEntityResource request) {
IQueryable<MyEntityModel> queryableData = default;
Expression<Func<MyEntityModel, bool>> whereLambdaExp = default;
queryableData = queryableData.Where(whereLambdaExp);
var param = Expression.Parameter(typeof(MyEntityModel));
var propExpr = Expression.Property(param, request.sortModel.ColId);
var lambdaExpression = Expression.Lambda(propExpr, param);
if (request.sortModel.Sort == "asc" || request.sortModel.Sort == "desc")
queryableData = queryableData.Provider.CreateQuery<MyEntityModel>(Expression.Call(typeof(Queryable), request.sortModel.Sort == "asc" ? "OrderBy" : "OrderByDescending",
new[] { typeof(MyEntityModel), propExpr.Type }, queryableData.Expression, Expression.Quote(lambdaExpression)));
queryableData = queryableData.Skip(request.StartRow).Take(request.EndRow - request.StartRow);
return queryableData.ToList();
}
I have a nullable datetime field and I have to convert the string date field to a nullable datetime type (using Expression)....I did this using the below.
Expression.Constant(Convert.ChangeType(value, Nullable.GetUnderlyingType(memberAccess.Type)));.
The memberAccess (mentioned above) is of type member expression. (From LinqExtensions.cs)
Now In the code I am using Expression.Equal Method.
Expression.Equal(memberAccess, filter);
This fails here, as memberaccess type is nullable but filter.type is not nullable...
Even if I try to convert the member access type to nullable using
ConstantExpression test = Expression.Constant(Nullable.GetUnderlyingType(memberAccess.Type)),
the Type is Runtime and not DateTime.
How to use Expression.Equal to compare nullable & non nullable field? IS there any way to convert the string type to a nullable datetime field? Either one of this will resolve my issue.
Ok..I did this way.
First Converted the type (string to datetime)
filter = Expression.Constant(
Convert.ChangeType(value, memberAccess.Type.GetGenericArguments()[0]));
then converted this expression to the desired type
Expression typeFilter = Expression.Convert(filter, memberAccess.Type);
Then used Expression.Equal(memberAccess, typeFilter)...
(memberAccess is MemberExpression and it takes the property type from model)
If you have values Nullable values other than dates, This is how You can create an Expression tree for nullable types, suppose you have a nullable field BoardId, you can create expression tree dynamically like this
var nameValue="BoardId=111";
public static Expression<Func<T, bool>> BuildWhereExpression<T>(string nameValueQuery ) where T : class
{
Expression<Func<T, bool>> predicate = null;
PropertyInfo prop = null;
var fieldName = nameValueQuery.Split("=")[0];
var fieldValue = nameValueQuery.Split("=")[1];
var properties = typeof(T).GetProperties();
foreach (var property in properties)
{
if (property.Name.ToLower() == fieldName.ToLower())
{
prop = property;
}
}
if (prop != null)
{
var isNullable = prop.PropertyType.IsNullableType();
var parameter = Expression.Parameter(typeof(T), "x");
var member = Expression.Property(parameter, fieldName);
if (isNullable)
{
var filter1 =
Expression.Constant(
Convert.ChangeType(fieldValue, member.Type.GetGenericArguments()[0]));
Expression typeFilter = Expression.Convert(filter1, member.Type);
var body = Expression.Equal(member, typeFilter);
predicate = Expression.Lambda<Func<T, bool>>(body, parameter);
}
else
{
if (prop.PropertyType == typeof(string) && likeOerator.ToLower() == "like")
{
var parameterExp = Expression.Parameter(typeof(T), "type");
var propertyExp = Expression.Property(parameterExp, prop);
MethodInfo method = typeof(string).GetMethod("Contains", new[] { typeof(string) });
var someValue = Expression.Constant(fieldValue, typeof(string));
var containsMethodExp = Expression.Call(propertyExp, method, someValue);
predicate = Expression.Lambda<Func<T, bool>>(containsMethodExp, parameterExp);
}
else
{
var constant = Expression.Constant(Convert.ChangeType(fieldValue, prop.PropertyType));
var body = Expression.Equal(member, constant);
predicate = Expression.Lambda<Func<T, bool>>(body, parameter); `enter code here`
}
}
}
return predicate;
}
1- This Solution first checks for the Nullable value and generate the expression.
This is How you can determine if the type is Nullable. I have created an extension method for that purpose
public static bool IsNullableType(this Type type)
{
return type.IsGenericType && (type.GetGenericTypeDefinition().Equals(typeof(Nullable<>)));
}
2- the second step is to check the type if its string then create an expression for a string.
3- the Third step is to check is value is not nullable not string then create an expression using equal
You should be using DateTime.Parse or, better, DateTime.ParseExact to convert your string to date. So, Expression.Call.
If you want to convert it to null date if the string is null, you can use Expression.Condition.
If you want to make the result nullable, I believe Expression.Convert can do that.
So something like (pseudocode):
Condition(
Equals(yourStringExpression, null),
Constant(null, typeof(DateTime?)),
Convert(
Call(DateTime.ParseExact, yourStringExpression, ...),
typeof(DateTime?)
)
)
You can pass Type in Expression.Constant:
ConstantExpression constant = Expression.Constant(value, member.Type);
BinaryExpression equalExpression = Expression.Equal(member, constant);
If I have a method that takes a boolean like:
public void Foo(boolean condition)
And call it like this:
Foo("MyField" == "MyValue");
Can I compose that into an expression tree in order to construct a query to some other datasource that will use MyField as one parameter and MyValue and another.
I can only seem to make that condition into a expression that evaluates to false.
UPDATE
var param = Expression.Parameter(typeof(Field), field);
var prop = Expression.PropertyOrField(param, "Name");
ConstantExpression #const = Expression.Constant(value, typeof(string));
var body = Expression.Equal(prop, #const);
var lambda = Expression.Lambda<Func<Field, bool>>(body, param);
Where Field is a class with two properties, Name and Value
Foo("MyField" == "MyValue") is, as noted at the bottom of the question, a constant false (right up at the compiler). You have a few choices here - the simplest of course is to do something like:
void Foo(Expression<Func<YourType,bool>> predicate) {...}
and call with
Foo(x => x.MyField == "MyValue");
then here, there is nothing left to do; we already have the expression. So I assume you mean "MyField" is a string only known at runtime, in which case:
void Foo<T>(string fieldName, T value) {
var param = Expression.Parameter(typeof(YourType), "x");
var body = Expression.Equal(
Expression.PropertyOrField(param, fieldName),
Expression.Constant(value, typeof(T))
);
var lambda = Expression.Lambda<Func<YourType, bool>>(body, param);
}
and call with Foo("MyField", "MyValue) (with an implicit <string> in there, courtesy of the compiler), or Foo("MyField", 123) if the prop is an int (implicit <int>),
The final scenario is where "MyValue" is also a string only known at runtime (emph: string) - in which case we'll need to parse it:
void Foo(string fieldName, string value) {
var param = Expression.Parameter(typeof(YourType), "x");
var prop = Expression.PropertyOrField(param, fieldName);
ConstantExpression #const;
if(prop.Type == typeof(string)) {
#const = Expression.Constant(value, typeof(string));
} else {
object parsed = TypeDescriptor.GetConverter(prop.Type)
.ConvertFromInvariantString(value);
#const = Expression.Constant(parsed, prop.Type);
}
var body = Expression.Equal(prop,#const);
var lambda = Expression.Lambda<Func<YourType, bool>>(body, param);
}
Here the call is always 2 strings - so Foo("MyField", "123") even when int.
In can create expression trees from delegates. For example, if you define your method so that it takes a delegate as a parameter, you can use it as follows:
public void Foo(Func<bool> fn)
{
// invoke the passed delegate
var result = fn();
}
Foo(() => "MyField" == "MyValue");
In order to create an expression tree, rather than execute the delegate, change the method as follows:
public void Foo(Expression<Func<bool>> expression)
{
// inspect your expression tree here
}
However, in your case, you will find that your expression is a boolean constant with a value of 'false', this is because the compiler has evaluated "MyField" == "MyValue" which is of course false.
If you just want name-value pairs, which not just use a Dictionary<string, string> ?
I started with the IQueryable extension methods from this example on CodePlex.
What i believe i need is an IQueryable extension method to "Where", where the method signature looks like:
public static IQueryable<T> Where<T>(this IQueryable<T> source, string columnName, string keyword)
and effectively does this (assuming T.columnName is of type string):
source.Where(p => p.ColumnName.Contains("keyword"))
using the above CodePlex example, i think i understand how he got the OrderBy method working, but my problem seems a bit more complex and I don't know how to get the Contains("keyword") part working.
Thanks in advance,
--Ed
Update: 9/13/2010 6:26pm PST
I thought the following would work, but end up getting a NotSupportedException (The LINQ expression node type 'Invoke' is not supported in LINQ to Entities.) when I execute the expression via Count(). Any ideas?
public static IQueryable<T> Where<T>(this IQueryable<T> source, string columnName, string keyword)
{
var type = typeof(T);
var property = type.GetProperty(columnName);
if (property.PropertyType == typeof(string))
{
var parameter = Expression.Parameter(type, "p");
var propertyAccess = Expression.MakeMemberAccess(parameter, property);
var sel = Expression.Lambda<Func<T, string>>(propertyAccess, parameter);
var compiledSel = sel.Compile();
return source.Where(item => compiledSel(item).Contains(keyword));
}
else
{
return source;
}
}
public static IQueryable<T> Where<T>(
this IQueryable<T> source, string columnName, string keyword)
{
var arg = Expression.Parameter(typeof(T), "p");
var body = Expression.Call(
Expression.Property(arg, columnName),
"Contains",
null,
Expression.Constant(keyword));
var predicate = Expression.Lambda<Func<T, bool>>(body, arg);
return source.Where(predicate);
}
Well a couple of years later, but if anybody still need this, here it is:
public static IQueryable<T> Has<T>(this IQueryable<T> source, string propertyName, string keyword)
{
if (source == null || propertyName.IsNull() || keyword.IsNull())
{
return source;
}
keyword = keyword.ToLower();
var parameter = Expression.Parameter(source.ElementType, String.Empty);
var property = Expression.Property(parameter, propertyName);
var CONTAINS_METHOD = typeof(string).GetMethod("Contains", new[] { typeof(string) });
var TO_LOWER_METHOD = typeof(string).GetMethod("ToLower", new Type[] { });
var toLowerExpression = Expression.Call(property, TO_LOWER_METHOD);
var termConstant = Expression.Constant(keyword, typeof(string));
var containsExpression = Expression.Call(toLowerExpression, CONTAINS_METHOD, termConstant);
var predicate = Expression.Lambda<Func<T, bool>>(containsExpression, parameter);
var methodCallExpression = Expression.Call(typeof(Queryable), "Where",
new Type[] { source.ElementType },
source.Expression, Expression.Quote(predicate));
return source.Provider.CreateQuery<T>(methodCallExpression);
}
I called my method "Has" just to keep it short, sample usage:
filtered = filtered.AsQueryable().Has("City", strCity)
And you can concatenate to make it even more expressive:
filtered = filtered.AsQueryable().Has("Name", strName).Has("City", strCity).Has("State", strState);
By the way, the "IsNull()" attached to the strings is just another simple extension method:
public static Boolean IsNull(this string str)
{
return string.IsNullOrEmpty(str);
}
The .Contains("keyword") part is exactly right in your example.
It's the p.ColumnName part that's going to cause trouble.
Now, there are a number of ways of doing this, generally involving either reflection or Expression<>, neither of which is particularly efficent.
The problem here is by passing the column name as a string, you are doing to undo that exact thing LINQ was invented to allow.
However, there are probably better ways of accomplishing your overall task besides that way.
So, let's look at alternate ways:
You want to be able to say :
var selector = new Selector("Column1", "keyword");
mylist.Where(item => selector(item));
and have it was the equivalent of
mylist.Where(item=> item.Column1.Contains("keyword"));
How 'bout we go with:
Func<MyClass, string> selector = i => i.Column1;
mylist.Where(item => selector(item).Contains("keyword"));
or
Func<MyClass, bool> selector = i => i.Column1.Contains("keyword");
mylist.Where(item => selector(item));
These are easily expanded for alternatives:
Func<MyClass, string> selector;
if (option == 1)
selector = i => i.Column1;
else
selector = i => i.Column2;
mylist.Where(item => selector(item).Contains("keyword"));
See, in generics type of the object works dynamically. So when p.ColumnName is taken as string, the Contains of string is been executed.
Generally for any lambda expression you specify, it parses the thing into an Expression and ultimately produces the output at runtime.
If you see my post :
http://www.abhisheksur.com/2010/09/use-of-expression-trees-in-lamda-c.html
you will understand how it works actually.