Develop Reference

Nullable comparisons in expression trees

I'm trying to build a dynamic LINQ query, in Entity Framework, from a user-provided set of collection criteria. Eventually, this will include more complex behaviors, but currently I just have a list of field names and values, and I want to return all the records in which the field names have those values.
My basic structure is this:
public IEnumerable<ThingViewModel> getMythings(SelectionCriteria selectionCriteria)
{
var predicate = constructPredicate<Thing>(selectionCriteria);
var things = this.dbContext.Things.Where(predicate).ToList();
return Mapper.Map<List<Thing>, List<ThingViewModel>>(things);
}
Where all the interesting work is in constructPredicate():
private static Expression<Func<T, bool>> constructPredicate<T>(SelectionCriteria selectionCriteria)
{
// using Pete Montgomery's PredicateBuilder:
// http://petemontgomery.wordpress.com/2011/02/10/a-universal-predicatebuilder/
var predicate = PredicateBuilder.True<T>();
foreach (var item in selectionCriteria.andList)
{
// Accessing foreach values in closures can result in unexpected results.
// http://stackoverflow.com/questions/14907987/access-to-foreach-variable-in-closure
var fieldName = item.fieldName;
var fieldValue = item.fieldValue;
var parameter = Expression.Parameter(typeof (T), "t");
var property = Expression.Property(parameter, fieldName);
var value = Expression.Constant(fieldValue);
var lambda = buildCompareLambda<T>(property, value, parameter);
predicate = predicate.And(lambda);
}
return predicate;
}
All of that seems like a perfectly reasonable structure, but it's in the buildCompareLambda() that I'm having difficulties. I'm not seeing a way to do this in a generic way, I'm having to create different methods for different types. I'd started by handling strings, and that was simple enough. I next tried to handle integers, but it turns out that the integer fields in the database are nullable, which introduced a whole new class of complexity.
My buildCompareLambda(), so far:
private static Expression<Func<T, bool>> buildCompareLambda<T>(
MemberExpression property,
ConstantExpression value,
ParameterExpression parameter)
{
Expression<Func<T, bool>> lambda = null;
if (property.Type == typeof (string))
lambda = buildStringCompareLambda<T>(property, value, parameter);
else if (property.Type.IsGenericType && Nullable.GetUnderlyingType(property.Type) != null)
lambda = buildNullableCompareLambda<T>(property, value, parameter);
if (lambda == null)
throw new Exception(String.Format("SelectrionCriteria cannot handle property type '{0}'", property.Type.Name));
return lambda;
}
As I said, buildStringCompareLambda is simple enough:
private static Expression<Func<T, bool>> buildStringCompareLambda<T>(
MemberExpression property,
ConstantExpression value,
ParameterExpression parameter)
{
var equalsMethod = typeof (string).GetMethod("Equals",
new[] {typeof (string), typeof (string)});
var comparison = Expression.Call(equalsMethod, property, value);
return Expression.Lambda<Func<T, bool>>(comparison, parameter);
}
But buildNullableCompareLambda() is getting ugly:
private static Expression<Func<T, bool>> buildNullableCompareLambda<T>(MemberExpression property,
ConstantExpression value,
ParameterExpression parameter)
{
var underlyingType = Nullable.GetUnderlyingType(property.Type);
if (underlyingType == typeof (int) || underlyingType == typeof (Int16) || underlyingType == typeof (Int32) ||
underlyingType == typeof (Int64) || underlyingType == typeof (UInt16) || underlyingType == typeof (UInt32) ||
underlyingType == typeof (UInt64))
{
var equalsMethod = underlyingType.GetMethod("Equals", new[] {underlyingType});
var left = Expression.Convert(property, underlyingType);
var right = Expression.Convert(value, underlyingType);
var comparison = Expression.Call(left, equalsMethod, new Expression[] {right});
return Expression.Lambda<Func<T, bool>>(comparison, parameter);
}
return null;
}
It's my intent to add support for more types, in buildNullableCompareLambda(), and to move the handling of each type into a function, so that the same code can be called from buildCompareLambda() and from buildNullableCompareLambda(). But that's for the future - currently I'm stuck on comparing ints. Currently, I'm converting both the property and the value to the underlying type, since I don't want to have separate functions for each integer type, and I don't want the user to have to care whether the EF models a field into an Int16 or an Int32. And that's working, for non-null fields.
I've been browsing around SO, and finding some answers, which is how I've gotten as far as I have, but none of the answers I've seen on handling nullable types really work for me, because they don't really handle nulls.
In my case, if the user passes me a selection criteria with an item that is supposed to equal null, I would want to return the records for which that field are null, and this bit about converting both sides to a base type doesn't seem to work. I'm getting an "Object reference not set to an instance of an object" exception.
In SQL, what I'd want is a "WHERE field IS NULL", if the value is null, or "WHERE field = 'value'", if it is not. And I'm not seeing how to build that kind of alternative into an expression tree.
Any ideas?
Added: It has been suggested that I use Expression.Equal().
With that, my loop becomes:
private static Expression<Func<T, bool>> constructPredicate<T>(SelectionCriteria selectionCriteria)
{
var predicate = PredicateBuilderEx.True<T>();
var foo = PredicateBuilder.True<T>();
foreach (var item in selectionCriteria.andList)
{
var fieldName = item.fieldName;
var fieldValue = item.fieldValue;
var parameter = Expression.Parameter(typeof (T), "t");
var property = Expression.Property(parameter, fieldName);
var value = Expression.Constant(fieldValue);
var comparison = Expression.Equal(property, value);
var lambda = Expression.Lambda<Func<T, bool>>(comparison, parameter);
predicate = PredicateBuilderEx.And(predicate, lambda);
}
return predicate;
}
And that doesn't work. I get an exception:
The binary operator Equal is not defined for the types
'System.Nullable`1[System.Int16]' and 'System.Int16'.

As is quite often the case, folks here may not quite come up with the answer, but they get most of the way, and close enough that I can work out the rest.
Expression.Equal requires both parameters to be of the same type. If one is nullable, they both need to be nullable. But that's not that hard to deal with:
private static Expression<Func<T, bool>> constructPredicate<T>(SelectionCriteria selectionCriteria)
{
var predicate = PredicateBuilderEx.True<T>();
var foo = PredicateBuilder.True<T>();
foreach (var item in selectionCriteria.andList)
{
var fieldName = item.fieldName;
var fieldValue = item.fieldValue;
var parameter = Expression.Parameter(typeof (T), "t");
var property = Expression.Property(parameter, fieldName);
var value = Expression.Constant(fieldValue);
var converted = Expression.Convert(value, property.Type);
var comparison = Expression.Equal(property, converted);
var lambda = Expression.Lambda<Func<T, bool>>(comparison, parameter);
predicate = PredicateBuilderEx.And(predicate, lambda);
}
return predicate;
}
Thanks, all.

As Lee states in his comment, you don't need to implement buildNullableCompareLambda<T> for each type. There is already a method that checks the types of the left and right expressions and calls the Equals method on them if they are a user-defined type, and does the lifting and proper comparison if they are nullable types. See here.
Your lambda is basically:
var property = Expression.Property(parameter, fieldName);
var value = Expression.Constant(fieldValue);
var lambda = Expression.Equal(property, value);
Edit:
It seems to me that this is a bug. Eric Lippert thinks so(link). The documentation states the scenario where they are both of the same type and what to do then:
If left.Type and right.Type are both non-nullable, the node is not
lifted. The type of the node is Boolean. If left.Type and right.Type
are both nullable, the node is lifted. The type of the node is
Boolean.
It doesn't exactly what would happen if one is nullable and the other isn't. In the same link referenced, Eric gives a workaround.

Is there a work arround on this code that generates a NotSupportedException from NHibernate?

The following is the code:
list = query.Where(x => ((string)x.GetType()
.GetProperty(propertyName).GetValue(this, null))
.EndsWith(filterValue)).ToList();
This code is a refactored code that if I break it to its object and property level it would take me repeating it hundred times all around my code project. According to my research in this site it has something to do with the SQL translation. But is there any work around where this code or its variant will work fine?

The solution is to create an Expression that returns the specified property and pass that expression to Where:
var query = session.Query<YourType>();
list = query.Where(GetExpression<YourType>(propertyName, filterValue)).ToList();
GetExpression looks something like this:
public static Expression<Func<T, bool>> GetExpression<T>(string propertyName,
string filterValue)
{
var parameter = Expression.Parameter(typeof(T));
var property = Expression.Property(parameter, propertyName);
var method = typeof(string).GetMethod("EndsWith", new [] { typeof(string) });
var body = Expression.Call(property, method,
Expression.Constant(filterValue));
return (Expression<Func<T, bool>>)Expression.Lambda(body, parameter);
}

Combined Expressions

I am new to using expressions and I am having some problems in an example I am working through.
What I am trying to achieve is to create an Expression which has 2 (or many) Expressions within.
For example:
public static Expression<Func<Occurrence, bool>> ReporterStartsWithAndClosed()
{
ParameterExpression occPar = Expression.Parameter(typeof(Occurrence));
MemberExpression recorderProp = Expression.Property(occPar, "Reporter");
MemberExpression fullnameProp = Expression.Property(recorderProp, "FullName");
ConstantExpression letter = Expression.Constant("A", typeof(string));
MethodInfo miStartsWith = typeof(string).GetMethod("StartsWith", new[] { typeof(string) });
MethodCallExpression mCall = Expression.Call(fullnameProp, miStartsWith, letter);
MemberExpression oiProp = Expression.Property(occPar, "OccurrenceIncident");
MemberExpression statusProp = Expression.Property(oiProp, "OccurreceIncidentStatus");
MemberExpression nameProp = Expression.Property(statusProp, "Name");
ConstantExpression name = Expression.Constant("Closed", typeof(string));
BinaryExpression equalTo = Expression.Equal(name, nameProp);
return ...?
}
The question I have, is how to I combine these expressions to return the correct type for this method. I.e. what is the syntax for combining the logic for mCall and equalTo Expressions.
My initial thought were that I should be using BlockExpressions but I couldn't get this to work.
Any help would be greatly appreciated.
Thanks
David

So to do a logical AND, use the AndAlso() method to generate the expression. Then to finish off your method, you'll want to put this combined method into an lambda expression.
Just some hints, I'd avoid writing out the types in your declarations, it makes everything harder to read. Also, you could call a method by name by using this overload of Call() so no need to get the MethodInfo object for the method.
I'd put it together like this (untested):
public static Expression<Func<Occurrence, bool>> ReporterStartsWithAndClosed(
string letter = "A")
{
// occurrence =>
// occurrence.Reporter.FullName.StartsWith("A")
//
// occurrence.OccurrenceIncident.OccurrenceIncidentStatus.Name == "Closed"
var occurrence = Expression.Parameter(typeof(Occurrence), "occurrence");
var reporter = Expression.Property(occurrence, "Reporter");
var fullName = Expression.Property(reporter, "FullName");
var startsWithLetter = Expression.Call(
fullName,
"StartsWith",
null,
Expression.Constant(letter, typeof(string))
);
var incident = Expression.Property(occurrence, "OccurrenceIncident");
var status = Expression.Property(incident, "OccurrenceIncidentStatus");
var name = Expression.Property(status, "Name");
var equalsClosed = Expression.Equal(
name,
Expression.Constant("Closed", typeof(string))
);
var body = Expression.AndAlso(startsWithLetter, equalsClosed);
return Expression.Lambda<Func<Occurrence, bool>>(body, occurrence);
}

Parse boolean condition into expression tree

If I have a method that takes a boolean like:
public void Foo(boolean condition)
And call it like this:
Foo("MyField" == "MyValue");
Can I compose that into an expression tree in order to construct a query to some other datasource that will use MyField as one parameter and MyValue and another.
I can only seem to make that condition into a expression that evaluates to false.
UPDATE
var param = Expression.Parameter(typeof(Field), field);
var prop = Expression.PropertyOrField(param, "Name");
ConstantExpression #const = Expression.Constant(value, typeof(string));
var body = Expression.Equal(prop, #const);
var lambda = Expression.Lambda<Func<Field, bool>>(body, param);
Where Field is a class with two properties, Name and Value

Foo("MyField" == "MyValue") is, as noted at the bottom of the question, a constant false (right up at the compiler). You have a few choices here - the simplest of course is to do something like:
void Foo(Expression<Func<YourType,bool>> predicate) {...}
and call with
Foo(x => x.MyField == "MyValue");
then here, there is nothing left to do; we already have the expression. So I assume you mean "MyField" is a string only known at runtime, in which case:
void Foo<T>(string fieldName, T value) {
var param = Expression.Parameter(typeof(YourType), "x");
var body = Expression.Equal(
Expression.PropertyOrField(param, fieldName),
Expression.Constant(value, typeof(T))
);
var lambda = Expression.Lambda<Func<YourType, bool>>(body, param);
}
and call with Foo("MyField", "MyValue) (with an implicit <string> in there, courtesy of the compiler), or Foo("MyField", 123) if the prop is an int (implicit <int>),
The final scenario is where "MyValue" is also a string only known at runtime (emph: string) - in which case we'll need to parse it:
void Foo(string fieldName, string value) {
var param = Expression.Parameter(typeof(YourType), "x");
var prop = Expression.PropertyOrField(param, fieldName);
ConstantExpression #const;
if(prop.Type == typeof(string)) {
#const = Expression.Constant(value, typeof(string));
} else {
object parsed = TypeDescriptor.GetConverter(prop.Type)
.ConvertFromInvariantString(value);
#const = Expression.Constant(parsed, prop.Type);
}
var body = Expression.Equal(prop,#const);
var lambda = Expression.Lambda<Func<YourType, bool>>(body, param);
}
Here the call is always 2 strings - so Foo("MyField", "123") even when int.

In can create expression trees from delegates. For example, if you define your method so that it takes a delegate as a parameter, you can use it as follows:
public void Foo(Func<bool> fn)
{
// invoke the passed delegate
var result = fn();
}
Foo(() => "MyField" == "MyValue");
In order to create an expression tree, rather than execute the delegate, change the method as follows:
public void Foo(Expression<Func<bool>> expression)
{
// inspect your expression tree here
}
However, in your case, you will find that your expression is a boolean constant with a value of 'false', this is because the compiler has evaluated "MyField" == "MyValue" which is of course false.
If you just want name-value pairs, which not just use a Dictionary<string, string> ?

IQueryable Extension: create lambda expression for querying a column for a keyword

I started with the IQueryable extension methods from this example on CodePlex.
What i believe i need is an IQueryable extension method to "Where", where the method signature looks like:
public static IQueryable<T> Where<T>(this IQueryable<T> source, string columnName, string keyword)
and effectively does this (assuming T.columnName is of type string):
source.Where(p => p.ColumnName.Contains("keyword"))
using the above CodePlex example, i think i understand how he got the OrderBy method working, but my problem seems a bit more complex and I don't know how to get the Contains("keyword") part working.
Thanks in advance,
--Ed
Update: 9/13/2010 6:26pm PST
I thought the following would work, but end up getting a NotSupportedException (The LINQ expression node type 'Invoke' is not supported in LINQ to Entities.) when I execute the expression via Count(). Any ideas?
public static IQueryable<T> Where<T>(this IQueryable<T> source, string columnName, string keyword)
{
var type = typeof(T);
var property = type.GetProperty(columnName);
if (property.PropertyType == typeof(string))
{
var parameter = Expression.Parameter(type, "p");
var propertyAccess = Expression.MakeMemberAccess(parameter, property);
var sel = Expression.Lambda<Func<T, string>>(propertyAccess, parameter);
var compiledSel = sel.Compile();
return source.Where(item => compiledSel(item).Contains(keyword));
}
else
{
return source;
}
}

public static IQueryable<T> Where<T>(
this IQueryable<T> source, string columnName, string keyword)
{
var arg = Expression.Parameter(typeof(T), "p");
var body = Expression.Call(
Expression.Property(arg, columnName),
"Contains",
null,
Expression.Constant(keyword));
var predicate = Expression.Lambda<Func<T, bool>>(body, arg);
return source.Where(predicate);
}

Well a couple of years later, but if anybody still need this, here it is:
public static IQueryable<T> Has<T>(this IQueryable<T> source, string propertyName, string keyword)
{
if (source == null || propertyName.IsNull() || keyword.IsNull())
{
return source;
}
keyword = keyword.ToLower();
var parameter = Expression.Parameter(source.ElementType, String.Empty);
var property = Expression.Property(parameter, propertyName);
var CONTAINS_METHOD = typeof(string).GetMethod("Contains", new[] { typeof(string) });
var TO_LOWER_METHOD = typeof(string).GetMethod("ToLower", new Type[] { });
var toLowerExpression = Expression.Call(property, TO_LOWER_METHOD);
var termConstant = Expression.Constant(keyword, typeof(string));
var containsExpression = Expression.Call(toLowerExpression, CONTAINS_METHOD, termConstant);
var predicate = Expression.Lambda<Func<T, bool>>(containsExpression, parameter);
var methodCallExpression = Expression.Call(typeof(Queryable), "Where",
new Type[] { source.ElementType },
source.Expression, Expression.Quote(predicate));
return source.Provider.CreateQuery<T>(methodCallExpression);
}
I called my method "Has" just to keep it short, sample usage:
filtered = filtered.AsQueryable().Has("City", strCity)
And you can concatenate to make it even more expressive:
filtered = filtered.AsQueryable().Has("Name", strName).Has("City", strCity).Has("State", strState);
By the way, the "IsNull()" attached to the strings is just another simple extension method:
public static Boolean IsNull(this string str)
{
return string.IsNullOrEmpty(str);
}

The .Contains("keyword") part is exactly right in your example.
It's the p.ColumnName part that's going to cause trouble.
Now, there are a number of ways of doing this, generally involving either reflection or Expression<>, neither of which is particularly efficent.
The problem here is by passing the column name as a string, you are doing to undo that exact thing LINQ was invented to allow.
However, there are probably better ways of accomplishing your overall task besides that way.
So, let's look at alternate ways:
You want to be able to say :
var selector = new Selector("Column1", "keyword");
mylist.Where(item => selector(item));
and have it was the equivalent of
mylist.Where(item=> item.Column1.Contains("keyword"));
How 'bout we go with:
Func<MyClass, string> selector = i => i.Column1;
mylist.Where(item => selector(item).Contains("keyword"));
or
Func<MyClass, bool> selector = i => i.Column1.Contains("keyword");
mylist.Where(item => selector(item));
These are easily expanded for alternatives:
Func<MyClass, string> selector;
if (option == 1)
selector = i => i.Column1;
else
selector = i => i.Column2;
mylist.Where(item => selector(item).Contains("keyword"));

See, in generics type of the object works dynamically. So when p.ColumnName is taken as string, the Contains of string is been executed.
Generally for any lambda expression you specify, it parses the thing into an Expression and ultimately produces the output at runtime.
If you see my post :
http://www.abhisheksur.com/2010/09/use-of-expression-trees-in-lamda-c.html
you will understand how it works actually.