Develop Reference

How to build dynamic query with Where and OR using Expression

I hope somebody can guide and help me with this. We have an inherited project that uses ExpressionHelper class. Basically, this Expression Helper will return an IQueryable that build a dynamic query base on the search term that the user provided.
For example, I have the below code where I pass 2 search terms.
IQueryable<UserEntity> modifiedQuery = _uow.UserRepository.GetAll();;
var searchTerms = new List<SearchTerm>
{
new SearchTerm { Name = "FirstName", Operator = "eq", Value = "Bob" },
new SearchTerm { Name = "FirstName", Operator = "eq", Value = "John" }
};
foreach (var searchTerm in searchTerms)
{
var propertyInfo = ExpressionHelper
.GetPropertyInfo<TEntity>(searchTerm.EntityName ?? searchTerm.Name);
var obj = ExpressionHelper.Parameter<TEntity>();
var left = ExpressionHelper.GetPropertyExpression(obj, propertyInfo);
var right = searchTerm.ExpressionProvider.GetValue(searchTerm.Value);
var comparisonExpression = searchTerm.ExpressionProvider
.GetComparison(left, searchTerm.Operator, right);
// x => x.Property == "Value"
var lambdaExpression = ExpressionHelper
.GetLambda<TEntity, bool>(obj, comparisonExpression);
// query = query.Where...
modifiedQuery = ExpressionHelper.CallWhere(modifiedQuery, lambdaExpression);
}
With the code above and using the below ExpressionHelper class, this generate the below SQL query when I check using SQLProfiler. Please notice the AND in the query. What I actually what is OR.
Constructed QUERY in SQL Profiler
SELECT
[Extent1].[FirstName] AS [FirstName],
FROM [dbo].[tblUser] AS [Extent1]
WHERE ([Extent1].[Conatact1] = N'Bob') AND ([Extent1].[Contact2] = N'John')
ExpressionHelper.cs
public static class ExpressionHelper
{
private static readonly MethodInfo LambdaMethod = typeof(Expression)
.GetMethods()
.First(x => x.Name == "Lambda" && x.ContainsGenericParameters && x.GetParameters().Length == 2);
private static MethodInfo[] QueryableMethods = typeof(Queryable)
.GetMethods()
.ToArray();
private static MethodInfo GetLambdaFuncBuilder(Type source, Type dest)
{
var predicateType = typeof(Func<,>).MakeGenericType(source, dest);
return LambdaMethod.MakeGenericMethod(predicateType);
}
public static PropertyInfo GetPropertyInfo<T>(string name)
=> typeof(T).GetProperties()
.Single(p => p.Name == name);
public static ParameterExpression Parameter<T>()
=> Expression.Parameter(typeof(T));
public static MemberExpression GetPropertyExpression(ParameterExpression obj, PropertyInfo property)
=> Expression.Property(obj, property);
public static LambdaExpression GetLambda<TSource, TDest>(ParameterExpression obj, Expression arg)
=> GetLambda(typeof(TSource), typeof(TDest), obj, arg);
public static LambdaExpression GetLambda(Type source, Type dest, ParameterExpression obj, Expression arg)
{
var lambdaBuilder = GetLambdaFuncBuilder(source, dest);
return (LambdaExpression)lambdaBuilder.Invoke(null, new object[] { arg, new[] { obj } });
}
public static IQueryable<T> CallWhere<T>(IQueryable<T> query, LambdaExpression predicate)
{
var whereMethodBuilder = QueryableMethods
.First(x => x.Name == "Where" && x.GetParameters().Length == 2)
.MakeGenericMethod(new[] { typeof(T) });
return (IQueryable<T>)whereMethodBuilder
.Invoke(null, new object[] { query, predicate });
}
public static IQueryable<TEntity> CallOrderByOrThenBy<TEntity>(
IQueryable<TEntity> modifiedQuery,
bool useThenBy,
bool descending,
Type propertyType,
LambdaExpression keySelector)
{
var methodName = "OrderBy";
if (useThenBy) methodName = "ThenBy";
if (descending) methodName += "Descending";
var method = QueryableMethods
.First(x => x.Name == methodName && x.GetParameters().Length == 2)
.MakeGenericMethod(new[] { typeof(TEntity), propertyType });
return (IQueryable<TEntity>)method.Invoke(null, new object[] { modifiedQuery, keySelector });
}
}
I have hard time understanding on how the query was created and how do I change it to become OR in the created query.
Hope someone can guide me and point to the right direction. Thank you!

Add to SearchTerm a new property (C# 6.0 syntax here):
// This is quite wrong. We should have an enum here, but Operator is
// done as a string, so I'm maintaining the "style"
// Supported LogicalConnector: and, or
public string LogicalConnector { get; set; } = "and";
}
Then:
private static IQueryable<TEntity> BuildQuery<TEntity>(IQueryable<TEntity> modifiedQuery, List<SearchTerm> searchTerms)
{
Expression comparisonExpressions = null;
var obj = ExpressionHelper.Parameter<TEntity>();
foreach (var searchTerm in searchTerms)
{
var propertyInfo = ExpressionHelper
.GetPropertyInfo<TEntity>(searchTerm.EntityName ?? searchTerm.Name);
var left = ExpressionHelper.GetPropertyExpression(obj, propertyInfo);
var right = searchTerm.ExpressionProvider.GetValue(searchTerm.Value);
var comparisonExpression = searchTerm.ExpressionProvider.GetComparison(left, searchTerm.Operator, right);
if (comparisonExpressions == null)
{
comparisonExpressions = comparisonExpression;
}
else if (searchTerm.LogicalConnector == "and")
{
comparisonExpressions = Expression.AndAlso(comparisonExpressions, comparisonExpression);
}
else if (searchTerm.LogicalConnector == "or")
{
comparisonExpressions = Expression.OrElse(comparisonExpressions, comparisonExpression);
}
else
{
throw new NotSupportedException(searchTerm.LogicalConnector);
}
}
if (comparisonExpressions != null)
{
// x => x.Property == "Value"
var lambdaExpression = ExpressionHelper.GetLambda<TEntity, bool>(obj, comparisonExpressions);
// query = query.Where...
modifiedQuery = ExpressionHelper.CallWhere(modifiedQuery, lambdaExpression);
}
return modifiedQuery;
}
Use it like:
var searchTerms = new List<SearchTerm>
{
new SearchTerm { Name = "PrimaryContact", Operator = "eq", Value = "Bob" },
new SearchTerm { Name = "SecondaryContact", Operator = "eq", Value = "Bob" },
new SearchTerm { Name = "PrimaryContact", Operator = "eq", Value = "John", LogicalConnector = "or", }
};
IQueryable<UserEntity> query = BuildQuery<UserEntity>(modifiedQuery, searchTerms);
Note that there is no way in this code to explicitly set brackets, that will be implicitly set as:
(((A opB b) opC C) opD D)
Where A, B, C, D are the SearchTerm[0], SearchTerm[1], SearchTerm[2], SearchTerm[3] and opB, opC, opD are the operators defined in SearchTerm[1].LogicalConnector, SearchTerm[2].LogicalConnector, SearchTerm[3].LogicalConnector.
While putting brackets is easy, choosing how to "describe" them is complex, unless you change significantly your SearchTerm collection (it couldn't be a "linear" array but it would need to be a tree).
P.S. I was wrong, you don't need an ExpressionVisitor. You need an ExpressionVisitor when you are trying to "merge" multiple LambdaExpressions that have distinct ParameterExpression. In this code we are able to have a single var obj = ExpressionHelper.Parameter<TEntity>() for all the query, so no problems merging the conditions. To make it clear: if you want to "merge" x1 => x1.Foo == "Foo1" with x2 => x2.Foo == "Foo2" then you need an ExpressionVisitor that replaces x2 with x1, otherwise you would get a wrong query like x1 => x1.Foo == "Foo1" || x2.Foo == "Foo2". In the code given we have only x1 (that is var obj = ExpressionHelper.Parameter<TEntity>()), so no problem.

Create Expression for List.Any clause

Consider this hierarchy of classes.
class Event
{
public Attendees[] AttendeesList;
}
class Attendees
{
public ComplexProperty Property;
public object Value;
}
class ComplexProperty
{
}
class Program
{
static void Main(string[] args)
{
// There are constants.
ComplexProperty constproperty = new ComplexProperty();
object constValue = 5;
// consider this linq query:
Event evnt = new Event();
var result = evnt.AttendeesList.Any((attnds) => attnds.Property == constproperty && attnds.Value == constValue);
// I want to create an Expression tree for above linq query. I need something like this:
ParameterExpression parameter = Expression.Parameter(typeof(Attendees));
Expression left = Expression.Property(parameter, typeof(ComplexProperty).GetProperty("Property"));
// complete this piece.......
}
}
I want to create a Linq.Expressions.Expression for evnt.AttendeesList.Any((attnds) => attnds.Property == constproperty && attnds.Value == constValue);
this linq query. How to query collection with Linq.Expressions looks similar, but I have Any in my linq expression.
Please help.

This will give you a start:
ParameterExpression parameter = Expression.Parameter(typeof(Attendees));
Expression left = Expression.Equal(Expression.Property(parameter, "Property"), Expression.Constant(constproperty));
var objType = constValue == null ? typeof(object) : constValue.GetType();
var convertLeft = Expression.Convert(Expression.Property(parameter, "Value"), objType);
var convertRight = Expression.Convert(Expression.Constant(constValue), objType);
Expression right = Expression.Equal(convertLeft, convertRight);
Expression joined = Expression.And(left, right);
var anyMethod = typeof(Queryable).GetMethods().Where(m => m.Name=="Any" && m.GetParameters().Count() == 2).First().MakeGenericMethod(typeof(Attendees));
var call = Expression.Call(anyMethod, Expression.Constant(evnt.AttendeesList, typeof(IQueryable<Attendees>)), Expression.Lambda(joined, parameter));
I changed Value to type int for ease of use. If you want to keep it as an object, you will need to add a Expression.Convert() call in the right Expression.

How to call a method with instance using Expression

For example I have some class with some property:
public class SomeClass
{
public Version Version { get; set; }
}
And I have a list of this type with sample data:
var list = new List<SomeClass>();
for (var i = 0; i < 1000; i++)
{
list.Add(new SomeClass
{
Version = new Version(i, i / 2, i / 3, i / 4),
});
}
I want to write method that filters by version using Version.Equals method:
var filterValue = new Version(12, 6, 4, 3);
var modelType = typeof(SomeClass);
var propertyType = typeof(Version);
var arg = Expression.Parameter(modelType, "x");
var property = Expression.Property(arg, "Version");
var value = Expression.Convert(Expression.Constant(filterValue), propertyType);
var versionEqualsMethod = typeof(Version).GetMethod("Equals", new[] { typeof(Version) });
/////////
Expression inst = null; // <-- ???
/////////
var expr = Expression.Call(inst, versionEqualsMethod, property, value);
var delegateType = typeof(Func<,>).MakeGenericType(modelType, typeof(bool));
var delegateValue = Expression.Lambda(delegateType, expr, arg).Compile();
var genericMethod =
typeof(Enumerable).GetMethods()
.First(
method =>
method.Name == "Where" && method.IsGenericMethodDefinition
&& method.GetGenericArguments().Length == 1 && method.GetParameters().Length == 2)
.MakeGenericMethod(modelType);
var result = genericMethod.Invoke(null, new object[] { list, delegateValue });
What do I use as instance in Expression.Call?
UPDATE
Solution is:
var expr = Expression.Call(property, versionEqualsMethod, value);

You normally would do:
var filterValue = new Version(12, 6, 4, 3);
var modelType = typeof(SomeClass);
var propertyType = typeof(Version);
var arg = Expression.Parameter(modelType, "x");
var property = Expression.Property(arg, "Version");
// Changes from here onward
var value = Expression.Constant(filterValue);
var versionEqualsMethod = typeof(Version).GetMethod("Equals", new[] { typeof(Version) });
var expr = Expression.Call(property, versionEqualsMethod, value);
Because the Equals would be used like:
model.Version.Equals(filterValue);
I'm not handling the model.Version == null case!
Note that you don't need the Expression.Convert.
And what you are doing is ok if the "containing method" (the method where you'll put this code) is non-generic, but normally it would be a generic method, that has as the generic parameter the modelType, so the last part of the code would be different (starting from var delegateType =), because you could use the TModelType generic type directly.

Maybe I'm missing out on something, but wouldn't this work:
var results = list.Where(sc => sc.Version == filterVersion);

What you are trying to accomplish is much easier to do with reflection. Check this running online example. (If I understand you correctly that is... It would be helpful, if you could provide the signature of the function you are trying to write.)
Implementation
public static class Extensions
{
public static IEnumerable<T> Filter<T>(
this IEnumerable<T> enumerable, string propertyName, object filterValue)
{
var elementType = typeof (T);
var property = elementType.GetProperty(propertyName);
return enumerable.Where(element =>
{
var propertyValue = property.GetMethod.Invoke(element, new object[] {});
return propertyValue.Equals(filterValue);
});
}
}
Usage
var list = new List<SomeClass>();
for (var i = 0; i < 1000; i++)
{
list.Add(new SomeClass {Version = new Version(i, i/2, i/3, i/4)});
}
var filteredList = list.Filter("Version", new Version(12, 6, 4, 3));
Console.WriteLine(filteredList.Single().Version);

Dealing with nulls indynamically created linq expression

I am dynamically creating a Linq expression based on string arrays and I'm running into a problem. The way that the expression is created and parenthesized it is causing this to throw an object null reference on Id 3. It creates this expression and If it were parenthesized correctly it wouldn't evaluate the second half of the expression and wouldn't throw an error I assume. Anyone have a way of creating the expression so it doesn't end up parenthesized like this?
{x => ((True And x.Id.ToString().ToLower().Contains("John")) Or ((x.Name != null) And x.Name.ToString().ToLower().Contains("John")))}
class Person
{
public int Id { get; set; }
public string Name { get; set; }
}
public class Test
{
public void someMethod()
{
var x = new List<Person>(new Person[] {
new Person { Id = 1, Name = "Jerry" },
new Person { Id = 2, Name = "Mary" },
new Person { Id = 3, Name = null },
new Person { Id = 4, Name = "John" },
new Person { Id = 5, Name = "Amy" }
});
var columns = new List<string>(new string[] {
"Name",
"Id"
});
var searchTerm = "John";
var searchColumns = columns.Select(a => new { ColName = a });
var type = typeof(Person);
ParameterExpression paramExpr = Expression.Parameter(type, "x");
Expression body = null;
var piList = new List<PropertyInfo>();
foreach (var s in searchColumns)
piList.Add(type.GetProperty(s.ColName));
if (piList[0].PropertyType.IsPrimitive || piList[0].PropertyType.Equals(typeof(DateTime)))
body = Expression.Constant(true);
else
body = Expression.NotEqual(Expression.Property(paramExpr, piList[0]), Expression.Constant(null, piList[0].PropertyType));
body = Expression.And(body,
Expression.Call(
Expression.Call(
Expression.Call(
Expression.Property(paramExpr, piList[0]),
typeof(Convert).GetMethod("ToString", Type.EmptyTypes)
),
typeof(string).GetMethod("ToLower", new Type[0])
),
typeof(string).GetMethod("Contains"),
Expression.Constant(searchTerm.ToLower())
));
for (int i = 1; i < piList.Count; i++)
{
Expression body1 = null;
if (piList[i].PropertyType.IsPrimitive || piList[i].PropertyType.Equals(typeof(DateTime)))
body1 = Expression.Constant(true);
else
body1 = Expression.NotEqual(Expression.Property(paramExpr, piList[i]), Expression.Constant(null, piList[i].PropertyType));
body =
Expression.Or(body,
Expression.And(body1,
Expression.Call(
Expression.Call(
Expression.Call(
Expression.Property(paramExpr, piList[i]),
typeof(Convert).GetMethod("ToString", Type.EmptyTypes)
),
typeof(string).GetMethod("ToLower", new Type[0])
),
typeof(string).GetMethod("Contains"),
Expression.Constant(searchTerm.ToLower())
)
));
}
var lambda = Expression.Lambda<Func<Person, bool>>(body, paramExpr);
}
}

Why not just build one of these. They both avoid the issue with null values for Name.
(x.Name ?? "").IndexOf("John", StringComparison.CurrentCultureIgnoreCase) >= 0
or, if you really want equality, not contains it as a substring
string.Equals(x.Name, "John", StringComparison.CurrentCultureIgnoreCase)
BTW - x.Id will never contain "John" and neither will lowercase strings.
Also, you might want to consider using a PredicateBuilder instead of building the expression directly.

Joining multiple where clauses in LINQ as OR instead of AND

Is there anyway to join LINQ where clauses as OR ?
var ints = new [] { 1, 3, 5, 7 };
var query = from i in ints select i;
query = query.Where (q => q == 3);
query = query..Where (q => q == 7);
What I want is the ability to dynamically add where clauses but make them use OR instead of AND

If you want to stay with your strong-typing Linq queries you should look into LinqKit and predicate building. I have used this for something similar and found it to work well with And / Or stacking of filters.
Check out the C#4.0/3.0 in a Nutshell excerpt for more in depth info. Here is a snip from my code:
//Setup the initial predicate obj then stack on others:
basePredicate = basePredicate.And(p => false);
var predicate1 = PredicateBuilder.True<Person>();
foreach (SearchParms parm in parms)
{
switch (parm.field)
{
case "firstname":
predicate1 = predicate1.And(p => p.FirstName.Trim().ToLower().Contains(sValue));
break;
//etc...
}
}
//Run a switch based on your and/or parm value to determine stacking:
if (Parm.isAnd) {
basePredicate = basePredicate.And(predicate1);
} else {
basePredicate = basePredicate.Or(predicate1);
}

How about something like this?
var query = from i in ints where CheckConditions(i) select i;
public bool CheckConditions(int i)
{
var conditions = WhereConditions; //an IEnumerable<Func<int, bool>> of dynamically added conditions
foreach (var condition in conditions)
{
if (condition(i)) return true;
}
return false;
}
You can probably expand this to be a bit cleverer but that's sort of how I'd do it.
EDIT: Sorry the first example was an AND, have changed it now to be an OR. So the first time it encounters a passing condition it returns true.

Using ExpressionVisitor to help to build the expression base on two expressions with OR/AND relationship. This answer is from Jeffery Zhao's blog.
internal class ParameterReplacer : ExpressionVisitor
{
public ParameterReplacer(ParameterExpression paramExpr)
{
this.ParameterExpression = paramExpr;
}
public ParameterExpression ParameterExpression { get; private set; }
public Expression Replace(Expression expr)
{
return this.Visit(expr);
}
protected override Expression VisitParameter(ParameterExpression p)
{
return this.ParameterExpression;
}
}
public static Expression<Func<T, bool>> And<T>(this Expression<Func<T, bool>> one, Expression<Func<T, bool>> another)
{
var candidateExpr = Expression.Parameter(typeof(T), "candidate");
var parameterReplacer = new ParameterReplacer(candidateExpr);
var left = parameterReplacer.Replace(one.Body);
var right = parameterReplacer.Replace(another.Body);
var body = Expression.And(left, right);
return Expression.Lambda<Func<T, bool>>(body, candidateExpr);
}
public static Expression<Func<T, bool>> Or<T>(this Expression<Func<T, bool>> one, Expression<Func<T, bool>> another)
{
var candidateExpr = Expression.Parameter(typeof(T), "candidate");
var parameterReplacer = new ParameterReplacer(candidateExpr);
var left = parameterReplacer.Replace(one.Body);
var right = parameterReplacer.Replace(another.Body);
var body = Expression.Or(left, right);
return Expression.Lambda<Func<T, bool>>(body, candidateExpr);
}

You can using Union method:
var ints = new [] { 1, 3, 5, 7 };
var query = ints.Where(q => q == 3);
query = query.Union(ints.Where(q => q == 7));

Are you talking about specifying more than one condition in the lambda?
query = query.Where(q => q == 3 ||
q == 7);

try this
var ints = new [] { 1, 3, 5, 7 };
var query = ints.select(X=>X).where(X=>X==3||X==7);

I am trying to do something similar. Here's what I came up with:
//various test cases
bool useTestCase1 = true;
bool useTestCase2 = true;
bool useTestCase3 = false;
query = query.Where(q =>
(q == 3 && useTestCase1 ) ||
(q == 7 && useTestCase2 ) ||
(q == 10 && useTestCase3 )
);