Develop Reference

Expression Tree: iterate through strings and check if they contained in another Expression

I want a function Expression> AnyColumnContains(string[] value)
that iterates through all Columns of a table and checks an array of values against the columns and returns true only if every value is contained in any column.
i already have a function that matches every column against one value but i have problems extending it to check the columns against every value
This is what i've got:
Expression<Func<T, bool>> AnyColumnContains<T>(string value){
var p = Expression.Parameter(typeof(T), "entity");
var fieldAccessors = typeof(T)
.GetProperties(BindingFlags.Instance | BindingFlags.Public)
.Where(f => f.PropertyType == typeof(string))
.Select(f => Expression.Property(p, f))
.ToArray();
var fieldArray = Expression.NewArrayInit(typeof(string), fieldAccessors);
var concatCall = Expression.Call(typeof(string).GetMethod(
"Concat", new[] { typeof(string[]) }), fieldArray);
var contains = Expression.Call(
concatCall,
typeof(string).GetMethod("Contains", new[] { typeof(string) }),
Expression.Constant(value));
return Expression.Lambda<Func<T, bool>>(contains, p);
}
I tried to use a own extension method and replaced Contains with it but the problem is that i use sqlite and the expression cannot be converted since the Provider doesn't know the methods
This is what i want:
Expression<Func<T, bool>> AnyColumnContains<T>(string[] values){
// ... //
var contains = // build Expression Tree that matches all values against concatCall and only returns true if all values are contained.
return Expression.Lambda<Func<T, bool>>(contains, p);
}

Rather than making an entirely new method from scratch, you can simply compose the method that you already have that's working.
We can use the following method to combine predicates together:
public static class PredicateBuilder
{
public static Expression<Func<T, bool>> True<T>() { return f => true; }
public static Expression<Func<T, bool>> False<T>() { return f => false; }
public static Expression<Func<T, bool>> Or<T>(
this Expression<Func<T, bool>> expr1,
Expression<Func<T, bool>> expr2)
{
var secondBody = expr2.Body.Replace(expr2.Parameters[0], expr1.Parameters[0]);
return Expression.Lambda<Func<T, bool>>
(Expression.OrElse(expr1.Body, secondBody), expr1.Parameters);
}
public static Expression<Func<T, bool>> And<T>(
this Expression<Func<T, bool>> expr1,
Expression<Func<T, bool>> expr2)
{
var secondBody = expr2.Body.Replace(expr2.Parameters[0], expr1.Parameters[0]);
return Expression.Lambda<Func<T, bool>>
(Expression.AndAlso(expr1.Body, secondBody), expr1.Parameters);
}
}
It relies on the following method to replace all instance of one expression with another:
public static Expression Replace(this Expression expression,
Expression searchEx, Expression replaceEx)
{
return new ReplaceVisitor(searchEx, replaceEx).Visit(expression);
}
internal class ReplaceVisitor : ExpressionVisitor
{
private readonly Expression from, to;
public ReplaceVisitor(Expression from, Expression to)
{
this.from = from;
this.to = to;
}
public override Expression Visit(Expression node)
{
return node == from ? to : base.Visit(node);
}
}
Now all we have to do is call the single value version of AnyColumnContains for each value and Or all of the results together:
public static Expression<Func<T, bool>> AnyColumnContains<T>(IEnumerable<string> values)
{
return values.Select(value => AnyColumnContains<T>(value))
.Aggregate((a, b) => a.Or(b));
}

Trouble with building a C# EntityFramework IQueryable Expression

So I'm attempting to build a semi complication Search expression, but I'm stuck trying to create a basic one. The expressions being used for getValueExpression look something like:
x => x.PropertyA != null ? x.PropertyA.ToShortDateString() : "" //nullable datetime
x => x.PropertyB //string property
x => x.PropertyC != null x.PropertyC.ToString() : "" //nullable int
Here is my function code, it currently errors when getValueExpression being of type Func that can't be compared to a string, which makes perfect sense and I understand why that is, but I'm having trouble figuring out how to make an expression that gets the value of getValueExpression to compare to the value being searched for. Any help or leads in the right direction would be greatly appreciated.
public static IQueryable<TSource> Search<TSource>(this IQueryable<TSource> source, Expression<Func<TSource, string>> getValueExpression, string searchOption, string searchValue)
{
var searchValueExpression = Expression.Constant(searchValue);
var comparisonExpression = Expression.Equal(getValueExpression, searchValueExpression);
var lambdaExpression = Expression.Lambda<Func<TSource, bool>>(comparisonExpression);
return source.Where(lambdaExpression);
}
I've attempted similar things like this, but have met failure with incorrect arguments amount exception:
var getValueExpressionValue = Expression.Call(getValueExpression.Compile().Method, parameterValueExpression);

Here is a method that will let you compose expressions; that is to say you can use the output of one expression as the input of another, creating a new expression taking the input that the first takes and the output that the second takes:
public static Expression<Func<TFirstParam, TResult>>
Compose<TFirstParam, TIntermediate, TResult>(
this Expression<Func<TFirstParam, TIntermediate>> first,
Expression<Func<TIntermediate, TResult>> second)
{
var param = Expression.Parameter(typeof(TFirstParam), "param");
var newFirst = first.Body.Replace(first.Parameters[0], param);
var newSecond = second.Body.Replace(second.Parameters[0], newFirst);
return Expression.Lambda<Func<TFirstParam, TResult>>(newSecond, param);
}
Which uses the following method to replace one expression with another:
internal class ReplaceVisitor : ExpressionVisitor
{
private readonly Expression from, to;
public ReplaceVisitor(Expression from, Expression to)
{
this.from = from;
this.to = to;
}
public override Expression Visit(Expression node)
{
return node == from ? to : base.Visit(node);
}
}
public static Expression Replace(this Expression expression,
Expression searchEx, Expression replaceEx)
{
return new ReplaceVisitor(searchEx, replaceEx).Visit(expression);
}
This lets you write:
public static IQueryable<TSource> Search<TSource>(this IQueryable<TSource> source,
Expression<Func<TSource, string>> getValueExpression,
string searchOption,
string searchValue)
{
var predicate = getValueExpression.Compose(value => value == searchValue);
return source.Where(predicate);
}

Here is how you can do it :
public static IQueryable<TSource> Search<TSource>(this IQueryable<TSource> source, Expression<Func<TSource, string>> getValueExpression, string searchOption, string searchValue)
{
// const searchValue
var searchValueExpression = Expression.Constant(searchValue);
// parameter x
var parameterExpression = Expression.Parameter(typeof(TSource));
// func(x)
var parameterGetValueExpression = Expression.Invoke(getValueExpression, parameterExpression);
// func(x) == searchValue
var comparisonExpression = Expression.Equal(parameterGetValueExpression, searchValueExpression);
// x => func(x) == searchValue
var lambdaExpression = Expression.Lambda<Func<TSource, bool>>(comparisonExpression, parameterExpression);
return source.Where(lambdaExpression);
}

Generic Linq to Entities filter method that accepts filter criteria and properties to be filtered

I've looked into many generic linq filtering questions and their answers here in SO but none of them satisfy my needs so I thought I should create a question.
I've created many of what I call "filter provider" classes, one for each entity class in my model, to provide a simplistic search for my application. I didn't want to go into more advanced solutions like Lucene.Net because a basic filtering with matching score would suffice.
Inside each one of these provider classes there are multiple methods that will receive the filtering terms and query specific properties, returning a score for each match based on the relevance of the property. Most methods will filter multiple properties at once, but not all.
Here are two of these methods:
private IQueryable<Retailer> MatchHighRelevanceFields(string searchTerm, IQueryable<Retailer> retailers)
{
var results = retailers.Where(r =>
(r.CompanyName != null && r.CompanyName.ToUpper().Contains(searchTerm))
|| (r.TradingName != null && r.TradingName.ToUpper().Contains(searchTerm))
);
return results;
}
private IQueryable<Retailer> MatchMediumRelevanceFields(string searchTerm, IQueryable<Retailer> retailers)
{
var results = retailers.Where(r =>
(r.Address.Street != null && r.Address.Street.ToUpper().Contains(searchTerm))
|| (r.Address.Complement != null && r.Address.Complement.ToUpper().Contains(searchTerm))
);
return results;
}
These methods are replicated ad nauseum throughout each provider class and I hope I could replace them for a single method that would receive the properties to be included in the query.
Something like:
public static IQueryable<T> Match<T>(string searchTerm, IQueryable<T> data, Expression<Func<T, string>> filterProperties)
{
var results = **build the query for each property in filterProperties**
return results;
}
But I really can't figure it out. I tried using reflection but it only worked with Linq to Objects and I need a solution for Linq to Entities.

So to solve this problem we need a few puzzle pieces first. The first puzzle piece is a method that can take an expression that computes a value, and then another expression that computes a new value taking the same type the first returns, and creates a new expression that represents the result of passing the result of the first function as the parameter to the second. This allows us to Compose expressions:
public static Expression<Func<TFirstParam, TResult>>
Compose<TFirstParam, TIntermediate, TResult>(
this Expression<Func<TFirstParam, TIntermediate>> first,
Expression<Func<TIntermediate, TResult>> second)
{
var param = Expression.Parameter(typeof(TFirstParam), "param");
var newFirst = first.Body.Replace(first.Parameters[0], param);
var newSecond = second.Body.Replace(second.Parameters[0], newFirst);
return Expression.Lambda<Func<TFirstParam, TResult>>(newSecond, param);
}
This relies on the following tool to replace all instances of one expression with another:
public static Expression Replace(this Expression expression,
Expression searchEx, Expression replaceEx)
{
return new ReplaceVisitor(searchEx, replaceEx).Visit(expression);
}
internal class ReplaceVisitor : ExpressionVisitor
{
private readonly Expression from, to;
public ReplaceVisitor(Expression from, Expression to)
{
this.from = from;
this.to = to;
}
public override Expression Visit(Expression node)
{
return node == from ? to : base.Visit(node);
}
}
We'll also need a tool to help us OR two predicate expressions together:
public static class PredicateBuilder
{
public static Expression<Func<T, bool>> True<T>() { return f => true; }
public static Expression<Func<T, bool>> False<T>() { return f => false; }
public static Expression<Func<T, bool>> Or<T>(
this Expression<Func<T, bool>> expr1,
Expression<Func<T, bool>> expr2)
{
var secondBody = expr2.Body.Replace(
expr2.Parameters[0], expr1.Parameters[0]);
return Expression.Lambda<Func<T, bool>>
(Expression.OrElse(expr1.Body, secondBody), expr1.Parameters);
}
public static Expression<Func<T, bool>> And<T>(
this Expression<Func<T, bool>> expr1,
Expression<Func<T, bool>> expr2)
{
var secondBody = expr2.Body.Replace(
expr2.Parameters[0], expr1.Parameters[0]);
return Expression.Lambda<Func<T, bool>>
(Expression.AndAlso(expr1.Body, secondBody), expr1.Parameters);
}
}
Now that we have this we can use Compose on each property selector to map it from the property results to whether or not that property value is non-null and contains the search term. We can then OR all of those predicates together to get a filter for your query:
public static IQueryable<T> Match<T>(
IQueryable<T> data,
string searchTerm,
IEnumerable<Expression<Func<T, string>>> filterProperties)
{
var predicates = filterProperties.Select(selector =>
selector.Compose(value =>
value != null && value.Contains(searchTerm)));
var filter = predicates.Aggregate(
PredicateBuilder.False<T>(),
(aggregate, next) => aggregate.Or(next));
return data.Where(filter);
}

You can do it with expression trees but it's not as simple as you might think.
public static IQueryable<T> Match<T>(this IQueryable<T> data, string searchTerm,
params Expression<Func<T, string>>[] filterProperties)
{
var parameter = Expression.Parameter(typeof (T), "source");
Expression body = null;
foreach (var prop in filterProperties)
{
// need to replace all the expressions with the one parameter (gist taken from Colin Meek blog see link on top of class)
//prop.body should be the member expression
var propValue =
prop.Body.ReplaceParameters(new Dictionary<ParameterExpression, ParameterExpression>()
{
{prop.Parameters[0], parameter}
});
// is null check
var isNull = Expression.NotEqual(propValue, Expression.Constant(null, typeof(string)));
// create a tuple so EF will parameterize the sql call
var searchTuple = Tuple.Create(searchTerm);
var matchTerm = Expression.Property(Expression.Constant(searchTuple), "Item1");
// call ToUpper
var toUpper = Expression.Call(propValue, "ToUpper", null);
// Call contains on the ToUpper
var contains = Expression.Call(toUpper, "Contains", null, matchTerm);
// And not null and contains
var and = Expression.AndAlso(isNull, contains);
// or in any additional properties
body = body == null ? and : Expression.OrElse(body, and);
}
if (body != null)
{
var where = Expression.Call(typeof (Queryable), "Where", new[] {typeof (T)}, data.Expression,
Expression.Lambda<Func<T, bool>>(body, parameter));
return data.Provider.CreateQuery<T>(where);
}
return data;
}
public static Expression ReplaceParameters(this Expression exp, IDictionary<ParameterExpression, ParameterExpression> map)
{
return new ParameterRebinder(map).Visit(exp);
}
Now you need to have a expressionvisitor to make all the expressions use one parameter
//http://blogs.msdn.com/b/meek/archive/2008/05/02/linq-to-entities-combining-predicates.aspx
public class ParameterRebinder : ExpressionVisitor
{
private readonly IDictionary<ParameterExpression, ParameterExpression> _map;
public ParameterRebinder(IDictionary<ParameterExpression, ParameterExpression> map)
{
_map = map;
}
protected override Expression VisitParameter(ParameterExpression node)
{
if (_map.ContainsKey(node))
{
return _map[node];
}
return base.VisitParameter(node);
}
}
Would use it like
var matches = retailers.Match("7", r => r.Address.Street, x => x.Address.Complement).ToList();
Warning - I checked this with linq to objects using the AsQueryable but didn't run it against EF.

You can use Linq.Dynamic to build the query.
public static IQueryable<T> Match<T>(
string searchTerm,
IQueryable<T> data,
params Expression<Func<T, string>>[] filterProperties) where T : class
{
var predicates = new List<string>();
foreach (var prop in filterProperties)
{
var lambda = prop.ToString();
var columnName = lambda.Substring(lambda.IndexOf('.') + 1);
var predicate = string.Format(
"({0} != null && {0}.ToUpper().Contains(#0))", columnName);
predicates.Add(predicate);
}
var filter = string.Join("||", predicates);
var results = data.Where(filter, searchTerm);
return results;
}
Usage.
var retailers = Match(
"asd", db.Retailers, r => r.CompanyName, r => r.TradingName);
var retailers = Match(
"asd", db.Retailers, r => r.Address.Street, r => r.Address.Complement);
Limitation.
The filter can only accept basic expression.
r => r.Name
r => r.PropA.Name
r => r.PropA.PropB.Name

Try to use Expressions like those all
http://www.codeproject.com/Articles/493917/Dynamic-Querying-with-LINQ-to-Entities-and-Express

Create linq query to search for contact the same way smartphone will do so

Let's say that in my database I have the table
**Table Contact**
Id, FirstName, LastName, Phone, Email, DateCreated
1 Tom Williams 3052548623 tom#gmail.com 2013-12-21 14:51:08
etc...
I would like to enable users to search for a contact entering a string. Let's say the user enters:
tom -> TRUE
tom wil -> TRUE
wil tom -> TRUE
tom XX -> FALSE
t w 3 # -> TRUE
wil 305 -> TRUE
(True means search found customer Tom, False means it did not find it)
I will be performing this type of search among different tables in my database. It will be nice if I dont have to build the query for a specific table.
The approach I am thinking on taking is to split the search string every time I find one or more spaces. Then I will be creating n number of searches and then performing an intersect?

You could do something like that, assuming you want only search in the string properties (so your samples will work if we considere Phone as a string property).
It's of course doable with numeric properties (but gets more complicated).
A method like that in an helper static class
public static Expression<Func<T, bool>> BuildPredicateForFilter<T>(string filterString)
{
//first, split search by space, removing white spaces, and putting this to upper case
var filters = filterString.Split(new []{" "}, StringSplitOptions.RemoveEmptyEntries).Select(m => m.ToUpper());
var parameter = Expression.Parameter(typeof (T), "m");
//get string.Contains() method
var containsMethod = typeof (string).GetMethod("Contains");
//get string.ToUpper() method
var toUpperMethod = typeof (string).GetMethod("ToUpper", new Type[]{});
//find all the string properties of your class
var properties = typeof(T).GetProperties().Where(m => m.PropertyType == typeof(string));
//for all the string properties, build a "m.<PropertyName>.ToUpper() expression
var members = properties.Select(p => Expression.Call(Expression.Property(parameter, p), toUpperMethod));
Expression orExpression = null;
//build the expression
foreach (var filter in filters)
{
Expression innerExpression = null;
foreach (var member in members)
{
innerExpression = innerExpression == null
? (Expression)Expression.Call(member, containsMethod, Expression.Constant(filter))
: Expression.OrElse(innerExpression, Expression.Call(member, containsMethod, Expression.Constant(filter)));
}
orExpression = orExpression == null
? innerExpression
: Expression.AndAlso(orExpression, innerExpression);
}
return Expression.Lambda<Func<T, bool>>(orExpression, new[]{parameter});
}
usage :
var result = <yourSource>.Where(Helper.BuildPredicateForFilter<TableName>("tom XX"));
for example, with "tom XX", the orExpression will look like
((((m.FirstName.ToUpper().Contains("TOM") OrElse
m.LastName.ToUpper().Contains("TOM")) OrElse
m.Phone.ToUpper().Contains("TOM"))
OrElse m.Email.ToUpper().Contains("TOM"))
AndAlso
(((m.FirstName.ToUpper().Contains("XX") OrElse
m.LastName.ToUpper().Contains("XX")) OrElse
m.Phone.ToUpper().Contains("XX")) OrElse
m.Email.ToUpper().Contains("XX")))
EDIT
or you could change the method to
public static IQueryable<T> FilterFor(this IQueryable<T> queryable, string filterString) {
//same
var predicate = Expression.Lambda<Func<T, bool>>(orExpression, new[]{parameter});
return queryable.Where(predicate);
}
then usage would simply be
<yourSource>.FilterFor("tom XX");

So what we're looking to do here is search through all of the fields in a type for a given value, doing a Contains search. We can write a method to do this.
First we'll need to use a PredicateBuilder, as we'll be dynamically generating a number of expressions that we want to OR together. Here is my definition of a PredicateBuilder capable of doing that:
public static class PredicateBuilder
{
public static Expression<Func<T, bool>> True<T>() { return f => true; }
public static Expression<Func<T, bool>> False<T>() { return f => false; }
public static Expression<Func<T, bool>> Or<T>(
this Expression<Func<T, bool>> expr1,
Expression<Func<T, bool>> expr2)
{
var secondBody = expr2.Body.Replace(expr2.Parameters[0], expr1.Parameters[0]);
return Expression.Lambda<Func<T, bool>>
(Expression.OrElse(expr1.Body, secondBody), expr1.Parameters);
}
public static Expression<Func<T, bool>> And<T>(
this Expression<Func<T, bool>> expr1,
Expression<Func<T, bool>> expr2)
{
var secondBody = expr2.Body.Replace(expr2.Parameters[0], expr1.Parameters[0]);
return Expression.Lambda<Func<T, bool>>
(Expression.AndAlso(expr1.Body, secondBody), expr1.Parameters);
}
}
This uses the following helper method/class to replace all instances of one Expression with another:
internal class ReplaceVisitor : ExpressionVisitor
{
private readonly Expression from, to;
public ReplaceVisitor(Expression from, Expression to)
{
this.from = from;
this.to = to;
}
public override Expression Visit(Expression node)
{
return node == from ? to : base.Visit(node);
}
}
public static Expression Replace(this Expression expression,
Expression searchEx, Expression replaceEx)
{
return new ReplaceVisitor(searchEx, replaceEx).Visit(expression);
}
Another tool that we'll use to solve this problem is a Compose method. It will take one expression, then another expression that takes as input the output of another, and produces a new expression that takes the input of the first and produces the output of the last.
public static Expression<Func<TFirstParam, TResult>>
Compose<TFirstParam, TIntermediate, TResult>(
this Expression<Func<TFirstParam, TIntermediate>> first,
Expression<Func<TIntermediate, TResult>> second)
{
var param = Expression.Parameter(typeof(TFirstParam), "param");
var newFirst = first.Body.Replace(first.Parameters[0], param);
var newSecond = second.Body.Replace(second.Parameters[0], newFirst);
return Expression.Lambda<Func<TFirstParam, TResult>>(newSecond, param);
}
Thanks to all of these tools, what's left is actually quite straightforward. We'll accept a query, a string to search for, and a series of selectors, each selecting out a field to search through. Then we initialize a filter, go through every single selector, use Compose to turn each selector into a predicate that performs a Contains check on the relevant search text, and then ORs that to the existing filter.
public static IQueryable<T> AnyFieldContains<T>(
this IQueryable<T> query,
string searchText,
params Expression<Func<T, string>>[] fieldSelectors)
{
var filter = PredicateBuilder.False<T>();
foreach (var selector in fieldSelectors)
{
filter = filter.Or(selector.Compose(
value => value.Contains(searchText)));
}
return query.Where(filter);
}
Now that we have all of this we can split the input that you have, and for each of those expressions we can call this method. Then you simply need to supply selectors for the fields that need to be searched through:
IQueryable<Foo> query = db.Foo;
string searchText = "wil tom";
var searchExpressions = searchText.Split(' ');
foreach (var expression in searchExpressions)
{
query = query.AnyFieldContains(expression,
foo => foo.FirstName,
foo => foo.LastName,
foo => foo.Phone);
}
var result = query.Any();
If you're really sure that you want to search every field (and I'm not sure if you are, it's likely many tables will have fields that shouldn't be searched, or have fields that will need some sort of work on your end to transform them into an appropriate string worth searching), then you can use reflection to generate all of the selectors, rather than typing out explicitly what you want to have searched. We can simply create an additional overload such that if no selectors are provided it will use "everything":
public static IQueryable<T> AnyFieldContains<T>(
this IQueryable<T> query,
string searchText)
{
return AnyFieldContains(query, searchText,
typeof(T).GetProperties()
.Select(prop => CreateSelector<T>(prop))
.ToArray());
}
private static Expression<Func<T, string>> CreateSelector<T>(PropertyInfo prop)
{
var param = Expression.Parameter(typeof(T));
Expression body = Expression.Property(param, prop);
if (prop.PropertyType == typeof(decimal?))
body = Expression.Call(body, typeof(SqlFunctions)
.GetMethod("StringConvert", new[] { typeof(decimal?) }));
else if (prop.PropertyType == typeof(double?))
body = Expression.Call(body, typeof(SqlFunctions)
.GetMethod("StringConvert", new[] { typeof(double?) }));
return Expression.Lambda<Func<T, string>>(body, param);
}

Specification pattern - creating compound specifications using lambdas (C#)

If I have a specification defined as an Expression as below:
public Expression<Func<Foo, bool>> IsSuperhuman =
x => x.CanFly && x.HasXRayVision;
And I want to define another specification 'IsSuperheroine' with the logic 'is superhuman and is female', how can I reuse the existing specification within the new one?

Have you checked out predicate builder in LinqKit? It builds up expressions by letting you and and or expressions together.

Here's a way to do it :
Expression<Func<Foo, bool>> IsSuperhuman = x => x.CanFly && x.HasXRayVision;
Expression<Func<Foo, bool>> IsSuperheroine = AndAlso(IsSuperhuman, x => x.IsFemale);
...
public static Expression<Func<T, TResult>> AndAlso<T, TResult>(Expression<Func<T, TResult>> expr1, Expression<Func<T, TResult>> expr2)
{
var arg = Expression.Parameter(typeof(T), expr1.Parameters[0].Name);
var andExpr = Expression.AndAlso(
ReplaceParameter(expr1.Body, expr1.Parameters[0], arg),
ReplaceParameter(expr2.Body, expr2.Parameters[0], arg));
return Expression.Lambda<Func<T, TResult>>(andExpr, arg);
}
public static Expression ReplaceParameter(Expression expr, ParameterExpression oldParam, ParameterExpression newParam)
{
return new ReplaceParameterVisitor(oldParam, newParam).Visit(expr);
}
internal class ReplaceParameterVisitor : ExpressionVisitor
{
private ParameterExpression _oldParam;
private ParameterExpression _newParam;
public ReplaceParameterVisitor(ParameterExpression oldParam, ParameterExpression newParam)
{
_oldParam = oldParam;
_newParam = newParam;
}
protected override Expression VisitParameter(ParameterExpression node)
{
if (node == _oldParam)
return _newParam;
return node;
}
}
It is probably not the simplest way to do it, but it works...