Develop Reference

How to use multiple 'Where' expressions and chain them together with AND and OR using C#/.NET?

I am trying to make a filtering system in my web app. The problem is I don't know how many filters will be requested from my client to API. I've build it so the array of the filters comes from a single string like this: ?sizeFilters=big,small,medium
Then I use a string[] names = sizeFilters.Split(','); to get single expression like Where(x => x.listOfSizes.contains(names[index]));
I need also to make the chain of the expression using AND and OR because I am gonna use another filter for example: '?typeFilters=normal,extra,spicy'
So I need to make it that the whole expressions looks like this but it might be few times longer, it needs to work with different size of arrays:
return items Where size is big OR small OR medium AND Where type is normal OR extra OR spicy
Where(x => x.Sizes == "Small" || x => x.Sizes == "Medium" || x => x.Sizes == "Big" &&
x => x.Types == "normal" || x => x.Types == "extra" || x => x.Types == "Spicy")

The simplest option would be, as others have noted, to build your ORs using Enumerable.Contains within an expression; and to build your ANDs by calling Where multiple times.
// using these values as an example
string[] sizeTerms = /* initialize */;
string[] typeTerms = /* initialize */;
IQueryable<Item> items = /* initialize */
if (sizeTerms.Any()) {
items = items.Where(x => sizeTerms.Contains(x.Size));
}
if (typeTerms.Any()) {
items = items.Where(x => typeTerms.Contains(x.Type));
}
If you want, you could wrap this logic into an extension method that takes an expression to filter against, and an IEnumerable<string> for the filter values; and constructs and applies the Contains method:
// using System.Reflection
// using static System.Linq.Expressions.Expression
private static MethodInfo containsMethod = typeof(List<>).GetMethod("Contains");
public static IQueryable<TElement> WhereValues<TElement, TFilterTarget>(
this IQueryable<TElement> qry,
Expression<Func<TElement, TFilterTarget>> targetExpr,
IEnumerable<string> values
) {
var lst = values.ToList();
if (!lst.Any()) { return qry; }
return qry.Where(
Lambda<Expression<Func<TElement, bool>>>(
Call(
Constant(lst),
containsMethod.MakeGenericMethod(typeof(T)),
targetExpr.Body
),
targetExpr.Parameters.ToArray()
)
);
}
and can be called like this:
qry = qry
.WhereValues(x => x.Size, sizeTerms)
.WhereValues(x => x.Type, typeTerms);
One caveat: the query will be built based on the values passed into the method; if they are later changed, the query won't reflect those changes. If this is an issue:
get the appropriate overload of Enumerable.Contains, instead of List.Contains, and
use the overload of Expression.Call which produces a static method call, instead of an instance method call.

I think following should work
var query = _context.Set<[Entity]>();
if (sizeFilterPresent)
{
query = query.Where(r => sizes.Contains(r.Size));
}
if(typesFilterPresent)
{
query = query.Where(r => types.Contains(r.Type));
}
var results = query.ToList();

You can try this,
var result = data.Where(p => sizeFilters.Contains(p.Size) && typeFilters.Contains(p.Type));

You can simply call .Where multiple times to AND expressions together. Dynamically OR-ing expressions together is much more difficult. You'll need to rebuild the Expression graph to include the OrElse operator, and ensure that all expressions are based on the same ParameterExpression.
public class Replacer : ExpressionVisitor
{
private readonly Dictionary<Expression, Expression> _replacements;
public Replacer(IEnumerable<Expression> before, IEnumerable<Expression> after)
{
_replacements = new Dictionary<Expression, Expression>(before.Zip(after, (a, b) => KeyValuePair.Create(a, b)));
}
public override Expression Visit(Expression node)
{
if (node != null && _replacements.TryGetValue(node, out var replace))
return base.Visit(replace);
return base.Visit(node);
}
}
public static Expression<Func<T, bool>> Or<T>(this Expression<Func<T, bool>> expr1, Expression<Func<T, bool>> expr2)
{
if (expr1 == null)
return expr2;
if (expr2 == null)
return expr1;
return Expression.Lambda<Func<T, bool>>(
Expression.OrElse(
expr1.Body,
new Replacer(expr2.Parameters, expr1.Parameters).Visit(expr2.Body)
),
expr1.Parameters);
}
public static Expression<Func<T, bool>> And<T>(this Expression<Func<T, bool>> expr1, Expression<Func<T, bool>> expr2)
{
if (expr1 == null)
return expr2;
if (expr2 == null)
return expr1;
return Expression.Lambda<Func<T, bool>>(
Expression.AndAlso(
expr1.Body,
new Replacer(expr2.Parameters, expr1.Parameters).Visit(expr2.Body)
),
expr1.Parameters);
}
// Usage
Expression<Func<TableObject, bool>> where = null;
if (...)
where = where.Or(x => sizeFilters.Contains(x.Size));
if (...)
where = where.Or(x => typeFilters.Contains(x.Type));
if (where!=null)
query = query.Where(where);

To make it look neat, my advice would be to create and extension method. This way you can use it as any other LINQ method. See extension methods demystified
Assuming your source is an IQuertyable<TSource>.
public static IQueryable<TSource> WhereAnd<TSource>(
this IQueryable<TSource> source,
IEnumerable<Expression<Func<TSource,bool>>> filterPredicates)
{
// TODO: handle null source, expressions;
IQueryable<TSource> filteredSource = source;
foreach (var predicate in filterPredicates)
{
filteredSource = filteredSource.Where(predicate);
}
}
Usage:
var predicates = new List<Expression<Func<TSource,bool>>>()
{
customer => customer.BirthDay.Year <= 1950,
customer => customer.CityId == GetCityId("New York"),
customer => customer.Gender == Gender.Male,
}
var oldNewYorkMaleCustomers = dbContext.Customers.WhereAnd(predicates).ToList();
Note: an empty collection of filterpredicates will filter by no predicate: you get the original data:
var emptyFilter = Queryable.Empty<Expression<Func<Customer, bool>>>();
var allCustomers = dbContext.Customers.WhereAnd(emptyFilter);

Error while re-using an expression using PredicateBuilder in LINQ

I am trying to re-use an expression in a LINQ query but I get an exception saying "Code supposed to be unreachable". I'm not very much versed with PredicateBuilder. So pardon me if I look stupid somewhere. Below are the code snippets.
The expression that I am building via PredicateBuilder for re-use:
private static Expression <Func <DiscussionTopic , bool >> TopicExpr(Guid topicId, TaskMember taskMember, bool invert = false)
{
var predicate = PredicateBuilder.True<DiscussionTopic >();
predicate = predicate.And(topic => topic.TopicType == TopicType.Topic && topic.DiscussionTopicId == topicId);
predicate = predicate.And(ExpressionStore.IsTopicAnsweredExpression(taskMember.PersonId));
predicate = predicate.And(topic => (topic.TopicGroup == null || topic.TopicGroup.TopicVisibility == TopicVisibilityType .AllParticipants ||
(topic.TopicGroup.TopicVisibilityTags.Any(tv => taskMember.Person.PeopleTags.Any(pt => pt.ContentTagId == tv.ContentTagId) || taskMember.TaskMemberTags.Any(tmt => tmt.ContentTagId == tv.ContentTagId)))));
predicate = predicate.And(topic => (topic.TopicVisibility == TopicVisibilityType.AllParticipants ||
(topic.TopicVisibilityTags.Any(tv => taskMember.Person.PeopleTags.Any(pt => pt.ContentTagId == tv.ContentTagId) || taskMember.TaskMemberTags.Any(tmt => tmt.ContentTagId == tv.ContentTagId)))));
return predicate;
}
Next, I am trying to use the above expression in a nested query as below:
public static IQueryable <TaskMember > ApplyTopicsCompletedFilter(this IQueryable<TaskMember > input, Guid[] topicIds, bool invert = false)
{
var predicate = PredicateBuilder.True<TaskMember >();
predicate = predicate.And(taskMember => taskMember.TaskMembership.Discussions.Any(discussion => topicIds.All(topicId =>
discussion.DiscussionTopics.AsQueryable().Any(TopicExpr(topicId, taskMember, invert)))));
input = input.Where(predicate);
return input;
}
Some points to mention which are skipped for brevity:
The ApplyTopicsCompletedFilter function is an extension method that receives an already constructed query as input by the caller. Therefore, AsExpandable() operator is applied in the caller and not here.
ExpressionStore.IsTopicAnsweredExpression in line 5 in TopicExpr is an external query that has a complex logic for checking if a member has answered a topic in a discussion. I have skipped the definition here but it's a normal expression with signature Expression<Func<DiscussionTopic, bool> returned by a function that takes a PersonId as a parameter.
If I remove AsQueryable() from line 4 in ApplyTopicsCompletedFilter function, I receive internal data error. Of course if I do that, I also have to do TopicExpr(....).Compile() in the same line otherwise the code won't compile.
Please help as I have wasted almost a day trying to find a nice way of sharing business logic across different functions in our application. Our code is quite complex and uses a lot of sub-queries as well. That's why I am looking for a way where I can share a piece of logic no matter if it appears in a parent query or a sub-query inside a parent query.
Edit:
I just commented everything inside TopicExpr and now it just looks like this:
private static Expression<Func<DiscussionTopic, bool>> TopicExpr(Guid topicId, TaskMember taskMember, bool invert = false)
{
var predicate = PredicateBuilder.True<DiscussionTopic>();
predicate = predicate.And(topic => topic.TopicType == TopicType.Topic && topic.DiscussionTopicId == topicId);
return predicate;
}
I have done this to make the query simple to test. Here is the complete flow now:
[Route("api/accounts/testPredicate")]
[HttpGet]
public IQueryable<apiPerson> TestPredicate()
{
return this.TestPredicate(this.CurrentAccountId, new[] { MembershipStatusType.Member }, new apiFilterData { TopicsCompleted = new[] { Guid.Parse("<some guid>") } }).Select(tm => tm.Person).ToApiObjects();
}
public IQueryable<TaskMember> TestPredicate(Guid currentAccountId, MembershipStatusType[] statusTypes = null, apiFilterData filterData = null)
{
var query = this.Db.Set<TaskMember>().Where(t => t.Person.AccountId == currentAccountId).AsExpandable();
if (statusTypes != null)
query = query.Where(t => statusTypes.Contains(t.Status));
if (filterData.TopicsCompleted != null && filterData.TopicsCompleted.Length > 0)
query = query.ApplyTopicsCompletedFilterWithPredicate(filterData.TopicsCompleted);
if (filterData.TopicsNotCompleted != null && filterData.TopicsNotCompleted.Length > 0)
query = query.ApplyTopicsCompletedFilterWithPredicate(filterData.TopicsNotCompleted, true);
return query;
}
private static Expression<Func<DiscussionTopic, bool>> TopicExpr(Guid topicId, TaskMember taskMember, bool invert = false)
{
var predicate = PredicateBuilder.True<DiscussionTopic>();
predicate = predicate.And(topic => topic.TopicType == TopicType.Topic && topic.DiscussionTopicId == topicId);
return predicate;
}
public static IQueryable<TaskMember> ApplyTopicsCompletedFilterWithPredicate(this IQueryable<TaskMember> input, Guid[] topicIds, bool invert = false)
{
var predicate = PredicateBuilder.True<TaskMember>();
predicate = predicate.And(taskMember => taskMember.TaskMembership.Discussions.AsQueryable().Any(discussion => topicIds.All(topicId =>
discussion.DiscussionTopics.AsQueryable().Any(TopicExpr(topicId, taskMember, invert)))));
input = input.Where(predicate);
return input;
}
public static IQueryable<apiTopicSimple> ToApiObjects(this IQueryable<DiscussionTopic> items, Guid currentUserId, MembershipStatusType membershipStatus, bool stagingSite)
{
return from t in items
let ms = membershipStatus
let uId = currentUserId
let sS = stagingSite
select new apiTopicSimple
{
//Property initialization
};
}
PS: These functions are in different files inside the project but I have put them here at one place for definition purposes so that someone can go through the complete flow.

My ObjectSet filter doesn't get translated into SQL

I have written this filter to get only documents that match certain periods of time from the database :
The Period entity is straightforward and contains two properties : DateFrom and DateTo.
I need to build a filter from lambdas, each one for each Period that is submitted to build the filter.
The filter, when is completely built, has to look like this :
ObjectSet.Where(d =>
(d.Date >= Period1.DateFrom && d.Date <= Period1.DateTo)
|| (d.Date >= Period2.DateFrom && d.Date <= Period2.DateTo)
|| (d.Date >= Period3.DateFrom && d.Date <= Period3.DateTo));
As you can guess, I have to dynamically build this filter because the number of submitted periods to build the filter can vary.
(The following is the Expression I use to combine the lambdas (each one for each period of time that have been submitted to build the filter)
private Expression<Func<T, bool>> CombineWithOr<T>(
Expression<Func<T, bool>> firstExpression,
Expression<Func<T, bool>> secondExpression)
{
var parameter = Expression.Parameter(typeof(T), "x");
var resultBody = Expression.Or(
Expression.Invoke(firstExpression, parameter),
Expression.Invoke(secondExpression, parameter));
return Expression.Lambda<Func<T, bool>>(resultBody, parameter);
}
Here is where I combine each lambda for each period there is to add to the document filter :
public IList<Document> GetDocuments(IList<Periods> periods)
{
Expression<Func<Document, bool>> resultExpression = n => false;
foreach (var submittedPeriod in periods)
{
var period = submittedPeriod;
Expression<Func<Document, bool>> expression =
d => (d.Date >= period.DateFrom && d.Date <= period.DateTo);
resultExpression = this.CombineWithOr(resultExpression, expression);
}
var query = this.ObjectSet.Where(resultExpression.Compile());
}
The problem is, when I launch deferred execution of the query ...
var documents = query.ToList();
... and I look at the resulting SQL, nothing is added to the SELECT statement.
If I execute the query without compiling the resulting Expression like this :
var query = this.ObjectSet.Where(resultExpression);
I get this exception :
The LINQ expression node type 'Invoke' is not supported in LINQ to
Entities.
That means that the Linq-To-Entities query provider doesn't know how to translate my filter into SQL code.
What is bugging me right now is how such a simple DateTime comparison from entities (Document and Period) that are both part of my Entity schema can mess up the provider ?
Any ideas how I can achieve such a filtering ?

try .AsExpandable() before the query. More information is here

The problem is with your combining function. SQL to Entities doesn't really like Invoke. Here's a combining function that worked for me:
public static class ExpressionCombiner
{
public static Expression<Func<T, bool>> Or<T>(Expression<Func<T, bool>> a, Expression<Func<T, bool>> b)
{
var parameter = Expression.Parameter(typeof(T), "x");
var substituter = new SubstituteParameter(parameter, p => true);
var resultBody = Expression.Or(
substituter.Visit(a.Body),
substituter.Visit(b.Body));
Expression<Func<T, bool>> combined = Expression.Lambda<Func<T, bool>>(resultBody, parameter);
return combined;
}
}
public class SubstituteParameter : ExpressionVisitor
{
private ParameterExpression toReplace;
private Func<ParameterExpression, bool> isReplacementRequiredFunc;
public SubstituteParameter(ParameterExpression toReplace, Func<ParameterExpression, bool> isReplacementRequiredFunc)
{
this.toReplace = toReplace;
this.isReplacementRequiredFunc = isReplacementRequiredFunc;
}
protected override Expression VisitParameter(ParameterExpression node)
{
return isReplacementRequiredFunc(node) ? toReplace : node;
}
}

Dynamically building queries

Hi and thanks for taking the time to answer my question.
After a year and a half of working with Java, I've decided to switch back to .NET. I must say that I feel at home in VS2012.
While working with Java I came across an implementation of hibernate that enabled for creating dynamic queries easily.
Imagine I had a form with 5 fields of which only one, any one, must be populated in order for me to filter the results by.
is there a way I can do the following in C#:
if(txtMunicipality.text.length > 0){
(x => x.municipality == txtMunicipality.text)
}
if(chkboxIsFinished){
(x => x.isfinished == true)
}
etc..
So I ccheck for every field and if the value has been populated then add that criteria to the query.. and after i'm done with the checks i execute the query. Is there a way to do this in C#?

The simplest way to do this is to compose two queries, i.e.
IQueryable<Foo> query = ... // or possibly IEnumerable<Foo>
if(!string.IsNullOrEmpty(txtMunicipality.text)) {
query = query.Where(x => x.municipality == txtMunicipality.text);
}
if(chkboxIsFinished) {
query = query.Where(x.isfinished);
}
You can also directly compose expression-trees and delegates; if you need that, please indicate which you have: an expression-tree vs a delegate.
Edit: here's how you would do that composing expressions rather than queries:
static class Program
{
static void Main()
{
Expression<Func<int, bool>> exp1 = x => x > 4;
Expression<Func<int, bool>> exp2 = x => x < 10;
Expression<Func<int, bool>> exp3 = x => x == 36;
var combined = (exp1.AndAlso(exp2)).OrElse(exp3);
// ^^^ equiv to x => (x > 4 && x < 10) || x == 36
}
static Expression<Func<T, bool>> OrElse<T>(this Expression<Func<T, bool>> x, Expression<Func<T, bool>> y)
{ // trivial cases
if (x == null) return y;
if (y == null) return x;
// rewrite using the parameter from x throughout
return Expression.Lambda<Func<T, bool>>(
Expression.OrElse(
x.Body,
SwapVisitor.Replace(y.Body, y.Parameters[0], x.Parameters[0])
), x.Parameters);
}
static Expression<Func<T, bool>> AndAlso<T>(this Expression<Func<T, bool>> x, Expression<Func<T, bool>> y)
{ // trivial cases
if (x == null) return y;
if (y == null) return x;
// rewrite using the parameter from x throughout
return Expression.Lambda<Func<T, bool>>(
Expression.AndAlso(
x.Body,
SwapVisitor.Replace(y.Body, y.Parameters[0], x.Parameters[0])
), x.Parameters);
}
class SwapVisitor : ExpressionVisitor
{
public static Expression Replace(Expression body, Expression from, Expression to)
{
return new SwapVisitor(from, to).Visit(body);
}
private readonly Expression from, to;
private SwapVisitor(Expression from, Expression to)
{
this.from = from;
this.to = to;
}
public override Expression Visit(Expression node)
{
return node == from ? to : base.Visit(node);
}
}
}

Yes, it is possible. The simplest way is with delegates, especially the anonymous ones.
For example:
Func<YourEntity, bool> filter = (_ => true); // Default value.
if (txtMunicipality.text.length > 0)
{
filter = (x => x.municipality == txtMunicipality.text);
}
else if (chkboxIsFinished)
{
filter = (x => x.isfinished == true);
}
Then you can use the filter delegate in a query, for example in a Where statement (which I suppose was your intent - if not, the example is still relevant, just not directly applicable)
/ LINQ syntax.
var entities = from e in context
where filter(e)
select e;
// Method syntax.
var entities = context.Where(x => filter(x));
// Or simply:
var entities = context.Where(filter);

In this article you can find some useful extension methods that allows you to combine predicates (it should work for NHibernate as well):
LINQ to Entities: Combining Predicates
You can then build a lambda expression like this:
Expression<Func<MyObject, bool>> predicate = x => true;
if(txtMunicipality.text.length > 0){
predicate = predicate.And(x => x.municipality == txtMunicipality.text);
}
if(chkboxIsFinished){
predicate = predicate.And(x => x.isfinished == true);
}

How to make a compound "or" clause in Linq?

If you're adding "and" conditions to a Linq query, it's easy to do it like so:
var q = MyTable;
if (condition1)
q = q.Where(t => t.Field1 == value1);
if (condition2)
q = q.Where(t => t.Field2 > t.Field3);
// etc.
Is there any clever way of doing the same thing, when you want to add "or" conditions?

You can use a PredicateBuilder and use it to build an Or based expression:
var predicate = PredicateBuilder.False<Product>();
predicate = predicate.Or (t => t.Field1 == value1);
predicate = predicate.Or (t => t.Field2 > t.Field3);
q = q.Where (predicate);
You can read more about it here:
http://www.albahari.com/nutshell/predicatebuilder.aspx
Replace the Product in PredicateBuilder.False<Product>() with your queried object.
Note that you start from a False predicate as you want to use Or. If You'd want an And predicate, Yuo should start from a True

Use the following:
var q = MyTable;
q = q.Where(
t => (condition1 && t.Field1 == value1) || (condition2 && t.Field2 > t.Field3));

var q = MyTable;
var conditions = new List<Func<T, bool>>();
if (condition1)
conditions.Add(t => ...);
if (condition2)
conditions.Add(t => ...);
q.Where(x => conditions.Any(y => y(x)));

There is one way to do this that involves using expression trees. This way you build the Boolean expression yourself. It's pretty straight forward the tricky part though is that you need to rebase the parameters because otherwise it is going to refer the original lambda expression. See below for an example:
static void Main(string[] args)
{
var source = new List<int> { 1, 2, 3 };
var any = new List<Expression<Func<int, bool>>>();
any.Add(x => x == 1);
any.Add(x => x == 3);
foreach (var item in source.AsQueryable().WhereDisjunction(any))
{
Console.WriteLine(item);
}
}
class RewriteSingleParameterUsage : ExpressionVisitor
{
public ParameterExpression Parameter { get; set; }
protected override Expression VisitParameter(ParameterExpression node)
{
return Parameter;
}
}
public static IQueryable<T> WhereDisjunction<T>(this IQueryable<T> source, IList<Expression<Func<T, bool>>> any)
{
switch (any.Count)
{
case 0: return source;
case 1: return source.Where(any[0]);
default:
var p = Expression.Parameter(any[0].Parameters[0].Type, any[0].Parameters[0].Name);
var rw = new RewriteSingleParameterUsage { Parameter = p };
var expr = rw.Visit(any[0].Body);
for (int i = 1; i < any.Count; i++)
{
expr = Expression.Or(expr, rw.Visit(any[i].Body));
}
return source.Where(Expression.Lambda<Func<T, bool>>(expr, p));
}
}
In the above example I'm being very harsh, I'm effectively replacing any parameter with this single new parameter that is being used to create the new expression. However, given the signature of this extension method it shouldn't really be possible to call this method with parameters such that it would cause an error. It is however going to be a problem if you involve more than one parameter.

This is the same answer I gave here
As Marc Gravell said it involves expression-tree combining.
This article shows you how to do that. It takes a bit of work to initially set it up. But its worth it.
Alternate solution is to use Predicate Builder. The article does not explain very well what is actually happening under-the-hood. But the above article explains it nicely