Develop Reference

Error while re-using an expression using PredicateBuilder in LINQ

I am trying to re-use an expression in a LINQ query but I get an exception saying "Code supposed to be unreachable". I'm not very much versed with PredicateBuilder. So pardon me if I look stupid somewhere. Below are the code snippets.
The expression that I am building via PredicateBuilder for re-use:
private static Expression <Func <DiscussionTopic , bool >> TopicExpr(Guid topicId, TaskMember taskMember, bool invert = false)
{
var predicate = PredicateBuilder.True<DiscussionTopic >();
predicate = predicate.And(topic => topic.TopicType == TopicType.Topic && topic.DiscussionTopicId == topicId);
predicate = predicate.And(ExpressionStore.IsTopicAnsweredExpression(taskMember.PersonId));
predicate = predicate.And(topic => (topic.TopicGroup == null || topic.TopicGroup.TopicVisibility == TopicVisibilityType .AllParticipants ||
(topic.TopicGroup.TopicVisibilityTags.Any(tv => taskMember.Person.PeopleTags.Any(pt => pt.ContentTagId == tv.ContentTagId) || taskMember.TaskMemberTags.Any(tmt => tmt.ContentTagId == tv.ContentTagId)))));
predicate = predicate.And(topic => (topic.TopicVisibility == TopicVisibilityType.AllParticipants ||
(topic.TopicVisibilityTags.Any(tv => taskMember.Person.PeopleTags.Any(pt => pt.ContentTagId == tv.ContentTagId) || taskMember.TaskMemberTags.Any(tmt => tmt.ContentTagId == tv.ContentTagId)))));
return predicate;
}
Next, I am trying to use the above expression in a nested query as below:
public static IQueryable <TaskMember > ApplyTopicsCompletedFilter(this IQueryable<TaskMember > input, Guid[] topicIds, bool invert = false)
{
var predicate = PredicateBuilder.True<TaskMember >();
predicate = predicate.And(taskMember => taskMember.TaskMembership.Discussions.Any(discussion => topicIds.All(topicId =>
discussion.DiscussionTopics.AsQueryable().Any(TopicExpr(topicId, taskMember, invert)))));
input = input.Where(predicate);
return input;
}
Some points to mention which are skipped for brevity:
The ApplyTopicsCompletedFilter function is an extension method that receives an already constructed query as input by the caller. Therefore, AsExpandable() operator is applied in the caller and not here.
ExpressionStore.IsTopicAnsweredExpression in line 5 in TopicExpr is an external query that has a complex logic for checking if a member has answered a topic in a discussion. I have skipped the definition here but it's a normal expression with signature Expression<Func<DiscussionTopic, bool> returned by a function that takes a PersonId as a parameter.
If I remove AsQueryable() from line 4 in ApplyTopicsCompletedFilter function, I receive internal data error. Of course if I do that, I also have to do TopicExpr(....).Compile() in the same line otherwise the code won't compile.
Please help as I have wasted almost a day trying to find a nice way of sharing business logic across different functions in our application. Our code is quite complex and uses a lot of sub-queries as well. That's why I am looking for a way where I can share a piece of logic no matter if it appears in a parent query or a sub-query inside a parent query.
Edit:
I just commented everything inside TopicExpr and now it just looks like this:
private static Expression<Func<DiscussionTopic, bool>> TopicExpr(Guid topicId, TaskMember taskMember, bool invert = false)
{
var predicate = PredicateBuilder.True<DiscussionTopic>();
predicate = predicate.And(topic => topic.TopicType == TopicType.Topic && topic.DiscussionTopicId == topicId);
return predicate;
}
I have done this to make the query simple to test. Here is the complete flow now:
[Route("api/accounts/testPredicate")]
[HttpGet]
public IQueryable<apiPerson> TestPredicate()
{
return this.TestPredicate(this.CurrentAccountId, new[] { MembershipStatusType.Member }, new apiFilterData { TopicsCompleted = new[] { Guid.Parse("<some guid>") } }).Select(tm => tm.Person).ToApiObjects();
}
public IQueryable<TaskMember> TestPredicate(Guid currentAccountId, MembershipStatusType[] statusTypes = null, apiFilterData filterData = null)
{
var query = this.Db.Set<TaskMember>().Where(t => t.Person.AccountId == currentAccountId).AsExpandable();
if (statusTypes != null)
query = query.Where(t => statusTypes.Contains(t.Status));
if (filterData.TopicsCompleted != null && filterData.TopicsCompleted.Length > 0)
query = query.ApplyTopicsCompletedFilterWithPredicate(filterData.TopicsCompleted);
if (filterData.TopicsNotCompleted != null && filterData.TopicsNotCompleted.Length > 0)
query = query.ApplyTopicsCompletedFilterWithPredicate(filterData.TopicsNotCompleted, true);
return query;
}
private static Expression<Func<DiscussionTopic, bool>> TopicExpr(Guid topicId, TaskMember taskMember, bool invert = false)
{
var predicate = PredicateBuilder.True<DiscussionTopic>();
predicate = predicate.And(topic => topic.TopicType == TopicType.Topic && topic.DiscussionTopicId == topicId);
return predicate;
}
public static IQueryable<TaskMember> ApplyTopicsCompletedFilterWithPredicate(this IQueryable<TaskMember> input, Guid[] topicIds, bool invert = false)
{
var predicate = PredicateBuilder.True<TaskMember>();
predicate = predicate.And(taskMember => taskMember.TaskMembership.Discussions.AsQueryable().Any(discussion => topicIds.All(topicId =>
discussion.DiscussionTopics.AsQueryable().Any(TopicExpr(topicId, taskMember, invert)))));
input = input.Where(predicate);
return input;
}
public static IQueryable<apiTopicSimple> ToApiObjects(this IQueryable<DiscussionTopic> items, Guid currentUserId, MembershipStatusType membershipStatus, bool stagingSite)
{
return from t in items
let ms = membershipStatus
let uId = currentUserId
let sS = stagingSite
select new apiTopicSimple
{
//Property initialization
};
}
PS: These functions are in different files inside the project but I have put them here at one place for definition purposes so that someone can go through the complete flow.

Creating a composite condition using anonymous filter method

I am trying to edit a search tool using linq,
What I like a filter in where clause is (ItemNumber == X AND ( StatementStatus == SatusA Or StatementStatus == StatusB ) )
But right now, it is like:
What I like a filter in where clause is (ItemNumber == X AND StatementStatus == SatusA Or StatementStatus == StatusB )
as AND has higher operational priority over OR the result is not what I want. :)
Could you please help?
using (var ctx = new MyContext()) {
Func<Statement, bool> filter = null;
if (!string.IsNullOrEmpty(request.ItemNumber))
filter = new Func<Statement, bool>(s => s.StatementDetails.Any(sd => sd.ItemNumber == request.ItemNumber));
if (request.StatusA)
filter = filter == null ? new Func<Statement, bool>(s => s.StatementStatus == StatementStatusType.StatusA) :
filter.And(s => s.StatementStatus == StatementStatusType.StatusA);
if (request.StatusB)
filter = filter == null ? new Func<Statement, bool>(s => s.StatementStatus == StatementStatusType.StatusB) :
filter.Or(s => s.StatementStatus == StatementStatusType.StatusB);
var results = ctx.Statements
.Include("StatementDetails")
.Include("StatementDetails.Entry")
.Where(filter)
.Take(100)
.Select(s => new StatementSearchResultDTO{ ....
}
}

That's happens not because AND have higher priority than OR. What happens in reality:
var firstFilter = ...; // itemNumber
var secondFilter = ...; // statusA
var firstAndSecondFilter = firstFilter.And(secondFilter); // itemNumber && statusA
var thirdFilter = ...; // statusB
var endFilter = firstAndSecondFilter.Or(thirdFilter) // (itemNumber && statusA) || statusB.
The problem - wrong control flow. You must to do something like that:
var filterByA = ...;
var filterByB = ...;
var filterByAorB = filterByA.Or(filterByB);
var filterByNumber = ...;
var endFiler = filterByNumber.And(filterByAorB);
And your code is bad, not just because it works wrong, but because it's hard to write code in such style. Reasons:
This code doesn't follow DRY principle. You have two same lambdas that checks for StatusA (look in your ternary operator) and two same lambdas that checks for StatusB
You have too long ternary operator with null checks. That's bad because you don't see general picture, your eyes focused on syntax problems. You may write and extension method AndNullable for funcs. Like this:
static Func<T1, TOut> AndNullable<T1, TOut>(this Func<T1, TOut> firstFunc, Func<T1, TOut> secondFunc) {
if (firstFunc != null) {
if (secondFunc != null)
return firstFunc.And(secondFunc);
else
return firstFunc;
}
else {
if (secondFunc != null)
return secondFunc;
else
return null;
}
}
And that same for Or. Now your code can be wroted like this:
Func<Statement, bool> filter = null;
if (request.StatusA)
filter = s => s.StatementStatus == StatementStatusType.StatusA;
if (request.StatusB)
filter = filter.OrNullable(s => s.StatementStatus == StatementStatusType.StatusB);
if (!string.IsNullOrEmpty(request.ItemNumber))
filter = filter.AndNullable(s => s.StatementDetails.Any(sd => sd.ItemNumber == request.ItemNumber));
Reads more better.
Your filter is global filter. Writing of global filter is simpler for few filter conditions and number of lines is small, but it's more complicated to understand your filter. Rewrite it in this way:
Func<Statement, bool> filterByStatusA = null;
Func<Statement, bool> filterByStatusB = null;
if (request.StatusA)
filterByStatusA = s => s.StatementStatus == StatementStatusType.StatusA;
if (request.StatusB)
filterByStatusB = s => s.StatementStatus == StatementStatusType.StatusB;
Func<Statement, bool> filterByStatuses = filterByStatusA.OrNullable(filterByStatusB);
Func<Statement, bool> filterByItemNumber = null;
if (!string.IsNullOrEmpty(request.ItemNumber))
filterByItemNumber = s => s.StatementDetails.Any(sd => sd.ItemNumber == request.ItemNumber);
Func<Statement, bool> endFilter = filterByItemNumber.And(filterByStatuses);
Okay, we have outthinked how we can write filters by combining them as Func<..> but we still have problems.
What problems we will got, if result filter is null? Answer: ArgumentNullException due to documentation. We must to think about this case.
What another problems we can got with using of simple Func<...>? Well, you must to know difference between IEnumerable<T> and IQueryable<T> interfaces. In simple words, all operations on IEnumerable causes simple iteratation over all elements (well, it's lazy, IEnumerable really slower than IQueryable). So, for example, combining of Where(filter), Take(100), ToList() on collection that have 10000 elements that are bad for this filter and 400 elements that are good will cause iterating over 10100 elements. If you wrote similar code for IQueryable the request of filtering will send on database server and this server will iterate only ~400 (or 1000, but not 10100), if you have configured indexes on database. So what happens in your code.
var results = ctx.Statements // you are getting DbSet<Statement> that implements interface IQueryable<Statement> (and IQueryable<T> implements IEnumerable<T>)
.Include("StatementDetails") // still IQueryable<Statement>
.Include("StatementDetails.Entry") // still IQueryable<Statement>
.Where(filter) // Cuz your filter is Func<..> and there are no extension methods on IQueryable that accepts Func<...> as parameter, your IQueryable<Statement> casted automatically to IEnumerable<Statement>. Full collection will be loaded in your memory and only then filtered. That's bad
.Take(100) // IEnumerable<Statement>
.Select(s => new StatementSearchResultDTO { .... // IEnumerable<Statement> -> IEnumerable<StatementSearchResultDTO>
}
Okay. Now you understand the problem. So, simple right code for you can be writed in this way:
using (var ctx = new MyContext()) {
results = ctx.Statements
.Include("StatementDetails")
.Include("StatementDetails.Entry")
.AsQueryable();
if (!string.IsNullOrEmpty(request.ItemNumber))
results = results.Where(s => s.StatementDetails.Any(sd => sd.ItemNumber == request.ItemNumber));
if (request.StatusA) {
if (request.StatusB)
results = results.Where(s => s.StatementStatus == StatementStatusType.StatusA ||
s.StatementStatus == StatementStatusType.StatusA);
else
results = results.Where(s => s.StatementStatus == StatementStatusType.StatusA);
}
else {
if (request.StatusB) {
results = results.Where(s => s.StatementStatus == StatementStatusType.StatusB);
}
else {
// do nothing
}
}
results = .Take(100)
.Select(s => new StatementSearchResultDTO{ ....
};
// .. now you can you results.
}
Yeah, totally ugly, but now your database solves how to find Statements that satisfy the filter. Therefore, this request is quickly as possible. Now we must understand what magic happens in code I written upper. Let's compare two examples of code:
results = results.Where(s => s.StatementDetails.Any(sd => sd.ItemNumber == request.ItemNumber));
And this:
Func<Statement, bool> filter = s => s.StatementDetails.Any(sd => sd.ItemNumber == request.ItemNumber);
results = results.Where(filter);
What the difference? Why first is more faster? Answer: when compiler sees first code, it examines that type of results is IQueryable<T> and IEnumerable<T> so that condition inside of brackets can have type Func<Statement, bool> (compiled function) or Expression<Func<Statement, bool>> (data, that can be compiled in function). And compiler chooses Expression (why - really dunno, just chooses). After request of first object query compiled not in C# statement, but in SQL statement and sends to server. Your SQL server can optimize request, because of indexes existing.
Well, the more better way - to write your own expressions. There are different ways to write your own expression, but there is a way to write it with not ugly syntax. The problem that you can't just invoke one expression from another - that doesn't supported by Entity Framework and can be not supported by another ORM's. So, we can use PredicateBuilder by Pete Montgomery: link. And then write two simple extensions on expressions suitable for us.
public static Expression<Func<T, bool>> OrNullable<T>(this Expression<Func<T, bool>> first, Expression<Func<T, bool>> second)
{
if (first != null && second != null)
return first.Compose(second, Expression.OrElse);
if (first != null)
return second;
if (second != null)
}
And that same for And. And now we can write our filter:
{
Expression<Func<Statement, bool>> filterByStatusA = null;
Expression<Func<Statement, bool>> filterByStatusB = null;
if (request.StatusA)
filterByStatusA = s => s.StatementStatus == StatementStatusType.StatusA;
if (request.StatusB)
filterByStatusB = s => s.StatementStatus == StatementStatusType.StatusB;
Expression<Func<Statement, bool>> filterByStatuses = filterByStatusA.OrNullable(filterByStatusB);
Expression<Func<Statement, bool>> filterByItemNumber = null;
if (!string.IsNullOrEmpty(request.ItemNumber))
filterByItemNumber = s => s.StatementDetails.Any(sd => sd.ItemNumber == request.ItemNumber);
Expression<Func<Statement, bool>> endFilter = filterByItemNumber.And(filterByStatuses);
requests = ...;
if (endFilter != null)
requests = requests.Where(endFilter);
}
You can got a problem, because class ExpressionVisitor in PredicateBuilder in .NET < 4.0 is sealed. You can get write your own ExpressionVisitor or just copy it from this article.

OK, here is the way I have solved it:
filter.And(s => (request.StatusA && s.StatementStatus == StatementStatusType.StatusA) ||
(request.StatusB && s.StatementStatus == StatementStatusType.StautsB) ||
!(request.StatusA || request.StatusB)); //None selected = All selected
Any comments?

Dynamic Linq Query Help?

How do you write a dynamic Linq query for the following simple search criteria?
1) StudentNumber
2) LastName
3) LastName and FirstName
//if (!String.IsNullOrEmpty(StudentNumber))
var results = (from s in Students
where s.StudentNumber == 1001
select s
);
//else if (!String.IsNullOrEmpty(LastName) & (String.IsNullOrEmpty(FirstName))
var results = (from s in Students
where s.LastName == "Tucker"
select s
);
//else if (!String.IsNullOrEmpty(LastName) & (!String.IsNullOrEmpty(FirstName))
var results = (from s in Students
where s.LastName == "Tucker" && s.FirstName == "Ron"
select s
);

You need to declare your results variable outside of any individual query. This will allow you append different filters based upon your varying criteria, and append as many filters as you need. An example:
var results = Students.AsEnumerable(); // use .AsQueryable() for EF or Linq-to-SQL
if (!string.IsNullorEmpty(StudentNumber))
{
results = results.Where(s => s.StudentNumber.Equals(StudentNumber));
}
else if (!string.IsNullOrEmpty(LastName))
{
results = results.Where(s => s.LastName.Equals(LastName));
if (!string.IsNullOrEmpty(FirstName))
{
results = results.Where(s => s.FirstName.Equals(FirstName));
// filter is in addition to predicate against LastName
}
}
// results can be used here
If dealing with Linq-to-Entities or -Sql, type the initial query with Students.AsQueryable(); so that the filtering happens at the database rather than inside the application.
Is there a way I can construct the WHERE clause first and use it in a
Linq query without if...else
If you want to build the entire where before the first step of the query, it's the same logic. You are conditionally building the predicate, so you will have some sort of if/else involved. However, to build the entire predicate first, you could build against a Func<Student, bool> for Linq to Objects.
Func<Student, bool> predicate;
if (!string.IsNullOrEmpty(StudentNumber))
{
predicate = s => s.StudentNumber.Equals(StudentNumber);
}
else if (!string.IsNullOrEmpty(LastName))
{
predicate = s => s.LastName.Equals(LastName);
if (!string.IsNullOrEmpty(FirstName))
{
Func<Student, bool> p = predicate;
predicate = s => p(s) && s.FirstName.Equals(FirstName);
}
}
else
{
predicate = s => true;
}
var query = Students.Where(predicate);
You'll notice it's the exact same if/else structure. You could collapse that down into a complicated conditional expression
Func<Student, bool> predicate;
predicate = s =>
!string.IsNullOrEmpty(StudentNumber)
? s.StudentNumber.Equals(StudentNumber)
: !string.IsNullOrEmpty(LastName)
? !string.IsNullOrEmpty(FirstName)
? s.LastName.Equals(LastName) && s.FirstName.Equals(FirstName)
: s.LastName.Equals(LastName)
: true;
var query = Students.Where(predicate);
But I find that pretty well difficult to follow, certainly as compared to the longer if/else. This predicate is also bigger than the one we build via the if/else, because this one contains all the logic, it's not just the logic we conditionally added.

How to make a compound "or" clause in Linq?

If you're adding "and" conditions to a Linq query, it's easy to do it like so:
var q = MyTable;
if (condition1)
q = q.Where(t => t.Field1 == value1);
if (condition2)
q = q.Where(t => t.Field2 > t.Field3);
// etc.
Is there any clever way of doing the same thing, when you want to add "or" conditions?

You can use a PredicateBuilder and use it to build an Or based expression:
var predicate = PredicateBuilder.False<Product>();
predicate = predicate.Or (t => t.Field1 == value1);
predicate = predicate.Or (t => t.Field2 > t.Field3);
q = q.Where (predicate);
You can read more about it here:
http://www.albahari.com/nutshell/predicatebuilder.aspx
Replace the Product in PredicateBuilder.False<Product>() with your queried object.
Note that you start from a False predicate as you want to use Or. If You'd want an And predicate, Yuo should start from a True

Use the following:
var q = MyTable;
q = q.Where(
t => (condition1 && t.Field1 == value1) || (condition2 && t.Field2 > t.Field3));

var q = MyTable;
var conditions = new List<Func<T, bool>>();
if (condition1)
conditions.Add(t => ...);
if (condition2)
conditions.Add(t => ...);
q.Where(x => conditions.Any(y => y(x)));

There is one way to do this that involves using expression trees. This way you build the Boolean expression yourself. It's pretty straight forward the tricky part though is that you need to rebase the parameters because otherwise it is going to refer the original lambda expression. See below for an example:
static void Main(string[] args)
{
var source = new List<int> { 1, 2, 3 };
var any = new List<Expression<Func<int, bool>>>();
any.Add(x => x == 1);
any.Add(x => x == 3);
foreach (var item in source.AsQueryable().WhereDisjunction(any))
{
Console.WriteLine(item);
}
}
class RewriteSingleParameterUsage : ExpressionVisitor
{
public ParameterExpression Parameter { get; set; }
protected override Expression VisitParameter(ParameterExpression node)
{
return Parameter;
}
}
public static IQueryable<T> WhereDisjunction<T>(this IQueryable<T> source, IList<Expression<Func<T, bool>>> any)
{
switch (any.Count)
{
case 0: return source;
case 1: return source.Where(any[0]);
default:
var p = Expression.Parameter(any[0].Parameters[0].Type, any[0].Parameters[0].Name);
var rw = new RewriteSingleParameterUsage { Parameter = p };
var expr = rw.Visit(any[0].Body);
for (int i = 1; i < any.Count; i++)
{
expr = Expression.Or(expr, rw.Visit(any[i].Body));
}
return source.Where(Expression.Lambda<Func<T, bool>>(expr, p));
}
}
In the above example I'm being very harsh, I'm effectively replacing any parameter with this single new parameter that is being used to create the new expression. However, given the signature of this extension method it shouldn't really be possible to call this method with parameters such that it would cause an error. It is however going to be a problem if you involve more than one parameter.

This is the same answer I gave here
As Marc Gravell said it involves expression-tree combining.
This article shows you how to do that. It takes a bit of work to initially set it up. But its worth it.
Alternate solution is to use Predicate Builder. The article does not explain very well what is actually happening under-the-hood. But the above article explains it nicely

How can I iterate over a collection and change values with LINQ extension methods?

Lets say I have a collection of Messages which has the properties "UserID" (int) and "Unread" (bool).
How can I use LINQ extension methods to set Unread = false, for any Message in the collection in whose UserID = 5?
So, I know I can do something like:
messages.Any(m => m.UserID == 5);
But, how do I set the Unread property of each of those with an extension method as well?
Note: I know I should not do this in production code. I'm simply trying to learn some more LINQ-fu.

Actually, this is possible using only the built-in LINQ extension methods without ToList.
I believe that this will perform very similarly to a regular for loop. (I haven't checked)
Don't you dare do this in real code.
messages.Where(m => m.UserID == 5)
.Aggregate(0, (m, r) => { m.Unread = false; return r + 1; });
As an added bonus, this will return the number of users that it modified.

messages.Where(m => m.UserID == 5).ToList().ForEach(m => m.Unread = false);
Then submit the changes.

Standard LINQ extension methods doesn't include side effects aimed methods. However you can either implement it yourself or use from Reactive Extensions for .NET (Rx) like this:
messages.Where(m => m.UserID == 5).Run(m => m.Unread = false);

As there is no explicit extension method that does a ForEach, you are stuck with either using a secondary library, or writing the foreach statement on your own.
foreach (Message msg in messages.Where(m => m.UserID == 5))
{
msg.Unread = false;
}
If you really want to use a Linq statement to accomplish this, create a copy the collection using the ToList() method, accessing the ForEach() method of the List type:
messages.Where(m => m.UserID == 5).ToList().ForEach(m => m.Unread = false);
or place the side-effect in a Where() statement:
messages.Where(m =>
{
if (m.UserID == 5) { m.Unread = false; return true; }
return false;
});
In either case, I prefer to use the explicit foreach loop as it doesn't make unnecessary copies and is clearer than the Where hack.

With LINQ you can't because LINQ is a query language/extension. There is however a project called MoreLinq, which defines an extension method called ForEach which allows you to pass an action which will be performed on every element.
So, you could do with MoreLinq:
messages.Where(m => m.UserID == 5).ForEach(m => m.Unread = false);
Best Regards,
Oliver Hanappi

This answer is in the spirit of providing a solution. On could create an extension which does both the predicate (Where extension) to weed out the items and the action needed upon those items.
Below is an extension named OperateOn which is quite easy to write:
public static void OperateOn<TSource>(this List<TSource> items,
Func<TSource, bool> predicate,
Action<TSource> operation)
{
if ((items != null) && (items.Any()))
{
items.All (itm =>
{
if (predicate(itm))
operation(itm);
return true;
});
}
}
Here is it in action:
var myList = new List<Item>
{ new Item() { UserId = 5, Name = "Alpha" },
new Item() { UserId = 5, Name = "Beta", UnRead = true },
new Item() { UserId = 6, Name = "Gamma", UnRead = false }
};
myList.OperateOn(itm => itm.UserId == 5, itm => itm.UnRead = true);
Console.WriteLine (string.Join(" ",
myList.Select (itm => string.Format("({0} : {1})",
itm.Name,
itm.UnRead ))));
/* Outputs this to the screen
(Alpha : True) (Beta : True) (Gamma : False)
*/
...
public class Item
{
public bool UnRead { get; set; }
public int UserId { get; set; }
public string Name { get; set; }
}

You should be able to just do it in a Select(), remember the lambda is a shortcut for a function, so you can put as much logic in there as you want, then return the current item being enumerated. And... why exactly wouldn't you do this in production code?
messages = messages
.Select(m =>
{
if (m.UserId == 5)
m.Unread = true;
return m;
});