Develop Reference

Filter List of a model using linq

I have a Model with List inside it, is there any way to filter that list?
What I'm trying is:
List<Booking> a = await _context.Set<Booking>().Include(u => u.BookingLine).Include(u => u.BookingStatus).ToListAsync();
Inside that Booking model, there is a List. I just want to get all the StatusId = 1 inside that BookingStatus.
Booking model and BookingStatus model has the same BookingId. I don't know where do I put the Where() inside that linq.
I tried like this but it returned an error:
_context.Set<Booking>().Include(u => u.BookingLine).Include(u => u.BookingStatus
.LastOrDefault(a => a.StatusId == 1 || a.StatusId == 7 || a.StatusId == 13)).ToListAsync();
Error:
System.InvalidOperationException: The expression 'u.BookingStatus.AsQueryable().LastOrDefault(a => (((a.StatusId == 1) OrElse (a.StatusId == 7)) OrElse (a.StatusId == 13)))' is invalid inside an 'Include' operation, since it does not represent a property access: 't => t.MyProperty'. To target navigations declared on derived types, use casting ('t => ((Derived)t).MyProperty') or the 'as' operator ('t => (t as Derived).MyProperty'). Collection navigation access can be filtered by composing Where, OrderBy(Descending), ThenBy(Descending), Skip or Take operations. For more information on including related data

I think you meant to use "Where" instead of "LastOrDefault".
something like:
_context.Set<Booking>().Include(u => u.BookingLine).Include(u => u.BookingStatus
.Where(a => a.StatusId == 1 || a.StatusId == 7 || a.StatusId == 13)).ToListAsync();
Microsoft example of Eager Loading:
using (var context = new BloggingContext())
{
var filteredBlogs = context.Blogs
.Include(
blog => blog.Posts
.Where(post => post.BlogId == 1)
.OrderByDescending(post => post.Title)
.Take(5))
.ToList();
}

How to efficiently rewrite LINQ expression 'DbSet.Where(...Any(...))' that could not be translated in EFCore 3.1?

I have been solving the issues with the infamous breaking changes of an Entity Framework Core3 upgrade. I used to have a working-well query like this:
// Assume someOtherList is a list populated here
var existingItemsList = myDbSet
.Where(s => someOtherList.Any(i => s.Name == i.Name && s.Id == i.RefId))
.Select(s => s.Id)
.ToList();
But it now throws:
System.InvalidOperationException: The LINQ expression 'DbSet<MyItem>
.Where(i => __someOtherList_0
.Any(i => i.Name == i.Name && i.Id == i.RefId))' could not be translated.
Either rewrite the query in a form that can be translated, or switch to
client evaluation explicitly by inserting a call to either
AsEnumerable(), AsAsyncEnumerable(), ToList(), or ToListAsync().
See https://go.microsoft.com/fwlink/?linkid=2101038 for more information.
How should I re-write my LINQ query ensuring its efficiency (i.e. do not download all the data to client before the filter)?
P.S: It turned out that I overlooked multiple uses of similar lines, the following statement the OP contained is not true: To experiment neglecting the performance concern, I have tried inserting AsEnumerable() before and after the Where() statement; but the exact same error did not cease to exist.

Using LINQKit, you can create extension methods that add expansions for handling Where(List.Any or List.All) conditions for EF Core, assuming your Lists are sufficiently short. These expand the testing out to be a series of tests && or || together.
Here are my WhereAny variations:
// searchTerms - IEnumerable<TSearch> where one must match for a row
// testFne(row,searchTerm) - test one of searchTerms against a row
// dbq.Where(r => searchTerms.Any(s => testFne(r,s)))
public static IQueryable<T> WhereAny<T,TSearch>(this IQueryable<T> dbq, IEnumerable<TSearch> searchTerms, Expression<Func<T, TSearch, bool>> testFne) {
var pred = PredicateBuilder.New<T>();
foreach (var s in searchTerms)
pred = pred.Or(r => testFne.Invoke(r, s));
return dbq.Where((Expression<Func<T,bool>>)pred.Expand());
}
// could be return dbq.Where(searchTerms.AnyIs(testFne));
// testFne(row,searchTerm) - test one of searchTerms against a row
// searchTerms - IEnumerable<TKey> where one must match for a row
// dbq.Where(r => searchTerms.Any(s => testFne(r,s)))
public static IQueryable<T> WhereAny<T,TSearch>(this IQueryable<T> dbq, Expression<Func<T, TSearch, bool>> testFne, IEnumerable<TSearch> searchTerms) =>
dbq.WhereAny(searchTerms, testFne);
// testFne(row,searchTerm) - test one of searchTerms against a row
// searchTerms - TSearch[] where one must match for a row
// dbq.Where(r => searchTerms.Any(s => testFne(r,s)))
public static IQueryable<T> WhereAny<T,TSearch>(this IQueryable<T> dbq, Expression<Func<T, TSearch, bool>> testFne, params TSearch[] searchTerms) {
var pred = PredicateBuilder.New<T>();
foreach (var s in searchTerms)
pred = pred.Or(r => testFne.Invoke(r, s));
return dbq.Where((Expression<Func<T,bool>>)pred.Expand());
}
(There are also similar WhereAll variations, and OrderBy variations, as well as a set of IEnumerable.)
With these, you can write
var ans = myDbSet.WhereAny(someOtherList, (d,s) => d.Name == s.Name && d.Id == s.RefId)
.Select(s => s.Id)
.ToList();

Entity Framework extension method ICollection / IQueryable

I need to check the same specific condition (where clause) many times:
return _ctx.Projects.Where(p => p.CompanyId == companyId &&
(p.Type == Enums.ProjectType.Open ||
p.Invites.Any(i => i.InviteeId == userId))).ToList()
The part after the '&&' will cause that the user isn't able to retrieve restricted projects.
I wanted to abstract this check to a function. In the future these conditions could change and I don't want to replace all the LINQ queries.
I did this with the following extension method:
public static IQueryable<Project> IsVisibleForResearcher(this IQueryable<Project> projects, string userId)
{
return projects.Where(p => p.Type == Enums.ProjectType.Open ||
p.Invites.Any(i => i.InviteeId == userId));
}
Now I can change the LINQ query to:
return _ctx.Projects.Where(p => p.CompanyId == companyId)
.IsVisibleForResearcher(userId).ToList()
This generates the same SQL query. Now my problem starts when I want to use this extension method on another DbSet that has projects.
Imagine that a company has projects. And I only want to retrieve the companies where the user can at least see one project.
return _ctx.Companies
.Where(c => c.Projects.Where(p =>
p.Type == Enums.ProjectType.Open ||
p.Invites.Any(i => i.InviteeId == userId))
.Any())
Here I also like to use the extension method.
return _ctx.Companies
.Where(c => c.Projects.AsQueryable().IsVisibleForCompanyAccount(userId).Any())
This throws following exception:
An exception of type 'System.NotSupportedException' occurred in
Remotion.Linq.dll but was not handled in user code
Additional information: Could not parse expression 'c.Projects.AsQueryable()': This overload of the method 'System.Linq.Queryable.AsQueryable' is currently not supported.
Than I created the following extension methods:
public static IEnumerable<Project> IsVisibleForResearcher(this ICollection<Project> projects, string userId)
{
return projects.Where(p => p.Type == Enums.ProjectType.Open ||
p.Invites.Any(i => i.InviteeId == userId));
}
But this didn't work also.
Does anyone has an idea?
Or a step in the right direction.
Btw I'm using Entity Framework Core on .NET Core
UPDATE:
Using a Expression<Func<>> resulted in the same exception:
'System.Linq.Queryable.AsQueryable' is currently not supported.
UPDATE 2
Thx #ivan-stoev for providing a solution.
I still have one problem. I also want to retrieve the count of 'visible' projects.
I fixed it by doing this:
var companies = _ctx.Companies
.WhereAny(c => c.Projects, Project.IsProjectVisibleForResearcher(userId))
.Select(c => new CompanyListDto
{
Id = c.Id,
Name = c.Name,
LogoId = c.LogoId,
ProjectCount = _ctx.Projects.Where(p => p.CompanyId == c.Id)
.Count(Project.IsProjectVisibleForResearcher(userId))
});
But I don't find a way to just use c.Projects instead of ctx.Projects.Where(p => p.CompanyId == c.Id)
The SQL that gets generated is correct, but I'd like to avoid this unneeded check.
Sincerely,
Brecht

Using expressions / custom methods inside the IQueryable<T> query expression has been always problematic and requires some expression tree post processing. For instance, LinqKit provides AsExpandable, Invoke and Expand custom extension methods for that purpose.
While not so general, here is a solution for your sample use cases w/o using 3rd party packages.
First, extract the expression part of the predicate in a method. The logical place IMO is the Project class:
public class Project
{
// ...
public static Expression<Func<Project, bool>> IsVisibleForResearcher(string userId)
{
return p => p.Type == Enums.ProjectType.Open ||
p.Invites.Any(i => i.InviteeId == userId);
}
}
Then, create a custom extension method like this:
public static class QueryableExtensions
{
public static IQueryable<T> WhereAny<T, E>(this IQueryable<T> source, Expression<Func<T, IEnumerable<E>>> elements, Expression<Func<E, bool>> predicate)
{
var body = Expression.Call(
typeof(Enumerable), "Any", new Type[] { typeof(E) },
elements.Body, predicate);
return source.Where(Expression.Lambda<Func<T, bool>>(body, elements.Parameters));
}
}
With this design, there is no need of your current extension method, because for Projects query you can use:
var projects = _ctx.Projects
.Where(p => p.CompanyId == companyId)
.Where(Project.IsVisibleForResearcher(userId));
and for Companies:
var companies = _ctx.Companies
.WhereAny(c => c.Projects, Project.IsVisibleForResearcher(userId));
Update: This solution is quite limited, so if you have different use cases (especially inside the Select expression as in your second update), you'd better resort to some 3rd party package. For instance, here is the LinqKit solution:
// LInqKit requires expressions to be put into variables
var projects = Linq.Expr((Company c) => c.Projects);
var projectFilter = Project.IsVisibleForResearcher(userId);
var companies = db.Companies.AsExpandable()
.Where(c => projects.Invoke(c).Any(p => projectFilter.Invoke(p)))
.Select(c => new CompanyListDto
{
Id = c.Id,
Name = c.Name,
LogoId = c.LogoId,
ProjectCount = projects.Invoke(c).Count(p => projectFilter.Invoke(p))
});

Get Method is null off IQueryable (Entity Framework)

I'm trying to pass lambda expressions and a type to my DAL. I have this statement:
(entities).GetType().GetMethod("Where")
"entities" is the Table of entities on the DataContext.
When I run the statement I get a null even though Linq.Table inherits IQueryable.
Anyone have an idea?
Here is the entire method:
public object GetResultSet(Dictionary<Type, Func<object, bool>> values)
{
using (ICSDataContext db = DataContextFactory.CreateDataContext<ICSDataContext>(DataContexts.ICS))
{
foreach (var entry in values)
{
var property = db.GetType().GetProperty(entry.Key.Name + "s");
IQueryable entities = (IQueryable)property.GetValue(db, null);
var whereMethod = (entities).GetType().GetMethod("Where")
.MakeGenericMethod(Type.GetType(entry.Key.AssemblyQualifiedName));
return whereMethod.Invoke(entities, new object[] { entry.Value });
}
}
return null;
}
Thanks

As an alternative you could do something like
db.Set<Type>()
which will return you the DBSet of the appropriate type, with Where accessible without reflection. Also you may want to use Expression> rather than Func, expressions work on queryables where as funcs work on enumerables. If you pass a func into a Where clause it pulls the entire dbset down and processes it in memory.
Typed expressions are also a little easier to work with (intellesence, type checking).
Expression<Func<User,bool>> filter = c=>c.FirstName == "Bob";
As another alternative you can look into System.Linq.Dynamic, ScottGu has a write up on it here. The article and the code are old, but it works with EF 6. It allows things like
.Where("CategoryId=2 and UnitPrice>3")

From answer by LukeH under here:
var where1 = typeof(Queryable).GetMethods()
.Where(x => x.Name == "Where")
.Select(x => new { M = x, P = x.GetParameters() })
.Where(x => x.P.Length == 2
&& x.P[0].ParameterType.IsGenericType
&& x.P[0].ParameterType.GetGenericTypeDefinition() == typeof(IQueryable<>)
&& x.P[1].ParameterType.IsGenericType
&& x.P[1].ParameterType.GetGenericTypeDefinition() == typeof(Expression<>))
.Select(x => new { x.M, A = x.P[1].ParameterType.GetGenericArguments() })
.Where(x => x.A[0].IsGenericType
&& x.A[0].GetGenericTypeDefinition() == typeof(Func<,>))
.Select(x => new { x.M, A = x.A[0].GetGenericArguments() })
.Where(x => x.A[0].IsGenericParameter
&& x.A[1] == typeof(bool))
.Select(x => x.M)
.SingleOrDefault();
Then this:
var gmi = where1.MakeGenericMethod(typeof(T));

C# predicate list passed to Linq Where clause

I have a long Linq Where clause that I would like to populate with a predicate list.
List<Expression<Func<Note, bool>>> filters = new List<Expression<Func<Note, bool>>>();
filters.Add(p => p.Title != null && p.Title.ToLower().Contains(searchString));
filters.Add(p => p.Notes != null && p.Notes.ToLower().Contains(searchString));
filters.Add(GlobalSearchUser((List < User > users = new List<User>() { p.user1, p.user2, p.user3, p.user4 }), searchString));
notes = dataAccess.GetList<Note>(pn => pn.ProjectVersionID == projectVersionID, filterExtensions.ToArray())
.Where(filters.ToArray()).Take(10).ToList();
However I'm getting this error:
cannot convert from 'System.Linq.Expressions.Expression<System.Func<project.Contracts.DTOs.Note,bool>>[]' to 'System.Func<project.Contracts.DTOs.Note,bool>'
Which is an error on the .where clause. Pulling out the .where compiles just fine.

I think great answer from Hogan can be simplified and shorten a bit by use of Any and All Linq methods.
To get items that fulfill all the conditions:
var resultAll = listOfItems.Where(p => filters.All(f => f(p)));
And to get the items that fulfill any condition:
var resultAny = listOfItems.Where(p => filters.Any(f => f(p)));

There are at least two errors in your code:
List<Expression<Func<Note, bool>>> filters = new List<Expression<Func<Note, bool>>>();
change it to
List<Func<Note, bool>> filters = new List<Func<Note, bool>>();
You don't need Expression trees here. You are using IEnumerable<>, not IQueryable<>
notes = dataAccess.GetList<Note>(pn => pn.ProjectVersionID == projectVersionID, filterExtensions.ToArray())
.Where(filters.ToArray()).Take(10).ToList();
There .Where() accepts a single predicate at a time. You could:
notes = dataAccess.GetList<Note>(pn => pn.ProjectVersionID == projectVersionID, filterExtensions.ToArray())
.Where(x => filters.All(x)).Take(10).ToList();
or various other solutions, like:
var notesEnu = dataAccess.GetList<Note>(pn => pn.ProjectVersionID == projectVersionID, filterExtensions.ToArray())
.AsEnumerable();
foreach (var filter in filters)
{
notesEmu = notesEmu.Where(filter);
}
notes = notesEnu.Take(10).ToList();
Because all the .Where() conditions are implicitly in &&.

You have to loop over your filters and run a test on each one.
You can do it with linq like this to return true if any of your filters are true:
.Where(p => { foreach(f in filters) if (f(p) == true) return(true); return(false)})
or like this to to return true if all of your filters are true:
.Where(p => { foreach(f in filters) if (f(p) == false) return(false); return(true)})

You can't just pass an array of predicates to the where method. You need to either iterate over the array and keep calling Where() for each expression in the array, or find a way to merge them all together into one expression and use that. You'll want to use LinqKit if you go the second route.