Develop Reference

Generic Query With PredicateBuilder in Linqkit

I've been using LinqKit to create generic queries for quite some time.
One thing that has always bothered me is the fact that you always have to test whether the value sent in the filter is valid.
For example: Suppose I have a string filter. Conditions can be Equal, StartsWith, EndsWith and Contains.
My method would look something like this:
public List<MyModel> Get(MyModelFilter filter)
{
if (string.IsNullOrEmpty(filter.prop))
{
predicate = predicate.And(_myModel => myModel.Prop.Contains(filter.prop));
}
// Plus a giant amount of if's with multiple filters
return DbSet.AsExpandable()
.Where(predicate)
.ToList();
}
To end this bunch of If's, I decided to create a generic method to apply the filter to the properties.
My idea is to pass the property where the filter will be applied, and the filter definition, and encapsulate the Expression creation logic
It would be something of the type:
public List<MyModel> Get(MyModelFilter filter)
{
predicate = predicate.And(_myModel => myModel.Prop, filter.PropFilterDefinition);
// Goodnye If's, Only others filter impl
return DbSet.AsExpandable()
.Where(predicate)
.ToList();
}
For this, I've created some extension methods to handle this
public static Expression<Func<TPredicate, bool>> And<TPredicate>(
this ExpressionStarter<TPredicate> predicate,
Func<TPredicate, string> property, StringFilterDefinition filter,
bool ignoreNull = true)
{
if (InvalidStringFilter(filter, ignoreNull))
{
return predicate;
}
// This is LinqKit's And Extension Method
return predicate.And(BuildPredicate(property, filter));
}
private static Expression<Func<TPredicate, bool>> BuildPredicate<TPredicate>(
Func<TPredicate, string> property,
StringFilterDefinition filter)
{
if (filter.Filter == StringFilterComparators.Equal)
{
return x => property.Invoke(x) == filter.Value;
}
if (filter.Filter == StringFilterComparators.BeginsWith)
{
return x => property.Invoke(x).StartsWith(filter.Value);
}
if (filter.Filter == StringFilterComparators.EndsWith)
{
return x => property.Invoke(x).EndsWith(filter.Value);
}
return x => property.Invoke(x).Contains(filter.Value);
}
private static bool InvalidStringFilter(
StringFilterDefinition filter,
bool ignoreNullValue = true)
{
if (filter?.Filter == null)
{
return true;
}
return ignoreNullValue && string.IsNullOrEmpty(filter.Value);
}
The problem is that the filter is not applied, and the answer is in Invoke right up there. EF can not translate the above expression to SQL.
The EF error is
Microsoft.EntityFrameworkCore.Query.Internal.SqlServerQueryCompilationContextFactory[8]
The LINQ expression '(__property_0.Invoke([x]) == __filter_Value_1)'
could not be translated and will be evaluated locally. To configure
this warning use the DbContextOptionsBuilder.ConfigureWarnings API
(event id 'RelationalEventId.QueryClientEvaluationWarning').
ConfigureWarnings can be used when overriding the
DbContext.OnConfiguring method or using AddDbContext on the
application service provider.
The question is:
How can I make this construction work?
Also, any suggestions on how best this?

You seem to forgot that besides the PredicateBuilder, the really useful feature provided by LINQKit AsExpandable, Expand and Invoke custom extension methods is to be able to correctly embed expressions inside the expression tree.
In order to utilize that feature, you should use Expression<Func<...>> instead of Func<...>. In the posted code, replace all occurrences of Func<TPredicate, string> with Expression<Func<TPredicate, string>> and the issue should be solved.

Pass linq to entities query as parameter and execute against different objects

I'm looking to implement a cache so that I check to see if specific queries have been executed already, and if they have, return the data using the in memory cache. Rather than implement the cache itself, I can make use of the in memory view that EntityFramework exposes through the Local property of the DBSet.
So, I want to pass the entire query to a method that will run it against the database and/or the in memory cache. However, I'm getting my Expression/Func/IQueryable mixed up and can't see out how to pass the query/expression:
So the base class would be something like this (this obviously isn't working yet...):
public abstract class ServiceBase<TEntity> where TEntity : class
{
Entities _context; //instantiated in the constructor
protected List<TEntity> GetData<TEntity>(Expression<Func<TEntity, TEntity>> query)
{
if (!HasQueryExecuted(query))
{
_context.Set<TEntity>().Select(query).Load();
AddQueryToExecutedList(query);
}
return _context.Set<TEntity>().Local.AsQueryable().Select(query).ToList();
}
}
And I would have multiple classes that define their queries something like this:
public class CustomersService : ServiceBase<Customer>
{
public List<Customer> GetCustomersWithOrders()
{
return GetData(c => c.Where(c => c.OrderCount > 0));
}
public List<Customer> GetLargestCustomersByOrder(int TopCount)
{
return GetData(c => c.OrderBy(c=>c.OrderCount).Take(TopCount));
}
}
public class ProductsService : ServiceBase<Product>
{
public List<Customer> GetAllProducts()
{
return GetData(p => p);
}
public List<Customer> GetMostOrderedProducts(int minimumOrders)
{
return GetData(p => p.Where(p=> p.OrderCount > minimumOrders)
.OrderByDescending(p=>p.OrderCount));
}
public List<Customer> GetAllProducts()
{
return GetData(p => p);
}
}
These are just contrived examples, but the point is the queries could be varied, not just limited to a where clause but making use of all the standard extension methods on IQueryable to query the underlying data.
First of all, is this possible? Can the GetData method define a parameter that can take any query? With the above example, I declared the query parameter to accept the same type as specified for the IQueryable extension Select method, but when I try and call it with a simple Where clause or OrderBy clause, I get various compilation errors such as :
Cannot implicitly convert type 'bool' to 'Entities.Customer
or
Cannot implicitly convert type 'System.Linq.IOrderedEnumerable' to 'Entities.Customer'
And yet, it compiles just fine if I run the same query directly against the context. So what am I doing wrong?

Your issue is this Expression<Func<TEntity, TEntity>> query
Simply put you are saying that your expression is a function type that takes a TEntity and returns a TEntity. Since your class is typed with Entities.Customer it will expect a function that takes an Entities.Customer and returns an Entities.Customer.
If you look at your services classes p => p will work fine but p => p.Where() will not because Where returns an IEnumerable.
I think you should take a different approach and take advantage of lazy-loading.
public IQueryable<TEntity> GetData<TEntity>() where TEntity : class
{
return _context.Set<TEntity>();
}
public IQueryable<Customer> GetAllProducts()
{
return GetData();
}
public IQueryable<Customer> GetMostOrderedProducts(int minimumOrders)
{
return GetData().Where(p => p.OrderCount > minimumOrders)
.OrderByDescending(p=>p.OrderCount));
}
You don't need to build the query into GetData because the query can be built at any point up until it's evaluated. If you really wanted you could return List<T> from here but you shouldn't really have to.
Otherwise this should do the trick for your current code.
protected List<TEntity> GetData<TEntity>(Func<IEnumerable<TEntity>,
IEnumerable<TEntity>> query) where TEntity : class
{
if (!HasQueryExecuted(query))
{
AddQueryToExecutedList(query);
return query(_context.Set<TEntity>()).ToList();
}
return query(_context.Set<TEntity>().Local).ToList();
}

PreprocessQuery event in LightSwitch fails

In LightSwitch I have a PreprocessQuery event as follows:
partial void ValidOrders_PreprocessQuery(ref IQueryable<Order> query)
{
query = query.Where(order => OrderIsValid(order));
}
public bool OrderIsValid(Order order)
{
return true;
}
This fails with a message (on the HTMLClient side !) "method cannot be null".
But this works fine:
partial void ValidOrders_PreprocessQuery(ref IQueryable<Order> query)
{
query = query.Where(order => true);
}
Can someone see why?
Thanks,
Paul

The query provider is only shown the method OrderIsValid, and as that method has already been compiled down to IL it can no longer "look into it" to see it's implementation, as it would need to to create Expression objects to represent it.
There are a number of options that you have, ranging from inlining the method as you did yourself, or having the method itself return an expression, rather than doing the work:
public Expression<Func<Order, bool>> OrderIsValid()
{
return order => true;
}
This would let you write:
partial IQueryable<Order> ValidOrders_PreprocessQuery(IQueryable<Order> query)
{
return query.Where(OrderIsValid());
}
As a side note, I would strongly advise you to not pass the query by reference, but rather return a new query instead; that would be the more idiomatic approach.

How much of a performane hit will i take from casting when trying to make this code mistake proof?

I have the following code:
public class CrudModel<T> : ICrudModel<T> where T : DomainBase
{
public IQueryable<T> GetAll()
{
return Repository.Query<T>();
}
}
the issue is that some of the objects (T) I need to do an extra filter so I created a separate method like this:
public IEnumerable<TR> GetAllwithinOrg<TR>() where TR : DomainBase, IFilterable
{
var items = Repository.Query<TR>();
return FilterwithinOrg(items);
}
where filter method looks like this:
public IEnumerable<TResult> FilterwithinOrg<TResult>(IEnumerable<TResult> linked) where TResult : IFilterable
{
var dict = GetDict();
return linked.Where(r => dict.ContainsKey(r.Id));
}
this all works fine but the issue is that I have to remember to call method 1 or method 2 (based on if the object supports the IFilterable interface
On this question, I got the suggestion to do this:
public IQueryable<T> GetAll()
{
var items = Repository.Query<T>();
if (typeof(IFilterable).IsAssignableFrom(typeof(T)))
{
items = FilterwithinOrg(items.Cast<IFilterable>()).Cast<T>().AsQueryable();
}
return items;
}
so I can support both use cases in one method. This seems to work but I am trying to understand what type of performance hit that I am going to take by doing this
items.Cast<IFilterable>()).Cast<T>().AsQueryable()
If it's bad then I will deal with remembering to call 2 separate methods from the outside but obvious it would be convenient to just have one. Any suggestions?
I think I will leave it in just as a backup if I forget to call the second method but wanted to again see if I can keep it to just one if possible to make it simpler for the outside caller.

How about having another method with where clause in the CrudModel class.
public IEnumerable<T> GetAll<T>(Func<T, bool> whereClause) where T : DomainBase
{
var items = Repository.Query<T>();
return items.Where(whereClause);
}
And call using
List<int> intList = new List<int>() { 1 };
intList.GetAll<int>((i) => sampledict.ContainsKey(i));
I felt it is not proper to make things complex by having logic cramped into one single GetAll method and since CrudModel seems to be generic, better to have generic method that accepts any condition.

First, I think it is a bit strange that you have a function GetAll, but for certain types, you start filtering, resulting in not getting all :)
Besides that, I do not think you have big performance loss... In essense you do one extra check inside you GetAll-method: typeof(IFilterable).IsAssignableFrom(typeof(T)) is like an ordinary cast. You will hardly feel it.
Perhapse the filter itself could be improved. You create a dictionary. Does the dictionary have the same values every call, or does it change. And why a dictionary if you only use the keys, and not the values? What about a HashSet<T>?

Casting time can be neglected comparing to the time spent for database query. However, you are querying the whole table and filter in-memory according to your code here:
public IEnumerable<TResult> FilterwithinOrg<TResult>(IEnumerable<TResult> linked) where TResult : IFilterable
{
var dict = GetDict();
return linked.Where(r => dict.ContainsKey(r.Id));
}
Remember that you need to filter query, not list, so you should change the method to accept and return IQueryable<TResult> instead like:
public IQueryable<TResult> FilterwithinOrg<TResult>(IQueryable<TResult> linked) where TResult : IFilterable
{
var dictKeys = GetDict().Keys.ToList();
return linked.Where(r => dictKeys.Contains(r.Id));
}
Noted that the filter expression has to have equivalent SQL expression or runtime-error will occur.

Passing a where clause in a predicate expression

I have the following predicate expression.
public IQueryable<T> Find(System.Linq.Expressions.Expression<Func<T, bool>> predicate)
{
return //boolean
}
Here is where I am calling the method:
public void TestMethod1()
{
author.AuthorName = authorName;
using (var context = new AspBlogRepository<Author>())
{
if (context.Find(e => e.AuthorName == authorName))
{
//do nothing
}
context.Add(author);
}
}
I get an error saying that I you cannot convert an IQueryable to bool. I just want to be able to use my predicate expression to see if the author is already in the database.
Any help would be much appreciated.
Thanks!

public IQueryable<T> Find(System.Linq.Expressions.Expression<Func<T, bool>> predicate)
{
return //boolean
}
The problem is here, if you indeed return boolean. If this method is inside your context, you would first need to resolve Author and then filter the Authors in your repository.
If this is an EF context, and you have access to the EF context in your repository, you should be able to get the db set like this:
var entitySet = context.Set<T>();
Then, you can run your predicate against the entitySet returning the filtered results.

Error says it all. context.Find(e => e.AuthorName == authorName) is returning IQueryable<T>. If is expecting a bool
So your usage of if (context.Find(e => e.AuthorName == authorName)) is wrong. Change it to
if (context.Find(e => e.AuthorName == authorName).Any())
{
}

context.Find returns an IQueryable<T>. Your code could instead look like:
if (context.Find(e => e.AuthorName == authorName).Count() == 1)
{
//do nothing
}