I have the following custom method created for Finding entities in our EF context. It can accept include expressions, which seem to work quite inconsistently. For example, specifying an include, creating a query and then creating a list on that query will not include (sometimes) the entity in the list.
public static IQueryable<T> Find<T>(Expression<Func<T, bool>> expression, params Expression<Func<T, object>>[] includeExpressions) where T : class
{
var query = Current.Set<T>().Where(expression);
foreach (var include in includeExpressions)
{
query = query.Include(include);
}
return query;
}
Any suggestions on why could this happen?
Related
I have a function shown below. It has Expression<Func<T, bool>> parameter where T is the entity called "Languages".
public async Task<List<T>> MyFilterAsync(Expression<Func<T, bool>> filter)
{
return await context.Set<T>().Where(filter).ToListAsync();
}
I want to call this function from my razor pages so that I can get records from 10 to 15 only (and not every record). So there is "filter" parameter of Expression<Func<T, bool>> type in this method. I want to make use of it.
So from C# code on my razor pages. I can call this like as shown below:
Expression<Func<Languages, bool>> filter = m => m.Name == "ABC";
The above code will give me lanaguages that have name "ABC". Now comes the modification part.
I want only 10 to 15 records so I need to modify it incude Skip(skip).Take(pageSize) for the where clause on Linq expression. The question is - is this can be done, so how?
The context.Set<T>() is a list of Languages so we can do skip and take, right?
I hope I am able to explain the question properly.
It's strange that you cannot modify your MyFilterAsync, are you sure? Because the requirement of ordering, skipping & taking need more arguments than just the filter. So it's best if you could write more overloads for your MyFilterAsync to accept more arguments and write similar code to what proposed by other users.
However here I'm trying to make it possible to keep your MyFilterAsync unchanged but still you can hook in the logic for ordering, skipping & taking. It's not magic at all but you still need to write other code: your own extension method to replace the default Where. It depends on how the extension method overloads are picked by the compiler. The default has the most generic type of TEntity for entitty type. You just need to make your extension method overload more specific on the type, e.g: the Languages type in your example. It can be your base entity type. When it's less general (more specific), your extension overloads will be used by the compiler instead of the default ones.
Here's how you can do to make it work:
//put this in the same namespace with the default
//extension methods defined in System.Linq.Queryable
public static class YaQueryableExtensions
{
static readonly AsyncLocal<int> _skip = new AsyncLocal<int>();
static readonly AsyncLocal<int> _take = new AsyncLocal<int>();
static class ExpressionBuffers<TEntity>
{
public static readonly AsyncLocal<Expression<Func<TEntity, object>>> OrderBy =
new AsyncLocal<Expression<Func<TEntity, object>>>();
}
//here is your own extension method for Where
//targeting the specific type of Languages
//which can be any base entity type (if you want it to apply on a broader scope)
public static IQueryable<Languages> Where(this IQueryable<Languages> source,
Expression<Func<Languages, bool>> filter)
{
return source.WhereWithExpressionBuffers(filter);
}
//the generic helper method which can be used on a specific closed type
//of T (this method can be made private)
public static IQueryable<T> WhereWithExpressionBuffers<T>(this IQueryable<T> source,
Expression<Func<T, bool>> filter)
{
source = Queryable.Where(source, filter);
//check for order-by (which should be chained first if any)
var orderBy = ExpressionBuffers<T>.OrderBy.Value;
if(orderBy != null)
{
source = source.OrderBy(orderBy);
ExpressionBuffers<T>.OrderBy.Value = null;
}
//check for skip
var skip = _skip.Value;
if (skip > 0)
{
source = source.Skip(_skip.Value);
_skip.Value = 0;
}
//check for take
var take = _take.Value;
if (take > 0)
{
source = source.Take(take);
_take.Value = 0;
}
return source;
}
public static Expression<Func<T, bool>> Skip<T>(this Expression<Func<T, bool>> filter, int skip)
{
_skip.Value = skip;
return filter;
}
public static Expression<Func<T, bool>> Take<T>(this Expression<Func<T, bool>> filter, int take)
{
_take.Value = take;
return filter;
}
public static Expression<Func<TEntity, bool>> OrderBy<TEntity>(this Expression<Func<TEntity, bool>> filter,
Expression<Func<TEntity,object>> orderBy)
{
ExpressionBuffers<TEntity>.OrderBy.Value = orderBy;
return filter;
}
}
Now is how you use it:
var result = await MyFilterAsync(filter.OrderBy(e => e.Name).Skip(skip).Take(pageSize));
The OrderBy, Skip and Take are chained on the filter instead (with our extension methods) so that they can be buffered for later using inside our own Where extension method where we can read the buffered expressions to build up the Where correctly the way we want).
NOTE: you should put your extension class in the same namespace with Queryable which is System.Linq so that your extension methods can become available automatically (and of course will be used instead of the default extension methods).
Yes just do it after sorting the items, and you may need to implement an interface for Name property to have the orderby property work with generics.
public interface IHasName
{
string Name { get; set; }
}
public async Task<List<T>> MyFilterAsync(Expression<Func<T, bool>> filter, int skip, int take)
where T : class, IHasName
{
return await context.Set<T>()
.Where(filter)
.OrderBy(x=> x.Name)
.Skip(skip)
.Take(take)
.ToListAsync();
}
For EF6, I had a method in my generic repository that I exposed to all service layers in order to retrieve entities from the database with any nested properties as needed:
public IQueryable<T> OldMethod(params Expression<Func<T, object>>[] includeProperties)
{
var queryable = set.AsQueryable();
return includeProperties.Aggregate(queryable, (current, includeProperty) => current.Include(includeProperty));
}
This way, I could use the method in the following way:
var data = repo.OldMethod(x => x.Papers, => x.People.Select(y => y.Addresses)).ToList();
In EF6, this would load the Papers navigation property, the People navigation property, and the Addresses navigation property on each person. This, as expected, throws an exception in EFCore. Because of the switch to Include-->ThenInclude method in EFCore, I'm not quite sure how to easily replicate this at my service layer which I'd like to not require any information about EntityFramework.
This has been asked many times since the initial release of EF Core. Earlier prerelease versions of EF Core even were supporting it, but then it has been removed from EF Core code (I guess in order to promote the new Include / ThenInclude pattern).
While Include / ThenInclude pattern looks more clear (besides the current Intellisense issues), it has one major drawback - requires access to EntityFrameworkQueryableExtensions, thus reference to Microsoft.EntityFrameworkCore assembly. While paramsExpression>` pattern has no such requirement.
The good thing is the one can relatively easily add that functionality. The EF6 source code is publicly available on GitHub, and from there we can see that it uses a method called TryParsePath to build dot separated string path which then is passed to the string overload of Include method.
The same can be applied in EF Core. We can probably use the EF6 code, but I'm going to provide my own version. It can be easily be seen that the supported constructs are member accessors or calls to method called Select with 2 arguments, the second being LambdaExpression.
Following is my interpretation of the above, encapsulated in two custom extension methods:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Linq.Expressions;
namespace Microsoft.EntityFrameworkCore
{
public static class IncludeExtensions
{
public static IQueryable<T> Include<T>(this IQueryable<T> source, IEnumerable<string> includePaths) where T : class
=> includePaths.Aggregate(source, (query, path) => query.Include(path));
public static IQueryable<T> Include<T>(this IQueryable<T> source, IEnumerable<Expression<Func<T, object>>> includePaths) where T : class
=> source.Include(includePaths.Select(e => GetIncludePath(e?.Body)));
static string GetIncludePath(Expression source, bool allowParameter = false)
{
if (allowParameter && source is ParameterExpression)
return null; // ok
if (source is MemberExpression member)
return CombinePaths(GetIncludePath(member.Expression, true), member.Member.Name);
if (source is MethodCallExpression call && call.Method.Name == "Select"
&& call.Arguments.Count == 2 && call.Arguments[1] is LambdaExpression selector)
return CombinePaths(GetIncludePath(call.Arguments[0]), GetIncludePath(selector.Body));
throw new Exception("Invalid Include path.");
}
static string CombinePaths(string path1, string path2)
=> path1 != null ? path1 + "." + path2 : path2;
}
}
The first is simply helper for calling multiple string includes (taken from my answer to Entity Framework Core 2.0.1 Eager Loading on all nested related entities). The second is the method in question, which converts the expressions to strings and call the first. The main work is done by GetIncludePath private method which recursively processes the expression based on the aforementioned rules, plus one additional rule - when navigating bottom up, it should end with lambda parameter.
Now the implementation of the method is question is simple as that:
public IQueryable<T> OldMethod(params Expression<Func<T, object>>[] includeProperties)
=> set.Include(includeProperties);
When I first started using EFCore (switching from EF6), I built these extension methods to translate the "old" way of including x => x.People.Select(y => y.Addresses) to strings like "People.Addresses", which are supported by EFCore too;
public static class Extensions
{
private class ReferencedPropertyFinder : ExpressionVisitor
{
private readonly Type _ownerType;
private readonly List<PropertyInfo> _properties = new List<PropertyInfo>();
private Expression _parameterExpression;
private int _currentPosition = 0;
public ReferencedPropertyFinder(Type ownerType)
{
_ownerType = ownerType;
}
public IReadOnlyList<PropertyInfo> Properties
{
get { return _properties; }
}
protected override Expression VisitMember(MemberExpression node)
{
var propertyInfo = node.Member as PropertyInfo;
if (propertyInfo != null) {
var currentParameter = GetParameter(node);
if (_parameterExpression == currentParameter) {
_properties.Insert(_currentPosition, propertyInfo);
} else {
_properties.Add(propertyInfo);
_parameterExpression = currentParameter;
_currentPosition = _properties.Count() - 1;
}
}
return base.VisitMember(node);
}
private ParameterExpression GetParameter(MemberExpression node)
{
if (node.Expression is ParameterExpression) {
return (ParameterExpression)node.Expression;
} else {
return GetParameter((MemberExpression)node.Expression);
}
}
}
private static IReadOnlyList<PropertyInfo> GetReferencedProperties<T, U>(this Expression<Func<T, U>> expression)
{
var v = new ReferencedPropertyFinder(typeof(T));
v.Visit(expression);
return v.Properties;
}
public static string ToPropertyPath<T>(this Expression<Func<T, object>> expression)
{
var properties = expression.GetReferencedProperties();
var path = string.Join(".", properties.Select(x => x.Name));
return path;
}
}
Incorporating these in your code, you could say something like:
public IQueryable<T> OldMethod(params Expression<Func<T, object>>[] includeProperties)
{
var queryable = set.AsQueryable();
return includeProperties.Aggregate(queryable, (current, includeProperty) =>
current.Include(includeProperty.ToPropertyPath()));
}
I am using a generic repository, like this:
itemsList = (from myrow in UoW.FileRepository.Get()
select new FileModel()
{record_id = myrow.type_id,
descr = myrow.descr}).ToList();});
And this is the Get method:
public virtual IEnumerable<TEntity> Get()
{
// _aQuery = _theDbContext.Set<TEntity>();
IEnumerable<TEntity> query = _aQuery;
return query;
}
How would I implement a generic linq lambda expression if I wanted to create a similar query to search for a particular string in a particular field? In my viewmodel I would like to call something like:
from myrow in UoW.FileRepository.Srch(nameofFieldToSearch, searchString).
The query would look something like this?
public IEnumerable<TEntity> Srch(Expression<Func<TEntity, bool>> expression)
{
IEnumerable<TEntity> srchList = _aQuery.ToList();
return srchList.Where(????);
}
Thank you for your suggestions.
EDIT-----------------------
I have all my queries like Get and Srch in a general repository class and for now just need to know how to declare the query in the repository class and how to call it with the search string from my viewmodel. I am not sure if there is a consensus as to where/when to materialize and compile? I saw another discussion http://www.fascinatedwithsoftware.com/blog/post/2012/01/10/More-on-Expression-vs-Func-with-Entity-Framework.aspx and I quote from it below to inquire whether that is the same approach being suggested here? Thank you again.
"The profiler told us that LoadMyEntities was being called many, many times and it was taking a large fraction of our CPU time. The simple change below solved the problem. Can you guess why?"
public IEnumerable<MyEntity> LoadMyEntities(Func<MyEntity, bool> predicate)
{return Context.MyEntities.Where(predicate);}
"The parameter is now a Func<> instead of an Expression>. The reason this makes a difference is that a predicate that's in the form of an Expression is passed to SQL server, but a predicate that's passed as a Func is not. Normally, you'd want SQL Server to do as much for you as possible, and an Expression would be the right choice, but in this case we'd like to pre-load the entire table in the context -- which is exactly what a Func will do.
The Where extension method has two flavors. One extends IQueryable and takes an Expression parameter. The other extends IEnumerable and takes a Func.
Because 'predicate' is now a Func, the Where that extends IEnumerable is used.
The Entity Framework's fluent interface for constructing SQL queries is based on IQueryables, not IEnumerables. Therefore, the fluency stops just before the Where. The part of the statement that gets passed to the Entity Framework is just Context.MyEntities.
Context.MyEntities therefore returns the entire table to the context.
The entire table is now filtered with the predicate, and the value we really want is returned.
The next time the method is called, the Entity Framework realizes that the record we want is already in the context. (In my case, we were querying by the primary key, and EF is apparently smart enough to know that if there's a record in the context with that ID, it won't find an additional such record in the database.) Since we don't go out to SQL Server, we save lots of time. Obviously there are occasions when you would not want this, but in our case it was exactly what we wanted. The table was relatively small, and the same context was queried hundreds of times.
In the original version, the predicate was an Expression, so the compiler used the Where that extends IQueryable. The predicate was thus passed to SQL Server, which dutifully returned just one row to the context. The next time we called LoadMyEntities, Entity Framework had to call SQL Server again."
Please take a look at msdn repository pattern
public virtual IEnumerable<TEntity> Get(
Expression<Func<TEntity, bool>> filter = null,
Func<IQueryable<TEntity>, IOrderedQueryable<TEntity>> orderBy = null,
string includeProperties = "")
{
IQueryable<TEntity> query = dbSet;
if (filter != null)
{
query = query.Where(filter);
}
foreach (var includeProperty in includeProperties.Split
(new char[] { ',' }, StringSplitOptions.RemoveEmptyEntries))
{
query = query.Include(includeProperty);
}
if (orderBy != null)
{
return orderBy(query).ToList();
}
else
{
return query.ToList();
}
}
This will provide more complete Generic Get method.
For search you could Try
public IEnumerable<TEntity> GlobalSearch(Expression<Func<TEntity, bool>> expression)
{
return Get(expression);
}
In DataProvider you could try something like this
Note This is not may be exact implementation, but will give you basic idea, how to invoke.
public List<Users> Search(List<string> userList)
{
Expression<Func<User, bool>> expression = x=>UserList.Contains(x.UserName);
return GlobalSearch(expression);
}
Example of Repository pattern
This would be a simple implementation using the Expression trees. Following is the complete solution:
Method to fetch the Expression for Srch method:
Public Expression<Func<TEntity, bool>> SrchExpression(string nameofFieldToSearch, string searchString)
{
var parameterType = Expression.Parameter(typeof(TEntity), "obj");
var memberExpression = Expression.Property(typeof(string), nameofFieldToSearch)
// Calls Extension method created underneath
var filtersMethodInfo = typeof(StringExtensions).GetMethod("Contains", new[] { typeof(string), typeof(string) });
var filtersConstantExpression = Expression.Constant(searchString, typeof(string));
var finalExpression = Expression.Call(null, filtersMethodInfo, memberExpression, filtersConstantExpression)
return Expression.Lambda<Func<TEntity, bool>>(finalExpression, parameterType)
}
// Create a String extension method for Contains
public static class StringExtensions
{
public static bool Contains(this string source, string searchString)
{
return source?.IndexOf(subString, StringComparison.OrdinalIgnoreCase) >= 0;
}
}
Now your Srch method shall look like:
public IEnumerable<TEntity> Srch(Expression<Func<TEntity, bool>> expression)
{
Func<TEntity, bool>> func = expression.Compile();
IEnumerable<TEntity> srchList = _aQuery.Where(o => func(o));
return srchList;
}
I have a generic repository in which I'm trying to include a function that accepts a variable list of child tables to eagerly load. The function looks thus:
public IQueryable<T> FindBy(Expression<Func<T, bool>> predicate, params Expression<Func<T, object>>[] includeEntities)
{
IQueryable<T> query = this._dbSet.Where(e => !e.Deleted).Where(predicate);
foreach (var entity in includeEntities)
{
query.Include(entity);
}
return query;
}
It works but I'm concerned about the object reference.
Using the function thus:
var foundEntities = Repository.Entities.FindBy(i => i.Id == targetId, i => i.Orders, i => i.Invoices);
The params passed in the includeEntites array are of type System.Linq.Expressions.PropertyExpression which is unfortunately an internal class so I can't make the function signature:
public IQueryable<T> FindBy(Expression<Func<T, bool>> predicate, params Expression<Func<T, System.Linq.Expressions.PropertyExpression>>[] includeEntities)
as I'd like. Any thoughts?
I'm trying to build a generic method that EF4.1 to look in both the Database and the Local memory for a particular row in a table that matches a particular criteria.
So far, this is what I have this.
This is the caller.
dbEntities.MyTables.LocalAndDb(delegate(MyTable s)
{ return s.Description.Contains("test"); });
This is LocalAndDb
public static object LocalAndDb<T>(this DbSet<T> myTable, Func<T, bool> function) where T : class
{
// look in local
var item = myTable.Local.Where(o => function((T)o)).FirstOrDefault()
// if not exist, look in the database
if (item == null)
{
Expression<Func<T, bool>> predicate = (u) => function(u);
item = myTable.Where(predicate).FirstOrDefault();
}
return item;
}
The problem is with this line.
item = myTable.Where(predicate).FirstOrDefault();
When it calls the database, it throws this error.
"The LINQ expression node type 'Invoke' is not supported in LINQ to Entities."
I imagine it's because I'm passing in an anonymous method and it doesn't know how to turn this into SQL. I thought converting it to an Expression object would do the trick but it's still not working for me.
What do I need to do to make a anonymous method become something that LINQ can turn into SQL?
To make this work, you need to pass the lambda expression to LocalAndDb as an expression tree (so that LINQ to Entities can analyze the code and translate it to SQL):
public static object LocalAndDb<T>(this DbSet<T> myTable,
Expression<Func<T, bool>> expr) where T : class {
// ...
if (item == null) {
item = myTable.Where(expr).FirstOrDefault();
}
return item;
}
Then, of course, the problem is that you cannot execute the expression tree when checking the in-memory data. One way to solve this is to use the Compile method of Expression<T>, but that will be a bit inefficient (depending on your scenario).
Another option is to just pass the condition as both function and expression tree:
public static object LocalAndDb<T>(this DbSet<T> myTable,
Func<T, boo> function, Expression<Func<T, bool>> expr) where T : class {
var item = myTable.Local.Where(o => function((T)o)).FirstOrDefault();
if (item == null) {
item = myTable.Where(expr).FirstOrDefault();
}
return item;
}
table.LocalAndDb(t => t.Foo > 10, t => t.Foo > 10);
This is a bit ugly, but it doesn't require inefficient compilation at runtime. If you want a slightly more sophisticated solution, then you can define your own type to keep pre-compiled functions:
class Precompiled<T1, T2> {
public Precompiled(Expression<Func<T1, T2>> expr) {
this.Expression = expr;
this.Function = expr.Compile();
}
public Expression<Func<T1,T2>> Expression { get; private set; }
public Func<T1,T2> Function { get; private set; }
}