I have a list which I must filter on child properties. The filter operator is dynamic and I'm using a predicate builder in order to combine several filters/lambdas.
For simplicity, let's say that I have two classes like this:
public class FirstClass
{
public int Id { get; set; }
public ICollection<SecondClass> MyList { get; set; }
}
public class SecondClass
{
public int ReferenceId { get; set; }
public int Value { get; set; }
}
My filter use a reference id, an operator type and a value, such that a pseudo code would be like this:
"list of FirstClass".Where(param1 =>
param1.MyList.Single(param2 =>
param2.ReferenceId == "reference id").Value "operatorType" "value")
The actual filter will be something like 123 eq 456, where the reference id is 123, operatorType is "eq" and value is 456.
If the operator just was equality, then the following works just fine:
Expression<Func<FirstClass, bool>> lambda =
param1 => param1.MyList.Single(param2 => param2.ReferenceId == id).Value == value;
Also, filtering only on FirstClass with dynamic expressions, works like a charm, e.g. filtering on Id (my ExpressionTypeDictionary is a dictionary for selecting an ExpressionType based on the provided operatorType):
var parameter = Expression.Parameter(typeof(FirstClass), "param1");
Expression body = parameter;
body = Expression.Property(body, "Id");
body = Expression.MakeBinary(ExpressionTypeDictionary[operatorType], body, value);
var lambda = Expression.Lambda<Func<FirstClass, bool>>(body, new[] { parameter });
I'm able to get the following to compile, but executing the filter on real data using EF Core returns an exception for querySource:
var parameter = Expression.Parameter(typeof(FirstClass), "param1");
Expression<Func<FirstClass, int>> left = param1 =>
param1.MyClass.Single(param2 => param2.ReferenceId == id).Value;
var body = Expression.MakeBinary(
ExpressionTypeDictionary[operatorType],
left.Body,
Expression.Constant(value));
var lambda = Expression.Lambda<Func<FirstClass, bool>>(body, new[] { parameter });
...
theList.Where(lambda);
Any suggestions are appreciated :)
I think rather than expression like this
Expression<Func<FirstClass, bool>> predicate =
x => x.MyList.Single(y => y.ReferenceId == id).Value [operator] value;
you'd better build an expression like this:
Expression<Func<FirstClass, bool>> predicate =
x => x.MyList.Any(y => y.ReferenceId == id && y.Value == value);
Here is how you can do that:
var innerParameter = Expression.Parameter(typeof(SecondClass), "y");
var innerPredicate = Expression.Lambda<Func<SecondClass, bool>>(
Expression.AndAlso(
Expression.Equal(Expression.Property(innerParameter, "ReferenceId"), Expression.Constant(id)),
Expression.MakeBinary(ExpressionTypeDictionary[operatorType], Expression.Property(innerParameter, "Value"), Expression.Constant(value))),
innerParameter);
var parameter = Expression.Parameter(typeof(FirstClass), "x");
var predicate = Expression.Lambda<Func<FirstClass, bool>>(
Expression.Call(
typeof(Enumerable), "Any", new Type[] { typeof(SecondClass) },
Expression.Property(parameter, "MyList"), innerPredicate),
parameter);
Related
I would like to generate dynamically a selector expression from some lambdas.
I want to declare a list of lambda expression like this
Expression<Func<MyEntity, object>> select1 = myentity => myentity.Label;
Expression<Func<MyEntity, object>> select2 = myentity => myentity.User.Name;
Expression<Func<MyEntity, object>> select3 = myentity => myentity.Fields.Where(1 == 1).Select(f => f.Code).FirstOrDefault();
And let's say i have a class :
class MyClass
{
public string Label { get; set; }
public string UserName { get; set; }
public string CodeField { get; set; }
}
I want to compose dynamically the selector expression using the declared expressions.
The goal is that I want to choose the data to recover, not all together.
Expression.Lambda<Func<MyEntity, MyClass>> selectExpression = ??
req.Select(selectExpression).ToList();
I want to generate a selector expression to have something like this
return req.Select(myentity => new MyClass {
Label = myentity.Label,
UserName = myentity.User.Name,
CodeField = myentity.Fields.Where(1 == 1).Select(f => f.Code).FirstOrDefault()
}).ToList();
Can i do this?
I succeeded for example like this but it's not the way that i'm look for
var entityT = Expression.Parameter(typeof(MyEntity), "entity");
var propertyA = Expression.Property(entityT, typeof(MyEntity).GetProperty("Label"));
var propertyB = Expression.Property(entityT, typeof(MyEntity).GetProperty("User"));
var propertyC = Expression.Property(propertyB, typeof(UserEntity).GetProperty("Name"));
var binding = Expression.MemberInit(Expression.New(typeof(MyClass)),
new[]
{
Expression.Bind(typeof(MyClass).GetProperty("Label"), propertyA),
Expression.Bind(typeof(MyClass).GetProperty("UserName"), propertyC),
});
var selectExpression = Expression.Lambda<Func<Benef, MyClass>>(binding, entityT);
return req.Select(selectExpression).ToList();
In the same idea, I was tempted to do this, it compiles but does'nt work:
var binding = Expression.MemberInit(Expression.New(typeof(T)),
new[]
{
Expression.Bind(typeof(T).GetProperty("Label"), select1.Body),
Expression.Bind(typeof(T).GetProperty("UserName"), select2.Body),
});
I have this error :
"variable 'myentity' of type 'MyEntity' referenced from scope '', but it is not defined"
Thank you for your answers.
Basically you need to extract expressions from each lambda and connect it with parameter from MyClass.
Something like: Expression.Bind(typeof(MyClass).GetParameter("x"), selectX.Body).
The only difficulty is that all selectX.Body needs to point to the same paramter, so each body expression needs to be adjusted.
Here is sample code:
class Program
{
static void Main(string[] args)
{
var mapped = entities
.Select(MakeExpression<MyEntity, MyClass>(select1, select2, select3))
.ToList();
}
// Create lambda expression
private static Expression<Func<TEntity, TModel>> MakeExpression<TEntity, TModel>(params Expression<Func<TEntity, object>>[] select)
{
var param = Expression.Parameter(typeof(TEntity));
// Map expressions [select1, ..., selectN] with properties
// For keeping things simple I map nth expression with nth property
// eg. select1 with first property from MyClass
var body = Expression.MemberInit(
Expression.New(typeof(TModel)),
typeof(TModel)
.GetProperties()
.Select((p, i) => Expression.Bind(p, MakeParam(param, select[i])))
.ToArray()
);
return Expression.Lambda<Func<TEntity, TModel>>(body, param);
}
// Replace parameter from given expression with param
// All expressions must have same MyEntity parameter
private static Expression MakeParam<TEntity>(ParameterExpression param, Expression<Func<TEntity, object>> select)
{
Expression body = select.Body;
return new ParamVisitor<TEntity>(param).Visit(body);
}
}
class ParamVisitor<TEntity> : ExpressionVisitor
{
private readonly ParameterExpression _param;
public ParamVisitor(ParameterExpression param)
{
this._param = param;
}
protected override Expression VisitParameter(ParameterExpression node)
{
if (node.Type == typeof(TEntity))
{
return this._param;
}
return base.VisitParameter(node);
}
}
I am trying to generate dynamic lambda expression for filtering ICollection field. In Linq it would look like this:
.Where(x => x.Remarks.Any(s => s.Remark.Description.Contains("filter")))
I have done the last part: s => (s.Remark.Description.Contains("filter"))
MethodInfo containsMethod = typeof(string).GetMethod("Contains", new[] { typeof(string) });
var parameterExpression = Expression.Parameter(Type.GetType("RemarksModel"), "s");
var constant = Expression.Constant("filter");
var property = Expression.Property(parameterExpression, "Remark");
property = Expression.Property(property, "Description");
var expression = Expression.Call(property, containsMethod, constant);
var lambda = Expression.Lambda<Func<remarkModel, bool>>(expression, parameterExpression);
Now I am having truble with adding first part with .Any() to this.
Model with collection looks like this:
public class ReadsModel {
public ICollection< RemarksModel > Remarks { get; set; }
}
public class RemarksModel {
[ForeignKey("RemarkId")]
public virtual RemarkModel Remark { get; set; }
[ForeignKey("ReadsId")]}
public virtual ReadsModel MeterRead { get; set; }
}
public class RemarkModel {
public string Description { get; set; }
}
If we remove the C# compiler extension method sugar and type inference from
x => x.Remarks.Any(lambda)
the actual expression would look like
(ReadsModel x) => Enumerable.Any<RemarksModel>(x.Remarks, lambda)
Hence the code for building it using the Expression class could be like this
var parameter = Expression.Parameter(typeof(ReadsModel), "x");
var body = Expression.Call(
typeof(Enumerable), // class containing the static method
nameof(Enumerable.Any), // method name
new Type[] { typeof(RemarksModel) }, // generic type arguments
Expression.Property(parameter, "Remarks"), lambda // method arguments
);
var predicate = Expression.Lambda<Func<ReadsModel, bool>>(body, parameter);
I've written some codes to make dynamic expressions for filtering my pagination.
I'm trying to make a dynamic expression of EF Core built-in functions for searching (EF.Functions.Like).
I've tried a way like bottom but it is an extension method and first parameters is not used when calling the method. I don't know how to follow the way ==> Ef => Function => Like.
The method should be used like this => Ef.Functions.Like("Property to search", "%Some Pattern")
var likeMethod = typeof(DbFunctionsExtensions)
.GetMethods()
.Where(p => p.Name == "Like")
.First();
string pattern = $"%{finalConstant}%";
ConstantExpression likeConstant = Expression.Constant(pattern,typeof(string));
// the member expression is the property expression for example p.Name
var likeMethodCall = Expression.Call(method: likeMethod, arguments: new[] { memberExpression, likeConstant });
var searchLambda = Expression.Lambda<Func<T, bool>>(likeMethodCall, parameter);
query = query.Where(searchLambda);
but it throw exception saying
Incorrect number of arguments supplied for call to method 'Boolean Like(Microsoft.EntityFrameworkCore.DbFunctions, System.String,
System.String)'\r\nParameter name: method
I implemented a dynamic search based on this article
.NET Core Npgsql.EntityFrameworkCore ILikeExpression
That's what I did:
I implement the [Searchable] attribute, with which I will mark the properties by which the search will be performed. Properties are only of type string, if necessary I can explain how to search for properties of type long and int.
[AttributeUsage(AttributeTargets.Property)]
public class SearchableAttribute : Attribute
{
}
An extension has been created for IQueryable , which takes the input string from the search and implements the Like function according to the specified properties
public static class QueryableExtension
{
public static IQueryable<TEntityDto> ExecuteQueryFilter<TEntityDto>(this IQueryable<TEntityDto> queryable, string query)
where TEntityDto : class, IEntityDto
{
// If the incoming request is empty, skip the search
if (string.IsNullOrEmpty(query))
{
return queryable;
}
// We get all properties with type of string marked with our attribute
var properties = typeof(TEntityDto).GetProperties()
.Where(p => p.PropertyType == typeof(string) &&
p.GetCustomAttributes(typeof(SearchableAttribute), true).FirstOrDefault() != null)
.Select(x => x.Name).ToList();
// If there are no such properties, skip the search
if (!properties.Any())
{
return queryable;
}
// Get our generic object
ParameterExpression entity = Expression.Parameter(typeof(TEntityDto), "entity");
// Get the Like Method from EF.Functions
var efLikeMethod = typeof(DbFunctionsExtensions).GetMethod("Like",
BindingFlags.Static | BindingFlags.Public | BindingFlags.NonPublic,
null,
new[] { typeof(DbFunctions), typeof(string), typeof(string) },
null);
// We make a pattern for the search
var pattern = Expression.Constant($"%{query}%", typeof(string));
// Here we will collect a single search request for all properties
Expression body = Expression.Constant(false);
foreach (var propertyName in properties)
{
// Get property from our object
var property = Expression.Property(entity, propertyName);
// Сall the method with all the required arguments
Expression expr = Expression.Call(efLikeMethod,
Expression.Property(null, typeof(EF), nameof(EF.Functions)), property, pattern);
// Add to the main request
body = Expression.OrElse(body, expr);
}
// Compose and pass the expression to Where
var expression = Expression.Lambda<Func<TEntityDto, bool>>(body, entity);
return queryable.Where(expression);
}
}
The Dto object itself looks like this:
public class CategoryDto : IEntityDto
{
public long Id { get; set; }
[Searchable]
public string Name { get; set; }
[Searchable]
public string IconKey { get; set; }
public long UploadId { get; private set; }
[Searchable]
public string UploadFileName { get; set; }
[Searchable]
public string CreatedBy { get; set; }
public DateTime Created { get; set; }
}
I tested this search method on one million records, with objects name in one to five words. The search process very fast. The performance benefit here is that Expression is converted on the database side as LINQ to SQL
Here's a working example
public static Expression<Func<T, bool>> Like<T>(Expression<Func<T, string>> prop, string keyword)
{
var concatMethod = typeof(string).GetMethod(nameof(string.Concat), new[] { typeof(string), typeof(string) });
return Expression.Lambda<Func<T, bool>>(
Expression.Call(
typeof(DbFunctionsExtensions),
nameof(DbFunctionsExtensions.Like),
null,
Expression.Constant(EF.Functions),
prop.Body,
Expression.Add(
Expression.Add(
Expression.Constant("%"),
Expression.Constant(keyword),
concatMethod),
Expression.Constant("%"),
concatMethod)),
prop.Parameters);
}
query = query.Where(Like<User>(u => u.UserName, "angel"));
As mentioned in the comment, you need to include EF.Functions as the first parameter:
var likeMethodCall = Expression.Call(likeMethod, new []
{
Expression.Property(null, typeof(EF).GetProperty("Functions")),
memberExpression,
likeConstant
});
I've got an function which generates an expression to filter a table by it's primary key, when passed in an Object[], this is very similar to Find function except that it doesn't materialize so you can pass an IQueryable around afterwards
public static Expression<Func<T, bool>> FilterByPrimaryKeyPredicate<T>(this DbContext dbContext, object[] id)
{
var keyProperties = dbContext.GetPrimaryKeyProperties<T>();
var parameter = Expression.Parameter(typeof(T), "e");
var body = keyProperties
// e => e.{propertyName} == new {id = id[i]}.id
.Select((p, i) => Expression.Equal(
Expression.Property(parameter, p.Name),
Expression.Convert(
Expression.PropertyOrField(Expression.Constant(new { id = id[i] }), "id"),
p.ClrType)))
.Aggregate(Expression.AndAlso);
return Expression.Lambda<Func<T, bool>>(body, parameter);
}
This works by first getting the primary keys for a table, it creates binary expression foreach property, the Id is wrapped in an anonymous type to leverage the query cache. This is working fine. However, I'd like to take this a step further.
I'd like to preserve the Expression so I don't have to generate it each time I pass on a new set of ids, How can I store this Expression while still leveraging the Query Cache?
Edit TL;DR
So I'm attempt to cache it using array access in a static class as suggest, however I'm encountering an error:
public class PrimaryKeyFilterContainer<T>
{
const string ANON_ID_PROP = "id";
static Expression<Func<T, bool>> _filter;
Type ANON_TYPE = new { id = (object)0 }.GetType();
public object[] id { get; set; }
public PrimaryKeyFilterContainer()
{
}
public Expression<Func<T, bool>> GetFilter(DbContext dbContext, object[] id)
{
this.id = id;
if(null == _filter)
{
var keyProperties = dbContext.GetPrimaryKeyProperties<T>();
var parameter = Expression.Parameter(typeof(T), "e");
var body = keyProperties
// e => e.PK[i] == id[i]
.Select((p, i) => Expression.Equal(
Expression.Property(parameter, p.Name),
Expression.Convert(BuildNewExpression(i),
p.ClrType)))
.Aggregate(Expression.AndAlso);
_filter = Expression.Lambda<Func<T, bool>>(body, parameter);
}
return _filter;
}
NewExpression BuildNewExpression(int index)
{
var currentObject = Expression.Constant(this);
var fieldAccess = Expression.PropertyOrField(currentObject, nameof(id));
var arrayAccess = Expression.ArrayAccess(fieldAccess, Expression.Constant(index));
return Expression.New(ANON_TYPE.GetConstructor(new[] { typeof(object) }), arrayAccess);
}
}
No coercion operator is defined between types '<>f__AnonymousType0`1[System.Object]' and 'System.Int32'
I'm getting closer but I'm not sure if it's going to work still.
As I mentioned in the comments, the main problem is that we cannot use array index access inside the expression tree - EF6 throws not supported exception and EF Core turns it into client evaluation.
So we need to store the keys in a class with dynamic count of properties and property types. Fortunately the System.Tuple generic classes provide such functionality, and can be used in both EF6 and EF Core.
Following is a class that implements the above idea:
public class PrimaryKeyFilter<TEntity>
where TEntity : class
{
object valueBuffer;
Func<object[], object> valueArrayConverter;
public PrimaryKeyFilter(DbContext dbContext)
{
var keyProperties = dbContext.GetPrimaryKeyProperties<TEntity>();
// Create value buffer type (Tuple) from key properties
var valueBufferType = TupleTypes[keyProperties.Count - 1]
.MakeGenericType(keyProperties.Select(p => p.ClrType).ToArray());
// Build the delegate for converting value array to value buffer
{
// object[] values => new Tuple(values[0], values[1], ...)
var parameter = Expression.Parameter(typeof(object[]), "values");
var body = Expression.New(
valueBufferType.GetConstructors().Single(),
keyProperties.Select((p, i) => Expression.Convert(
Expression.ArrayIndex(parameter, Expression.Constant(i)),
p.ClrType)));
valueArrayConverter = Expression.Lambda<Func<object[], object>>(body, parameter).Compile();
}
// Build the predicate expression
{
var parameter = Expression.Parameter(typeof(TEntity), "e");
var valueBuffer = Expression.Convert(
Expression.Field(Expression.Constant(this), nameof(this.valueBuffer)),
valueBufferType);
var body = keyProperties
// e => e.{propertyName} == valueBuffer.Item{i + 1}
.Select((p, i) => Expression.Equal(
Expression.Property(parameter, p.Name),
Expression.Property(valueBuffer, $"Item{i + 1}")))
.Aggregate(Expression.AndAlso);
Predicate = Expression.Lambda<Func<TEntity, bool>>(body, parameter);
}
}
public Expression<Func<TEntity, bool>> Predicate { get; }
public void SetValues(params object[] values) =>
valueBuffer = valueArrayConverter(values);
static readonly Type[] TupleTypes =
{
typeof(Tuple<>),
typeof(Tuple<,>),
typeof(Tuple<,,>),
typeof(Tuple<,,,>),
typeof(Tuple<,,,,>),
typeof(Tuple<,,,,,>),
typeof(Tuple<,,,,,,>),
typeof(Tuple<,,,,,,,>),
};
}
You can create and store an instance of the class. Then use the expression returned by the Predicate property inside the query. And SetValues method to set the parameters.
The drawback is that the value storage is bound to the class instance, hence it cannot be used concurrently. The original approach works well in all scenarios, and the performance impact IMO should be negligible, so you might consider staying on it.
My issue is I need to query on the value of a property in a generic class. The property is tagged with an attribute.
See the following code:
var rowKeyProperty = EFUtil.GetClassPropertyForRowKey<T>();
var tenantKeyProperty = EFUtil.GetClassPropertyForTenantKey<T>();
var queryResult =
objContext.CreateObjectSet<T>().Single(l => (((int) tenantKeyProperty.GetValue(l, null)) == tenantKey) &&
(((int)rowKeyProperty.GetValue(l, null)) == KeyValue));
The rowKeyProperty and tenantKeyProperty are of type System.Reflection.PropertyInfo.
I understand why I am getting the error. When the linq query is translated to SQL, it can't understand the property.GetValue.
However, I'm completely stumped as to a work around here. Does anyone have any ideas how to achieve this? Thx.
You need to actually build up the Expression objects to represent the expression that you want this to mimic, in this case the expression you want to represent is:
l => l.SomeProperty == SomeValue
So you need to build up each component of that bit by bit, from creating the parameter, defining the equality operator, the property access, the constant value, etc.
public static Expression<Func<TItem, bool>> PropertyEquals<TItem, TValue>(
PropertyInfo property, TValue value)
{
var param = Expression.Parameter(typeof(TItem));
var body = Expression.Equal(Expression.Property(param, property),
Expression.Constant(value));
return Expression.Lambda<Func<TItem, bool>>(body, param);
}
Once you have all of that you can call it using the data that you have:
var queryResult = objContext.CreateObjectSet<T>()
.Where(PropertyEquals<T, int>(tenantKeyProperty, tenantKey))
.Where(PropertyEquals<T, int>(rowKeyProperty, KeyValue))
.Single();
Appendix here... Following #Servy answer and based on this topic with a nice answer by #TomBrothers, you can use the same logic to make a StartsWith (or similar) function:
public static Expression<Func<TItem, bool>> PropertyStartsWith<TItem>(PropertyInfo propertyInfo, string value)
{
var param = Expression.Parameter(typeof(TItem));
var m = Expression.MakeMemberAccess(param, propertyInfo);
var c = Expression.Constant(value, typeof(string));
var mi = typeof(string).GetMethod("StartsWith", new Type[] { typeof(string) });
var body = Expression.Call(m, mi, c);
return Expression.Lambda<Func<TItem, bool>>(body, param);
}
In this case, it forces value to be a string.
It is more correct to specify the type in Expression.Constant(value, typeof(TValue)))
public static Expression<Func<TItem, bool>> PropertyEquals<TItem, TValue>(
string property, TValue value)
{
var xParameter = Expression.Parameter(typeof(TItem));
var body = Expression.Equal(Expression.Property(xParameter, property), Expression.Constant(value, typeof(TValue)));
return Expression.Lambda<Func<TItem, bool>>(body, xParameter);
}
Or, like this, to check the property. ChangeType
public static Expression<Func<TItem, bool>> PropertyEquals<TItem, TValue>(
string property, TValue value)
{
var xParameter = Expression.Parameter(typeof(TItem));
var type = typeof(TItem).GetProperty(property).PropertyType;
value = ChangeType<TValue>(value);
BinaryExpression body = Expression.Equal(Expression.Property(xParameter, property), Expression.Constant(value, type));
return Expression.Lambda<Func<TItem, bool>>(body, xParameter);
}
What is it for. I check all class references to classes, I look for "..ID" entries. Somewhere I have a type "int" and "int?".
public class BudgetLimit : BaseRecord
{
[Required]
public int DepartmentID { get; set; }
public virtual Department Department { get; set;}
public int? ProjectID { get; set; }
public virtual Project Project { get; set; }
}
You add .AsEnableable after the LINQ statement.
e.g objectdata.AsEnumerable()
enter link description here