I am trying to generate dynamic lambda expression for filtering ICollection field. In Linq it would look like this:
.Where(x => x.Remarks.Any(s => s.Remark.Description.Contains("filter")))
I have done the last part: s => (s.Remark.Description.Contains("filter"))
MethodInfo containsMethod = typeof(string).GetMethod("Contains", new[] { typeof(string) });
var parameterExpression = Expression.Parameter(Type.GetType("RemarksModel"), "s");
var constant = Expression.Constant("filter");
var property = Expression.Property(parameterExpression, "Remark");
property = Expression.Property(property, "Description");
var expression = Expression.Call(property, containsMethod, constant);
var lambda = Expression.Lambda<Func<remarkModel, bool>>(expression, parameterExpression);
Now I am having truble with adding first part with .Any() to this.
Model with collection looks like this:
public class ReadsModel {
public ICollection< RemarksModel > Remarks { get; set; }
}
public class RemarksModel {
[ForeignKey("RemarkId")]
public virtual RemarkModel Remark { get; set; }
[ForeignKey("ReadsId")]}
public virtual ReadsModel MeterRead { get; set; }
}
public class RemarkModel {
public string Description { get; set; }
}
If we remove the C# compiler extension method sugar and type inference from
x => x.Remarks.Any(lambda)
the actual expression would look like
(ReadsModel x) => Enumerable.Any<RemarksModel>(x.Remarks, lambda)
Hence the code for building it using the Expression class could be like this
var parameter = Expression.Parameter(typeof(ReadsModel), "x");
var body = Expression.Call(
typeof(Enumerable), // class containing the static method
nameof(Enumerable.Any), // method name
new Type[] { typeof(RemarksModel) }, // generic type arguments
Expression.Property(parameter, "Remarks"), lambda // method arguments
);
var predicate = Expression.Lambda<Func<ReadsModel, bool>>(body, parameter);
Related
I am trying to build a lambda expression that will do an "ILIKE" search on my models, based on various client-side parameters such as field name. I have something that works pretty well for properties that are not nested. The problem arises when I want to search on a nested ICollection property. The minimum model is at the bottom of the question.
What works
Let's say the client sends that he wants to search for
f = {
"filterdatafield": "name",
"filtervalue": "test"
}
Then this code will build the required expression:
string MyType="Field";
ParameterExpression p=null;
#nullable enable
Type? x = Type.GetType(MyType);
if (x is null)
{
throw new Exception("Cannot find type " + MyType);
}
#nullable disable
p = Expression.Parameter(x);
Expression property = Expression.Property(p, f.filterdatafield);
var functions = Expression.Property(null, typeof(EF).GetProperty(nameof(EF.Functions)));
var likeFunction = typeof(NpgsqlDbFunctionsExtensions).GetMethod(nameof(NpgsqlDbFunctionsExtensions.ILike), new Type[] { functions.Type, typeof(string), typeof(string) });
var pattern = Expression.Constant($"%{f.filtervalue}%", typeof(string));
MethodCallExpression call = Expression.Call(likeFunction,
Expression.Property(null, typeof(EF), nameof(EF.Functions)), property, pattern);
Expression exp = Expression.Lambda(call, p);
return exp;
What the problem is
OK. Now, let's say that instead he wanted to search for
f = {
"filterdatafield": "fieldoperators",
"filtervalue": "test"
}
The assumption is that he meant to search in the name field of the operators. That's a nested property. How to get the ILIKE lambda for that?
What I've tried
string MyType="Field";
ParameterExpression p=null;
#nullable enable
Type? x = Type.GetType(MyType);
if (x is null)
{
throw new Exception("Cannot find type " + MyType);
}
#nullable disable
p = Expression.Parameter(x);
Expression property = Expression.Property(p, f.filterdatafield);
var functions = Expression.Property(null, typeof(EF).GetProperty(nameof(EF.Functions)));
var likeFunction = typeof(NpgsqlDbFunctionsExtensions).GetMethod(nameof(NpgsqlDbFunctionsExtensions.ILike), new Type[] { functions.Type, typeof(string), typeof(string) });
var pattern = Expression.Constant($"%{f.filtervalue}%", typeof(string));
if (property.Type == typeof(ICollection<FieldOperator>)) {
var fieldParam = Expression.Parameter(typeof(Field),"f");
var operatorsParam = Expression.Parameter(typeof(FieldOperator), "myops");
var lvl1 = Expression.Property(operatorsParam, "Operator");
var lvl2 = Expression.Property(lvl1, "Name");
var compareExpression = Expression.Call(likeFunction,
Expression.Property(null, typeof(EF), nameof(EF.Functions)), lvl2, pattern);
var lambdaForTheAnyCallPredicate = Expression.Lambda<Func<FieldOperator,Boolean>>(compareExpression, operatorsParam);
var collectionProperty = Expression.Property(fieldParam, "FieldOperators");
var resultExpression = ExpressionExtensions.CallAny(collectionProperty, lambdaForTheAnyCallPredicate);
Expression exp = Expression.Lambda<Func<Field, Boolean>>(resultExpression, p);
return exp;
}
The ExpressionExtensions.CallAny method is from this answer
This does generate a seemingly valid expression, however it fails when trying to be translated to SQL by the Entity Framework:
The LINQ expression 'DbSet<Field>()
.Where(f => (IEnumerable<FieldOperator>)f.FieldOperators
.Any(myops => __Functions_0
.ILike(
matchExpression: myops.Operator.Name,
pattern: "%test%")))' could not be translated. Either rewrite the query in a form that can be translated, or switch to client evaluation explicitly by inserting a call to 'AsEnumerable', 'AsAsyncEnumerable', 'ToList', or 'ToListAsync'. See https://go.microsoft.com/fwlink/?linkid=2101038 for more information.
Model
public partial class Field
{
public Field()
{
FieldOperators = new HashSet<FieldOperator>();
}
public long FieldId { get; set; }
public string Name { get; set; }
//various other properties
public virtual ICollection<FieldOperator> FieldOperators { get; set; }
}
public partial class FieldOperator
{
public long FieldId { get; set; }
public long OperatorId { get; set; }
public virtual Field Field { get; set; }
public virtual Oem Operator { get; set; }
}
public partial class Oem
{
public long OemId { get; set; }
public string Name { get; set; }
//other properties omitted
}
Not sure what is
ParameterExpression p = typeof(Field);
in both places. It doesn't compile, so (according to the output) assuming it is
ParameterExpression p = Expression.Parameter(typeof(Field), "f");
And here is the problem. In the code in question you have
// (1)
ParameterExpression p = Expression.Parameter(typeof(Field), "f");
// ...
// (2)
var fieldParam = Expression.Parameter(typeof(Field), "f");
// ...
// (3)
var collectionProperty = Expression.Property(fieldParam, "FieldOperators");
// ...
// (4)
Expression exp = Expression.Lambda<Func<Field, Boolean>>(resultExpression, p);
in (2) you are creating a new parameter with the same name and type as (1), which you are using in (3) as part of the body of the lambda (4). However you are passing (1) as lambda parameter in (4) while the body uses (2).
And that's the problem. Parameter expressions are identified by instance, not by name. So although the expression looks fine, it isn't - if you try to Compile() it to delegate, you'd get a runtime exception. Similar is when EF Core trying to translate it.
It is a common mistake when manipulating expression trees with code. Because C# does not allow having parameters with the same name inside one and the same scope. But Expression API does not care about ParameterExpression names - as you can see, they are optional argument of Expression.Parameter.
With that being said, simply replace (2) with
var fieldParam = p;
or remove it and just use p in place of fieldParam, and the problem will be solved.
Also the collectionProperty variable is redundant since it is the same as property variable. So the final code should be something like this:
var parameter = Expression.Parameter(typeof(Field), "f");
var property = Expression.Property(parameter, f.filterdatafield);
var functions = Expression.Property(null, typeof(EF).GetProperty(nameof(EF.Functions)));
var likeFunction = typeof(NpgsqlDbFunctionsExtensions).GetMethod(nameof(NpgsqlDbFunctionsExtensions.ILike), new Type[] { functions.Type, typeof(string), typeof(string) });
var pattern = Expression.Constant($"%{f.filtervalue}%", typeof(string));
if (property.Type == typeof(ICollection<FieldOperator>))
{
var operatorsParam = Expression.Parameter(typeof(FieldOperator), "myops");
var lvl1 = Expression.Property(operatorsParam, "Operator");
var lvl2 = Expression.Property(lvl1, "Name");
var compareExpression = Expression.Call(likeFunction,
Expression.Property(null, typeof(EF), nameof(EF.Functions)), lvl2, pattern);
var anyPredicate = Expression.Lambda<Func<FieldOperator, bool>>(compareExpression, operatorsParam);
var body = Expression.Call(typeof(Enumerable),
nameof(Enumerable.Any), new[] { operatorsParam.Type },
property, anyPredicate);
var predicate = Expression.Lambda<Func<Field, bool>>(body, parameter);
return predicate;
}
I would like to generate dynamically a selector expression from some lambdas.
I want to declare a list of lambda expression like this
Expression<Func<MyEntity, object>> select1 = myentity => myentity.Label;
Expression<Func<MyEntity, object>> select2 = myentity => myentity.User.Name;
Expression<Func<MyEntity, object>> select3 = myentity => myentity.Fields.Where(1 == 1).Select(f => f.Code).FirstOrDefault();
And let's say i have a class :
class MyClass
{
public string Label { get; set; }
public string UserName { get; set; }
public string CodeField { get; set; }
}
I want to compose dynamically the selector expression using the declared expressions.
The goal is that I want to choose the data to recover, not all together.
Expression.Lambda<Func<MyEntity, MyClass>> selectExpression = ??
req.Select(selectExpression).ToList();
I want to generate a selector expression to have something like this
return req.Select(myentity => new MyClass {
Label = myentity.Label,
UserName = myentity.User.Name,
CodeField = myentity.Fields.Where(1 == 1).Select(f => f.Code).FirstOrDefault()
}).ToList();
Can i do this?
I succeeded for example like this but it's not the way that i'm look for
var entityT = Expression.Parameter(typeof(MyEntity), "entity");
var propertyA = Expression.Property(entityT, typeof(MyEntity).GetProperty("Label"));
var propertyB = Expression.Property(entityT, typeof(MyEntity).GetProperty("User"));
var propertyC = Expression.Property(propertyB, typeof(UserEntity).GetProperty("Name"));
var binding = Expression.MemberInit(Expression.New(typeof(MyClass)),
new[]
{
Expression.Bind(typeof(MyClass).GetProperty("Label"), propertyA),
Expression.Bind(typeof(MyClass).GetProperty("UserName"), propertyC),
});
var selectExpression = Expression.Lambda<Func<Benef, MyClass>>(binding, entityT);
return req.Select(selectExpression).ToList();
In the same idea, I was tempted to do this, it compiles but does'nt work:
var binding = Expression.MemberInit(Expression.New(typeof(T)),
new[]
{
Expression.Bind(typeof(T).GetProperty("Label"), select1.Body),
Expression.Bind(typeof(T).GetProperty("UserName"), select2.Body),
});
I have this error :
"variable 'myentity' of type 'MyEntity' referenced from scope '', but it is not defined"
Thank you for your answers.
Basically you need to extract expressions from each lambda and connect it with parameter from MyClass.
Something like: Expression.Bind(typeof(MyClass).GetParameter("x"), selectX.Body).
The only difficulty is that all selectX.Body needs to point to the same paramter, so each body expression needs to be adjusted.
Here is sample code:
class Program
{
static void Main(string[] args)
{
var mapped = entities
.Select(MakeExpression<MyEntity, MyClass>(select1, select2, select3))
.ToList();
}
// Create lambda expression
private static Expression<Func<TEntity, TModel>> MakeExpression<TEntity, TModel>(params Expression<Func<TEntity, object>>[] select)
{
var param = Expression.Parameter(typeof(TEntity));
// Map expressions [select1, ..., selectN] with properties
// For keeping things simple I map nth expression with nth property
// eg. select1 with first property from MyClass
var body = Expression.MemberInit(
Expression.New(typeof(TModel)),
typeof(TModel)
.GetProperties()
.Select((p, i) => Expression.Bind(p, MakeParam(param, select[i])))
.ToArray()
);
return Expression.Lambda<Func<TEntity, TModel>>(body, param);
}
// Replace parameter from given expression with param
// All expressions must have same MyEntity parameter
private static Expression MakeParam<TEntity>(ParameterExpression param, Expression<Func<TEntity, object>> select)
{
Expression body = select.Body;
return new ParamVisitor<TEntity>(param).Visit(body);
}
}
class ParamVisitor<TEntity> : ExpressionVisitor
{
private readonly ParameterExpression _param;
public ParamVisitor(ParameterExpression param)
{
this._param = param;
}
protected override Expression VisitParameter(ParameterExpression node)
{
if (node.Type == typeof(TEntity))
{
return this._param;
}
return base.VisitParameter(node);
}
}
I've written some codes to make dynamic expressions for filtering my pagination.
I'm trying to make a dynamic expression of EF Core built-in functions for searching (EF.Functions.Like).
I've tried a way like bottom but it is an extension method and first parameters is not used when calling the method. I don't know how to follow the way ==> Ef => Function => Like.
The method should be used like this => Ef.Functions.Like("Property to search", "%Some Pattern")
var likeMethod = typeof(DbFunctionsExtensions)
.GetMethods()
.Where(p => p.Name == "Like")
.First();
string pattern = $"%{finalConstant}%";
ConstantExpression likeConstant = Expression.Constant(pattern,typeof(string));
// the member expression is the property expression for example p.Name
var likeMethodCall = Expression.Call(method: likeMethod, arguments: new[] { memberExpression, likeConstant });
var searchLambda = Expression.Lambda<Func<T, bool>>(likeMethodCall, parameter);
query = query.Where(searchLambda);
but it throw exception saying
Incorrect number of arguments supplied for call to method 'Boolean Like(Microsoft.EntityFrameworkCore.DbFunctions, System.String,
System.String)'\r\nParameter name: method
I implemented a dynamic search based on this article
.NET Core Npgsql.EntityFrameworkCore ILikeExpression
That's what I did:
I implement the [Searchable] attribute, with which I will mark the properties by which the search will be performed. Properties are only of type string, if necessary I can explain how to search for properties of type long and int.
[AttributeUsage(AttributeTargets.Property)]
public class SearchableAttribute : Attribute
{
}
An extension has been created for IQueryable , which takes the input string from the search and implements the Like function according to the specified properties
public static class QueryableExtension
{
public static IQueryable<TEntityDto> ExecuteQueryFilter<TEntityDto>(this IQueryable<TEntityDto> queryable, string query)
where TEntityDto : class, IEntityDto
{
// If the incoming request is empty, skip the search
if (string.IsNullOrEmpty(query))
{
return queryable;
}
// We get all properties with type of string marked with our attribute
var properties = typeof(TEntityDto).GetProperties()
.Where(p => p.PropertyType == typeof(string) &&
p.GetCustomAttributes(typeof(SearchableAttribute), true).FirstOrDefault() != null)
.Select(x => x.Name).ToList();
// If there are no such properties, skip the search
if (!properties.Any())
{
return queryable;
}
// Get our generic object
ParameterExpression entity = Expression.Parameter(typeof(TEntityDto), "entity");
// Get the Like Method from EF.Functions
var efLikeMethod = typeof(DbFunctionsExtensions).GetMethod("Like",
BindingFlags.Static | BindingFlags.Public | BindingFlags.NonPublic,
null,
new[] { typeof(DbFunctions), typeof(string), typeof(string) },
null);
// We make a pattern for the search
var pattern = Expression.Constant($"%{query}%", typeof(string));
// Here we will collect a single search request for all properties
Expression body = Expression.Constant(false);
foreach (var propertyName in properties)
{
// Get property from our object
var property = Expression.Property(entity, propertyName);
// Сall the method with all the required arguments
Expression expr = Expression.Call(efLikeMethod,
Expression.Property(null, typeof(EF), nameof(EF.Functions)), property, pattern);
// Add to the main request
body = Expression.OrElse(body, expr);
}
// Compose and pass the expression to Where
var expression = Expression.Lambda<Func<TEntityDto, bool>>(body, entity);
return queryable.Where(expression);
}
}
The Dto object itself looks like this:
public class CategoryDto : IEntityDto
{
public long Id { get; set; }
[Searchable]
public string Name { get; set; }
[Searchable]
public string IconKey { get; set; }
public long UploadId { get; private set; }
[Searchable]
public string UploadFileName { get; set; }
[Searchable]
public string CreatedBy { get; set; }
public DateTime Created { get; set; }
}
I tested this search method on one million records, with objects name in one to five words. The search process very fast. The performance benefit here is that Expression is converted on the database side as LINQ to SQL
Here's a working example
public static Expression<Func<T, bool>> Like<T>(Expression<Func<T, string>> prop, string keyword)
{
var concatMethod = typeof(string).GetMethod(nameof(string.Concat), new[] { typeof(string), typeof(string) });
return Expression.Lambda<Func<T, bool>>(
Expression.Call(
typeof(DbFunctionsExtensions),
nameof(DbFunctionsExtensions.Like),
null,
Expression.Constant(EF.Functions),
prop.Body,
Expression.Add(
Expression.Add(
Expression.Constant("%"),
Expression.Constant(keyword),
concatMethod),
Expression.Constant("%"),
concatMethod)),
prop.Parameters);
}
query = query.Where(Like<User>(u => u.UserName, "angel"));
As mentioned in the comment, you need to include EF.Functions as the first parameter:
var likeMethodCall = Expression.Call(likeMethod, new []
{
Expression.Property(null, typeof(EF).GetProperty("Functions")),
memberExpression,
likeConstant
});
I am trying to write a dynamic select statement. I have the following:
public class MainList
{
public string Prop1{ get; set; }
public string Prop2{ get; set; }
public string Prop3{ get; set; }
}
public class SearchObject
{
public string Prop1{ get; set; }
}
I want build the expression like the following
var newList = MainList.Select(n => new SearchObject { Prop1 = n.Prop1});
The code I am using creates a list based on MainList. I then create the select expression by passing the SearchObject type and the parameters I want to populate, for now. It runs until the second to last line.
public void Start()
{
List<MainList> newList = new List<MainList>(); //This has a ton list objects
var result = newList.Select(CreateSelect<SearchObject>("Prop1"));
}
public static Func<MainList, T> CreateSelect<T>(string fields)
{
var par = Expression.Parameter(typeof(T), "n");
var newInstance= Expression.New(typeof(T));
var bindings = fields.Split(',').Select(o => o.Trim())
.Select(n => {
var p = typeof(T).GetProperty(n);
var original = Expression.Property(par, p);
return Expression.Bind(p, original);
}
);
var newT= Expression.MemberInit(newInstance, bindings);
var lambda = Expression.Lambda<Func<MainList, T>>(newT, par); //ERROR HAPPENS HERE
return lambda.Compile();
}
The error I get is:
Additional information: ParameterExpression of type 'WebApplication.SearchObject' cannot be used for delegate parameter of type 'WebApplication.MainList'
I am unsure on the meaning of the error and also how to resolve the issue.
The first issue is, as already mentioned by Jeroen van Langen, the type of the parameter must be MainList.
The second issue is the usage of the Expression.Bind. Since the source and target are different types, you cannot use one and the same PropertyInfo. The first argument must be a PropertyInfo of the target type T, while the second - expression coming from the source type MainList (in your case, Expression.Property on the parameter with the specified property name).
The correct implementation is something like this:
public static Func<MainList, T> CreateSelect<T>(string fields)
{
var parameter = Expression.Parameter(typeof(MainList), "n");
var bindings = fields.Split(',')
.Select(name => name.Trim())
.Select(name => Expression.Bind(
typeof(T).GetProperty(name),
Expression.Property(parameter, name)
));
var newT = Expression.MemberInit(Expression.New(typeof(T)), bindings);
var lambda = Expression.Lambda<Func<MainList, T>>(newT, parameter);
return lambda.Compile();
}
The exception ParameterExpression of type 'WebApplication.SearchObject' cannot be used for delegate parameter of type 'WebApplication.MainList' explained:
Meaning: There is a mismatch between the ParameterExpression type typeof(T) and the Expression.Lambda Func<MainList, T> --> MainList
Your:
var par = Expression.Parameter(typeof(T), "n");
should be:
var par = Expression.Parameter(typeof(MainList), "n");
I have a list which I must filter on child properties. The filter operator is dynamic and I'm using a predicate builder in order to combine several filters/lambdas.
For simplicity, let's say that I have two classes like this:
public class FirstClass
{
public int Id { get; set; }
public ICollection<SecondClass> MyList { get; set; }
}
public class SecondClass
{
public int ReferenceId { get; set; }
public int Value { get; set; }
}
My filter use a reference id, an operator type and a value, such that a pseudo code would be like this:
"list of FirstClass".Where(param1 =>
param1.MyList.Single(param2 =>
param2.ReferenceId == "reference id").Value "operatorType" "value")
The actual filter will be something like 123 eq 456, where the reference id is 123, operatorType is "eq" and value is 456.
If the operator just was equality, then the following works just fine:
Expression<Func<FirstClass, bool>> lambda =
param1 => param1.MyList.Single(param2 => param2.ReferenceId == id).Value == value;
Also, filtering only on FirstClass with dynamic expressions, works like a charm, e.g. filtering on Id (my ExpressionTypeDictionary is a dictionary for selecting an ExpressionType based on the provided operatorType):
var parameter = Expression.Parameter(typeof(FirstClass), "param1");
Expression body = parameter;
body = Expression.Property(body, "Id");
body = Expression.MakeBinary(ExpressionTypeDictionary[operatorType], body, value);
var lambda = Expression.Lambda<Func<FirstClass, bool>>(body, new[] { parameter });
I'm able to get the following to compile, but executing the filter on real data using EF Core returns an exception for querySource:
var parameter = Expression.Parameter(typeof(FirstClass), "param1");
Expression<Func<FirstClass, int>> left = param1 =>
param1.MyClass.Single(param2 => param2.ReferenceId == id).Value;
var body = Expression.MakeBinary(
ExpressionTypeDictionary[operatorType],
left.Body,
Expression.Constant(value));
var lambda = Expression.Lambda<Func<FirstClass, bool>>(body, new[] { parameter });
...
theList.Where(lambda);
Any suggestions are appreciated :)
I think rather than expression like this
Expression<Func<FirstClass, bool>> predicate =
x => x.MyList.Single(y => y.ReferenceId == id).Value [operator] value;
you'd better build an expression like this:
Expression<Func<FirstClass, bool>> predicate =
x => x.MyList.Any(y => y.ReferenceId == id && y.Value == value);
Here is how you can do that:
var innerParameter = Expression.Parameter(typeof(SecondClass), "y");
var innerPredicate = Expression.Lambda<Func<SecondClass, bool>>(
Expression.AndAlso(
Expression.Equal(Expression.Property(innerParameter, "ReferenceId"), Expression.Constant(id)),
Expression.MakeBinary(ExpressionTypeDictionary[operatorType], Expression.Property(innerParameter, "Value"), Expression.Constant(value))),
innerParameter);
var parameter = Expression.Parameter(typeof(FirstClass), "x");
var predicate = Expression.Lambda<Func<FirstClass, bool>>(
Expression.Call(
typeof(Enumerable), "Any", new Type[] { typeof(SecondClass) },
Expression.Property(parameter, "MyList"), innerPredicate),
parameter);