Develop Reference

How do I write a dynamic predicate builder for a where statement in linq / EF Core?

I'm working with an IQueryable<SomeRandomObject> that is pulled using an EF Core 3.1 data context.
I'm pretty sure I can dynamically build a predicate for .Where() so that I can pass a string in for what column, and what value.
Of course this doesn't work, but some pseudo-code might be:
IQueryable myQueryable = stuffFromContext;
var columnName = "memberid";
var searchValue = "1234";
var results = myQueryable.Where(x=> someMagicColumnFunction(columnName, searchvalue))
I've only done research at this point, and predicate building is not my area of expertise.
Can someone help me create a function that I can pass in the parameters my IQueryable, a string representing the column name, and a string for the search (full equality for now, no 'like').
I'd love to see how this is done. I can't find a solid example anywhere on how to do something small like this. Most of the examples are everything and the kitchen sink!

Assuming the type of the rows in myQueryable are TQueryable then you can create a myQueryable specific function to generate the lambda:
Expression<Func<TQueryable, bool>> EqualsFilter<TCol>(string columnName, TCol searchValue) {
// build x => x.{columnName} == searchValue
// (TQueryable x)
var xParam = Expression.Parameter(typeof(TQueryable), "x");
// x.{columnName}
var colExpr = Expression.Property(xParam, columnName);
// {searchValue}
var constExpr = Expression.Constant(searchValue);
// x.{columnName} == {searchValue}
var lambdaBody = Expression.MakeBinary(ExpressionType.Equal, colExpr, constExpr);
// (TQueryable x) => x.{columnName} == {searchValue}
var lambda = Expression.Lambda<Func<TQueryable, bool>>(lambdaBody, xParam);
return lambda;
}
Once you have the method, you can use it like:
var myQueryable = stuffFromContext;
var columnName = "memberid";
var searchValue = "1234";
var results = myQueryable.Where(EqualsFilter(columnName, searchvalue));
However, if myQueryable has a complex or anonymous type (because of a Select or Join) you need to replace the Where as C# can only infer types from parameters, so you need the myQueryable parameter to get the entity type you are filtering. Using a generic version of EqualsFilter as a helper method, you have:
public static class IQueryableExt {
static Expression<Func<T, bool>> EqualsFilter<T, TCol>(string columnName, TCol searchValue) {
// build x => x.{columnName} == searchValue
// (T x)
var xParam = Expression.Parameter(typeof(Accounts), "x");
// x.{columnName}
var colExpr = Expression.Property(xParam, columnName);
// {searchValue}
var constExpr = Expression.Constant(searchValue);
// x.{columnName} == {searchValue}
var lambdaBody = Expression.MakeBinary(ExpressionType.Equal, colExpr, constExpr);
// (T x) => x.{columnName} == {searchValue}
var lambda = Expression.Lambda<Func<T, bool>>(lambdaBody, xParam);
return lambda;
}
public static IQueryable<T> WhereColumnEquals<T, TCol>(this IQueryable<T> src, string columnName, TCol searchValue)
=> src.Where(EqualsFilter<T, TCol>(columnName, searchValue));
}
Which you can now use like:
var myQueryable = stuffFromContext;
var columnName = "memberid";
var searchValue = "1234";
var results = myQueryable.WhereColumnEquals(columnName, searchvalue);

Issue with C# lambda return System.Linq.Expressions.Expression

I have a big problem with assigning the following code and I don't know how to solve it and I want to use lambda operators.
System.Linq.Expressions.Expression expression = null;
if (typeof(ICustomTypeDescriptor).IsAssignableFrom(sourceType))
{
System.Linq.Expressions.Expression expression2 = System.Linq.Expressions.Expression.Convert(parameterExpression, typeof(ICustomTypeDescriptor));
expression = (ICustomTypeDescriptor t, object o) =>
{
t.GetProperties()[propertyName].GetValue(o)
}
;
return System.Linq.Expressions.Expression.Invoke(expression, new System.Linq.Expressions.Expression[]
{
expression2,
parameterExpression
});
}
The problem is in this part
enter image description here
Can it be declared in any other way ?
ERROR
Unable to convert lambda expression element to "Expression" type because it is not a delegate type

You are using multilines lambda expression, which is ok. But you need to end each line with ";".
I think this would work but without the whole code it will be hard to be sure :
System.Linq.Expressions.Expression expression = null;
if (typeof(ICustomTypeDescriptor).IsAssignableFrom(sourceType))
{
System.Linq.Expressions.Expression expression2 = System.Linq.Expressions.Expression.Convert(parameterExpression, typeof(ICustomTypeDescriptor));
expression = (ICustomTypeDescriptor t, object o) =>
{
t.GetProperties()[propertyName].GetValue(o);
};
return System.Linq.Expressions.Expression.Invoke(expression, new System.Linq.Expressions.Expression[]
{
expression2,
parameterExpression
});
}
You can also go for singleline lambda :
System.Linq.Expressions.Expression expression = null;
if (typeof(ICustomTypeDescriptor).IsAssignableFrom(sourceType))
{
System.Linq.Expressions.Expression expression2 = System.Linq.Expressions.Expression.Convert(parameterExpression, typeof(ICustomTypeDescriptor));
expression = (ICustomTypeDescriptor t, object o) => t.GetProperties()[propertyName].GetValue(o);
return System.Linq.Expressions.Expression.Invoke(expression, new System.Linq.Expressions.Expression[]
{
expression2,
parameterExpression
});
}
Please care, I haven't tested the code to check if I'm not wrong.

Instead of this line,
System.Linq.Expressions.Expression expression = null;
Create a delegate and assign a variable to your expression.
delegate System.Linq.Expressions.Expression Myexpression(ICustomTypeDescriptor t, object o);
Myexpression expression = (ICustomTypeDescriptor t, object o) => t.GetProperties()[propertyName].GetValue(o);

Thanks guys, I'm sorry for not writing for so long, I found a solution and changed the code a bit, I tried with every proposal but unfortunately it threw the same error.
Maybe my solution for someone will be helpful in the future :)
private static LambdaExpression GetLambdaWithPropertyNullCheck(
IEnumerable source,
string propertyName,
ParameterExpression parameterExpression,
Type sourceType)
{
string[] strArray = propertyName.Split('.');
LambdaExpression lambdaExpression;
if (strArray.GetLength(0) > 1)
{
System.Linq.Expressions.Expression expression1 = parameterExpression.GetValueExpression(propertyName, sourceType);
if (expression1.Type != typeof(object))
expression1 = (System.Linq.Expressions.Expression)System.Linq.Expressions.Expression.Convert(expression1, typeof(object));
System.Linq.Expressions.Expression expression2 = (System.Linq.Expressions.Expression)null;
string propertyName1 = string.Empty;
int length = strArray.GetLength(0);
for (int index = 0; index < length; ++index)
{
if (index == 0)
{
expression2 = (System.Linq.Expressions.Expression)System.Linq.Expressions.Expression.Equal(parameterExpression.GetValueExpression(strArray[index], sourceType), (System.Linq.Expressions.Expression)System.Linq.Expressions.Expression.Constant((object)null));
propertyName1 = strArray[index];
}
else if (index < length - 1)
{
propertyName1 = propertyName1 + (object)'.' + strArray[index];
expression2 = (System.Linq.Expressions.Expression)System.Linq.Expressions.Expression.OrElse(expression2, (System.Linq.Expressions.Expression)System.Linq.Expressions.Expression.Equal(parameterExpression.GetValueExpression(propertyName1, sourceType), (System.Linq.Expressions.Expression)System.Linq.Expressions.Expression.Constant((object)null)));
}
}
lambdaExpression = System.Linq.Expressions.Expression.Lambda((System.Linq.Expressions.Expression)System.Linq.Expressions.Expression.Condition(expression2, (System.Linq.Expressions.Expression)System.Linq.Expressions.Expression.Constant((object)null), expression1, typeof(object)), parameterExpression);
}
else
lambdaExpression = System.Linq.Expressions.Expression.Lambda(parameterExpression.GetValueExpression(propertyName, sourceType), parameterExpression);
return lambdaExpression;
}

How do i append a simple expression onto IQueryable

I have the following method:
public List<Customer> SearchTest(string city, int skip, int take)
{
EcomContext db = new EcomContext();
var results = db.Customers.Where(n => n.City == city).OrdeyBy(n => n.Name).Skip(skip).Take(10);
results = AddDeleteCheck<Customer>(results);
return results.ToList()
}
And this reusable method:
private IQueryable<T> AddArchivedCheck<T>(IQueryable<T> data)
{
var parameter = Expression.Parameter(typeof(T));
var e1 = Expression.Equal(Expression.Property(parameter, "Archived"), Expression.Constant(false));
var e2 = data.Expression;
var e3 = Expression.Lambda<Func<T, bool>>(Expression.AndAlso(e1, e2), parameter);
return data.Where(e3);
}
I want to be able to call this method from a number of different functions so i have made it generic. It should take the expression from the IQueryable object and add a check onto this (Archived == false).
I am getting this error:
The binary operator AndAlso is not defined for the types 'System.Boolean' and 'System.Linq.IQueryable` [Ecom.Customer]

It is easier than what you wrote:
private static IQueryable<T> AddArchivedCheck<T>(IQueryable<T> data)
{
var parameter = Expression.Parameter(typeof(T));
var e1 = Expression.Equal(Expression.Property(parameter, "Archived"), Expression.Constant(false));
var lambda = Expression.Lambda<Func<T, bool>>(e1, parameter);
return data.Where(lambda);
}
Remember that in Linq:
var result = query.Where(condition1).Where(condition2);
is equivalent to:
var result = query.Where(condition1 && condition2);

Dynamic extraction optimization of nested properties values

I have small piece of code responsible for dynamic extraction of properties values from objects instances through reflection:
public static object ExtractValue(object source, string property)
{
var props = property.Split('.');
var type = source.GetType();
var arg = Expression.Parameter(type, "x");
Expression expr = arg;
foreach (var prop in props)
{
var pi = type.GetProperty(prop);
if (pi == null)
throw new ArgumentException(string.Format("Field {0} not found.", prop));
expr = Expression.Property(expr, pi);
type = pi.PropertyType;
}
var delegateType = typeof(Func<,>).MakeGenericType(source.GetType(), type);
var lambda = Expression.Lambda(delegateType, expr, arg);
var compiledLambda = lambda.Compile();
var value = compiledLambda.DynamicInvoke(source);
return value;
}
It can extract values of nested properties, like: ExtractValue(instance, "PropA.PropB.PropC").
Despite the fact I like this method and its implementation, when, say, PropB is null, DynamicInvoke() just throws NullReferenceException (wrapped by TargetInvocationException). Because I needed to know which exact property is null is such case, I modified its body a bit (standard step-by-step extraction chain):
public static object ExtractValue(object source, string property)
{
var props = property.Split('.');
for (var i = 0; i < props.Length; i++)
{
var type = source.GetType();
var prop = props[i];
var pi = type.GetProperty(prop);
if (pi == null)
throw new ArgumentException(string.Format("Field {0} not found.", prop));
source = pi.GetValue(source, null);
if (source == null && i < props.Length - 1)
throw new ArgumentNullException(pi.Name, "Extraction interrupted.");
}
return source;
}
Now it looks a bit worse (I like lambdas) but behaves much better, not only because it gives more meaningful information of what has failed, but also because this version is about 66 times faster than the first one (coarse test below):
var model = new ModelA
{
PropB = new ModelB {PropC = new ModelC {PropD = new ModelD {PropE = new ModelE {PropF = "hey"}}}}
};
const int times = 1000000;
var start = DateTime.Now;
for (var i = 0; i < times; i++)
ExtractValueFirst(model, "PropB.PropC.PropD.PropE.PropF");
var ticks_first = (DateTime.Now - start).Ticks;
Console.WriteLine(":: first - {0} iters tooks {1} ticks", times, ticks_first);
start = DateTime.Now;
for (var i = 0; i < times; i++)
ExtractValueSecond(model, "PropB.PropC.PropD.PropE.PropF");
var ticks_second= (DateTime.Now - start).Ticks;
Console.WriteLine(":: second - {0} iters tooks {1} ticks", times, ticks_second);
Console.WriteLine("ticks_first/ticks_second: {0}", (float)ticks_first / ticks_second);
Console.ReadLine();
How can this code be optimized in .NET to perform even faster (caching, direct IL maybe, etc)?

You can increase performance significantly by caching the compiled delegates:
static readonly ConcurrentDictionary<Tuple<Type,string>,Delegate> _delegateCache = new ConcurrentDictionary<Tuple<Type,string>,Delegate>();
public static object ExtractValue(object source, string expression)
{
Type type = source.GetType();
Delegate del = _delegateCache.GetOrAdd(new Tuple<Type,string>(type,expression),key => _getCompiledDelegate(key.Item1,key.Item2));
return del.DynamicInvoke(source);
}
// if you want to acces static aswell...
public static object ExtractStaticValue(Type type, string expression)
{
Delegate del = _delegateCache.GetOrAdd(new Tuple<Type,string>(type,expression),key => _getCompiledDelegate(key.Item1,key.Item2));
return del.DynamicInvoke(null);
}
private static Delegate _getCompiledDelegate(Type type, string expression)
{
var arg = Expression.Parameter(type, "x");
Expression expr = arg;
foreach (var prop in property.Split('.'))
{
var pi = type.GetProperty(prop);
if (pi == null) throw new ArgumentException(string.Format("Field {0} not found.", prop));
expr = Expression.Property(expr, pi);
type = pi.PropertyType;
}
var delegateType = typeof(Func<,>).MakeGenericType(source.GetType(), type);
var lambda = Expression.Lambda(delegateType, expr, arg);
return lambda.Compile();
}

I've done some execution time measurements, which are presented below:
private static Func<object, object> _cachedFunc;
private static Delegate _cachedDel;
static void Main(string[] args)
{
var model = new ModelA
{
PropB = new ModelB {PropC = new ModelC {PropD = new ModelD {PropE = new ModelE {PropF = "hey"}}}}
};
const string property = "PropB.PropC.PropD.PropE.PropF";
var watch = new Stopwatch();
var t1 = MeasureTime(watch, () => ExtractValueDelegate(model, property), "compiled delegate dynamic invoke");
var t2 = MeasureTime(watch, () => ExtractValueCachedDelegate(model, property), "compiled delegate dynamic invoke / cached");
var t3 = MeasureTime(watch, () => ExtractValueFunc(model, property), "compiled func invoke");
var t4 = MeasureTime(watch, () => ExtractValueCachedFunc(model, property), "compiled func invoke / cached");
var t5 = MeasureTime(watch, () => ExtractValueStepByStep(model, property), "step-by-step reflection");
var t6 = MeasureTime(watch, () => ExtractValueStandard(model), "standard access (model.prop.prop...)");
Console.ReadLine();
}
public static long MeasureTime<T>(Stopwatch sw, Func<T> funcToMeasure, string funcName)
{
const int times = 100000;
sw.Reset();
sw.Start();
for (var i = 0; i < times; i++)
funcToMeasure();
sw.Stop();
Console.WriteLine(":: {0, -45} - {1} iters tooks {2, 10} ticks", funcName, times, sw.ElapsedTicks);
return sw.ElapsedTicks;
}
public static object ExtractValueDelegate(object source, string property)
{
var ptr = GetCompiledDelegate(source.GetType(), property);
return ptr.DynamicInvoke(source);
}
public static object ExtractValueCachedDelegate(object source, string property)
{
var ptr = _cachedDel ?? (_cachedDel = GetCompiledDelegate(source.GetType(), property));
return ptr.DynamicInvoke(source);
}
public static object ExtractValueFunc(object source, string property)
{
var ptr = GetCompiledFunc(source.GetType(), property);
return ptr(source); //return ptr.Invoke(source);
}
public static object ExtractValueCachedFunc(object source, string property)
{
var ptr = _cachedFunc ?? (_cachedFunc = GetCompiledFunc(source.GetType(), property));
return ptr(source); //return ptr.Invoke(source);
}
public static object ExtractValueStepByStep(object source, string property)
{
var props = property.Split('.');
for (var i = 0; i < props.Length; i++)
{
var type = source.GetType();
var prop = props[i];
var pi = type.GetProperty(prop);
if (pi == null)
throw new ArgumentException(string.Format("Field {0} not found.", prop));
source = pi.GetValue(source, null);
if (source == null && i < props.Length - 1)
throw new ArgumentNullException(pi.Name, "Extraction interrupted.");
}
return source;
}
public static object ExtractValueStandard(ModelA source)
{
return source.PropB.PropC.PropD.PropE.PropF;
}
private static Func<object, object> GetCompiledFunc(Type type, string property)
{
var arg = Expression.Parameter(typeof(object), "x");
Expression expr = Expression.Convert(arg, type);
var propType = type;
foreach (var prop in property.Split('.'))
{
var pi = propType.GetProperty(prop);
if (pi == null) throw new ArgumentException(string.Format("Field {0} not found.", prop));
expr = Expression.Property(expr, pi);
propType = pi.PropertyType;
}
expr = Expression.Convert(expr, typeof(object));
var lambda = Expression.Lambda<Func<object, object>>(expr, arg);
return lambda.Compile();
}
private static Delegate GetCompiledDelegate(Type type, string property)
{
var arg = Expression.Parameter(type, "x");
Expression expr = arg;
var propType = type;
foreach (var prop in property.Split('.'))
{
var pi = propType.GetProperty(prop);
if (pi == null) throw new ArgumentException(string.Format("Field {0} not found.", prop));
expr = Expression.Property(expr, pi);
propType = pi.PropertyType;
}
var delegateType = typeof(Func<,>).MakeGenericType(type, propType);
var lambda = Expression.Lambda(delegateType, expr, arg);
return lambda.Compile();
}
Btw: As you can see I've omitted storing compiled lambdas inside dictionary (like in the answer qiven by CSharpie), because dictionary lookup is time consuming when you compare it to compiled lambdas execution time.

Convert LambdaExpression to String, Including Values

I have a method which converts a LambdaExpression to a string. I use these strings as keys for a cache.
string p = "x";
var a = LambdaToString<MyType>(m => m.P == p);
is different from this:
string p = "y";
var a = LambdaToString<MyType>(m => m.P == p);
However, the current state of my LambdaToString method is producing the same output regardless of the value of p. Which is:
(MyType.P == value(ConsoleApplication1.Program+<>c__DisplayClass0).p)
What I would like my LambdaToString function to do is to resolve the "value(class).p" portion of the expression into the actual literal string of "x" or "y" as the case may be.
Here is the current state of my LambdaToString method. I am not sure what I would need to do to modify it to produce the outputs I want:
public static string LambdaToString<T>(Expression<Func<T, bool>> expression)
{
string body = expression.Body.ToString();
foreach (var parm in expression.Parameters)
{
var parmName = parm.Name;
var parmTypeName = parm.Type.Name;
body = body.Replace(parmName + ".", parmTypeName + ".");
}
return body;
}

I use these strings as keys for a cache.
It's incorrect in a lot of circumstances, even it works in your project. Using Expression.ToString() as keys can be defeated easily by:
//counter-example 1
Expression<Func<string, bool>> exp1 = s => s == "a";
Expression<Func<string, bool>> exp2 = ss => ss == "a";
//the two will be considered different in your cache solution
//but they are essentially the same, well that's not the worst, see next
//counter-example 2
Expression<Func<int, bool>> exp3 = i => i > 10;
Expression<Func<long, bool>> exp4 = i => i > 10;
//the two will be considered the same in your cache solution
//of course they are different, probably hences runtime exceptions
The above is not an answer at all. If you don't care about that, let's continue based on "using strings as keys".
You want to cache the expressions, but identify those look-same expressions with constants in them. Then why not build the key with expression+constant? In your example code, the keys will be:
"m => m.P == p ##SPECIAL_SEPERATOR## x"
"m => m.P == p ##SPECIAL_SEPERATOR## y"
and yes, if one constant contains values like "##SPECIAL_SEPERATOR##", everything is going to crash. This is not a rigorous solution from the very beginning, because you choose strings as the cache key.
If you decide to choose another cache approach, check this.

Well, to get p value, you could do (probably easier and more robust way to do this, but).
public static string LambdaToString<T>(Expression<Func<T, bool>> expression)
{
BinaryExpression binaryExpression = expression.Body as BinaryExpression;
Expression right = binaryExpression.Right;//right part of the "==" of your predicate
var objectMember = Expression.Convert(right, typeof(object));//convert to object, as we don't know what's in
var getterLambda = Expression.Lambda<Func<object>>(objectMember);
var getter = getterLambda.Compile();
var valueYouWant = getter();//here's the "x" or "y"
//...
or shorter
Expression right = (expression.Body as BinaryExpression).Right;
var valueYouWant = Expression.Lambda(right).Compile().DynamicInvoke();
CAUTION
Of course, this won't fit a lot of scenarii, it's just the basic to understand how to get a value. It won't work if your predicate is
var x = 1;
var y = 2;
var result = LambdaToString<YourType>(v => v.A== x && v.B == y)

Here is my answer. Ideally this would be able to handle any possible Expression that is thrown at it. Right now it most likely does not, but it handled all the simple, common things I threw at it in my tests.
If you come up with an example this doesn't handle, leave it in the comments and I will try to modify the function to handle it.
public static string LambdaToString<T>(Expression<Func<T, bool>> expression)
{
var replacements = new Dictionary<string, string>();
WalkExpression(replacements, expression);
string body = expression.Body.ToString();
foreach (var parm in expression.Parameters)
{
var parmName = parm.Name;
var parmTypeName = parm.Type.Name;
body = body.Replace(parmName + ".", parmTypeName + ".");
}
foreach (var replacement in replacements)
{
body = body.Replace(replacement.Key, replacement.Value);
}
return body;
}
private static void WalkExpression(Dictionary<string, string> replacements, Expression expression)
{
switch (expression.NodeType)
{
case ExpressionType.MemberAccess:
string replacementExpression = expression.ToString();
if (replacementExpression.Contains("value("))
{
string replacementValue = Expression.Lambda(expression).Compile().DynamicInvoke().ToString();
if (!replacements.ContainsKey(replacementExpression))
{
replacements.Add(replacementExpression, replacementValue.ToString());
}
}
break;
case ExpressionType.GreaterThan:
case ExpressionType.GreaterThanOrEqual:
case ExpressionType.LessThan:
case ExpressionType.LessThanOrEqual:
case ExpressionType.OrElse:
case ExpressionType.AndAlso:
case ExpressionType.Equal:
var bexp = expression as BinaryExpression;
WalkExpression(replacements, bexp.Left);
WalkExpression(replacements, bexp.Right);
break;
case ExpressionType.Call:
var mcexp = expression as MethodCallExpression;
foreach (var argument in mcexp.Arguments)
{
WalkExpression(replacements, argument);
}
break;
case ExpressionType.Lambda:
var lexp = expression as LambdaExpression;
WalkExpression(replacements, lexp.Body);
break;
case ExpressionType.Constant:
//do nothing
break;
default:
Trace.WriteLine("Unknown type");
break;
}

very quick and dirty solution would be to pass the parameter name and its value and just replace it.
public static string LambdaToString<T>(Expression<Func<T, bool>> expression, string value,string paramName )
{
string body = expression.Body.ToString().Replace(paramName,value);
foreach (var parm in expression.Parameters)
{
var parmName = parm.Name;
var parmTypeName = parm.Type.Name;
body = body.Replace(parmName + ".", parmTypeName + ".");
}
return body;
}