Develop Reference

Handling null values building expressions with child items

We are using Expressions to build a refined select statement (being used by GraphQL .Net) that only queries the properties asked for.
We parse the raw query and use a recursive method to build an expression that we use to generate our select (via entity framework)
Our code is HEAVILY based on this answer provided by #Svyatoslav Danyliv.
Our expressionbuilder class looks like this:
public class ExpressionBuilder
{
public Expression<Func<TSource, TTarget>> BuildSelector<TSource, TTarget>(string members)
{
return BuildSelector<TSource, TTarget>(members.Split(',').Select(m => m.Trim()));
}
public Expression<Func<TSource, TTarget>> BuildSelector<TSource, TTarget>(IEnumerable<string> members)
{
var parameter = Expression.Parameter(typeof(TSource), "e");
var body = NewObject(typeof(TTarget), parameter, members.Select(m => m.Split('.')));
return Expression.Lambda<Func<TSource, TTarget>>(body, parameter);
}
private Expression NewObject(Type targetType, Expression source, IEnumerable<string[]> memberPaths, int depth = 0)
{
var bindings = new List<MemberBinding>();
var target = Expression.Constant(null, targetType);
foreach (var memberGroup in memberPaths.GroupBy(path => path[depth]))
{
var memberName = memberGroup.Key;
var targetMember = Expression.PropertyOrField(target, memberName);
var sourceMember = Expression.PropertyOrField(source, memberName);
var childMembers = memberGroup.Where(path => depth + 1 < path.Length).ToList();
Expression targetValue = null;
if (!childMembers.Any())
{
targetValue = sourceMember;
}
else
{
if (IsEnumerableType(targetMember.Type, out var sourceElementType) &&
IsEnumerableType(targetMember.Type, out var targetElementType))
{
var sourceElementParam = Expression.Parameter(sourceElementType, "e");
targetValue = NewObject(targetElementType, sourceElementParam, childMembers, depth + 1);
targetValue = Expression.Call(typeof(Enumerable), nameof(Enumerable.Select),
new[] { sourceElementType, targetElementType }, sourceMember,
Expression.Lambda(targetValue, sourceElementParam));
targetValue = CorrectEnumerableResult(targetValue, targetElementType, targetMember.Type);
}
else
{
targetValue = NewObject(targetMember.Type, sourceMember, childMembers, depth + 1);
}
}
bindings.Add(Expression.Bind(targetMember.Member, targetValue));
}
return Expression.MemberInit(Expression.New(targetType), bindings);
}
private bool IsEnumerableType(Type type, out Type elementType)
{
foreach (var intf in type.GetInterfaces())
{
if (intf.IsGenericType && intf.GetGenericTypeDefinition() == typeof(IEnumerable<>))
{
elementType = intf.GetGenericArguments()[0];
return true;
}
}
elementType = null;
return false;
}
private bool IsSameCollectionType(Type type, Type genericType, Type elementType)
{
var result = genericType.MakeGenericType(elementType).IsAssignableFrom(type);
return result;
}
private Expression CorrectEnumerableResult(Expression enumerable, Type elementType, Type memberType)
{
if (memberType == enumerable.Type)
return enumerable;
if (memberType.IsArray)
return Expression.Call(typeof(Enumerable), nameof(Enumerable.ToArray), new[] { elementType }, enumerable);
if (IsSameCollectionType(memberType, typeof(List<>), elementType)
|| IsSameCollectionType(memberType, typeof(ICollection<>), elementType)
|| IsSameCollectionType(memberType, typeof(IReadOnlyList<>), elementType)
|| IsSameCollectionType(memberType, typeof(IReadOnlyCollection<>), elementType))
return Expression.Call(typeof(Enumerable), nameof(Enumerable.ToList), new[] { elementType }, enumerable);
throw new NotImplementedException($"Not implemented transformation for type '{memberType.Name}'");
}
}
and the usage is as follows:
var builder = new ExpressionBuilder();
var statement = builder.BuildSelector<Payslip, Payslip>(selectStatement);
The select statement above looks like "id,enrollment.id" where we are selecting parentobject.id and parentobject.enrollment.id
This statement variable is then used in a EntityQueryable.Select(statement) against a filtered database collection there we have essentially done the 'where' aspect of the query.
When working correctly the sql generated is ONLY the fields requested, and it works great. However in this instance it is trying to access the .Id property of a null value and constantly fails.
It is currently failing because the "Enrollment" property of the parentObject is not set, thus null.
We get the following error
System.InvalidOperationException: Nullable object must have a value.
at System.Nullable`1.get_Value()
at lambda_method2198(Closure , QueryContext , DbDataReader , ResultContext , SingleQueryResultCoordinator )
at Microsoft.EntityFrameworkCore.Query.Internal.SingleQueryingEnumerable`1.Enumerator.MoveNext()
I've tried to implement a try catch expression but keep failing, and end up with the exact same error. At this time, we aren't hitting the database, so we don't know if the value will be null or not (it works fine if it's not null).
I am by no means an expert on expressions, and am really struggling to find just where I should be wrapping this value, or even how. Ideally I'd just like to return null, but worst case a default enrollment object could be detected client side and possibly handled ok.
** UPDATE!! **
Reading into this I think that this is a result of this issue. The workaround would involve building something like this null check into the expression builder.
.Select(r => new Something()
{
X1 = r.X1,
X2 = r.X2,
X = new X()
{
Name= r.X.Name
},
XX = r.XXNullableId == null ? null : new XX()
{
XNumber = r.XX.XNumber
}
})
I have created a VERY simple EF Core Application here that creates a simple three table database and recreates the problem.
I believe the workaround mentioned above will work, it's just a matter of figuring out how to add it to the expression builder.

So I figured this one out (sort of) by special casing the property that causes the issue.
Due to the nature of our model, this situation will only occur in a few places, it's easier to add some workarounds for them, that have the expression builder do some strange null check on every property.
I created a simple attribute that I put on the 'Object' Property and have it point to the property I need to check for null.
That looks like this:
[System.AttributeUsage(System.AttributeTargets.Property, Inherited = true, AllowMultiple = true)]
public class ExpressionBuilderNullCheckForPropertyAttribute : Attribute
{
public ExpressionBuilderNullCheckForPropertyAttribute(string propertyName)
{
this.PropertyName = propertyName;
}
public string PropertyName { get; set; }
}
So now, my 'Person' Class (from the linked example repo above) now looks like this.
public class Person
{
[Required]
[Key, DatabaseGenerated(DatabaseGeneratedOption.Identity)]
public Guid Id { get; set; }
public string Name { get; set; }
public int Age { get; set; }
public Nullable<Guid> VetId { get; set; }
[ExpressionBuilderNullCheckForProperty(nameof(VetId))]
public Vet Vet { get; set; }
public List<Pet> Pets { get; set; }
}
Then I added a check into the ExpressionBuilder class to check for that attribute, if it is present, lookup the property it has mentioned, then perform a null check. If that property is null, then return a default object for that type. Which, in this case, is null.
if (targetMember.Member.GetCustomAttributesData().Any(x => x.AttributeType == typeof(ExpressionBuilderNullCheckForPropertyAttribute)))
{
var att = (ExpressionBuilderNullCheckForPropertyAttribute)targetMember.Member.GetCustomAttributes(true).First(x => x.GetType() == typeof(ExpressionBuilderNullCheckForPropertyAttribute));
var propertyToCheck = att.PropertyName;
var valueToCheck = Expression.PropertyOrField(source, propertyToCheck);
var nullCheck = Expression.Equal(valueToCheck, Expression.Constant(null, targetMember.Type));
var checkForNull = Expression.Condition(nullCheck, Expression.Default(targetMember.Type), targetValue);
bindings.Add(Expression.Bind(targetMember.Member, checkForNull));
}
else
{
bindings.Add(Expression.Bind(targetMember.Member, targetValue));
}
I have updated the example repo if anyone wants to see the fix working.

Variable 'x.Sub' of type 'SubType' referenced from scope '' but it is not defined error

Check this fiddle for the error: https://dotnetfiddle.net/tlz4Qg
I have two classes like this:
public class ParentType{
private ParentType(){}
public int Id { get; protected set; }
public SubType Sub { get; protected set; }
}
public class SubType{
private SubType(){}
public int Id { get; protected set; }
}
I am going to transform a multilevel anonymous expression to a multilevel non-anonymous expression. To achieve this I have an expression like the below-mentioned one:
x => new
{
x.Id,
Sub = new
{
x.Sub.Id
}
}
To achieve that goal, I have transformed it to an expression like this:
x => new ParentType()
{
Id = x.Id,
Sub = new SubType()
{
Id = x.Sub.Id
},
}
But when I call Compile() method, I get the following error:
Variable 'x.Sub' of type 'SubType' referenced from scope '' but it is not defined
Here is my visitor class:
public class ReturnTypeVisitor<TIn, TOut> : ExpressionVisitor
{
private readonly Type funcToReplace;
private ParameterExpression currentParameter;
private ParameterExpression defaultParameter;
private Type currentType;
public ReturnTypeVisitor() => funcToReplace = typeof(Func<,>).MakeGenericType(typeof(TIn), typeof(object));
protected override Expression VisitNew(NewExpression node)
{
if (!node.Type.IsAnonymousType())
return base.VisitNew(node);
if (currentType == null)
currentType = typeof(TOut);
var ctor = currentType.GetPrivateConstructor();
if (ctor == null)
return base.VisitNew(node);
NewExpression expr = Expression.New(ctor);
IEnumerable<MemberBinding> bindings = node.Members.Select(x =>
{
var mi = currentType.GetProperty(x.Name);
//if the type is anonymous then I need to transform its body
if (((PropertyInfo)x).PropertyType.IsAnonymousType())
{
//This section is became unnecessary complex!
//
var property = (PropertyInfo)x;
var parentType = currentType;
var parentParameter = currentParameter;
currentType = currentType.GetProperty(property.Name).PropertyType;
currentParameter = Expression.Parameter(currentType, currentParameter.Name + "." + property.Name);
//I pass the inner anonymous expression to VisitNew and make the non-anonymous expression from it
var xOriginal = VisitNew(node.Arguments.FirstOrDefault(a => a.Type == property.PropertyType) as NewExpression);
currentType = parentType;
currentParameter = parentParameter;
return (MemberBinding)Expression.Bind(mi, xOriginal);
}
else//if type is not anonymous then simple find the property and make the memberbinding
{
var xOriginal = Expression.PropertyOrField(currentParameter, x.Name);
return (MemberBinding)Expression.Bind(mi, xOriginal);
}
});
return Expression.MemberInit(expr, bindings);
}
protected override Expression VisitLambda<T>(Expression<T> node)
{
if (typeof(T) != funcToReplace)
return base.VisitLambda(node);
defaultParameter = node.Parameters.First();
currentParameter = defaultParameter;
var body = Visit(node.Body);
return Expression.Lambda<Func<TIn, TOut>>(body, currentParameter);
}
}
And use it like this:
public static Expression<Func<Tin, Tout>> Transform<Tin, Tout>(this Expression<Func<Tin, object>> source)
{
var visitor = new ReturnTypeVisitor<Tin, Tout>();
var result = (Expression<Func<Tin, Tout>>)visitor.Visit(source);
return result;// result.Compile() throw the aforementioned error
}
Here is the extension methods used inside my Visitor class:
public static ConstructorInfo GetPrivateConstructor(this Type type) =>
type.GetConstructor(BindingFlags.Instance | BindingFlags.Public | BindingFlags.NonPublic, null, Type.EmptyTypes, null);
// this hack taken from https://stackoverflow.com/a/2483054/4685428
// and https://stackoverflow.com/a/1650895/4685428
public static bool IsAnonymousType(this Type type)
{
var markedWithAttribute = type.GetCustomAttributes(typeof(CompilerGeneratedAttribute), inherit: false).Any();
var typeName = type.Name;
return markedWithAttribute
&& (typeName.StartsWith("<>") || type.Name.StartsWith("VB$"))
&& typeName.Contains("AnonymousType");
}
Update
Here is the .Net Fiddle link for the problem: https://dotnetfiddle.net/tlz4Qg
Update
I have removed the extra codes that seems to be out of the problem scope.

The cause of the problem in question is the line
currentParameter = Expression.Parameter(currentType, currentParameter.Name + "." + property.Name);
inside VisitNew method.
With your sample, it creates a new parameter called "x.Sub", so if we mark the parameters with {}, the actual result is
Sub = new SubType()
{
Id = {x.Sub}.Id
},
rather than expected
Sub = new SubType()
{
Id = {x}.Sub.Id
},
In general you should not create new ParameterExpressions except when remapping lambda expressions. And all newly created parameters should be passed to Expression.Lambda call, otherwise they will be considered "not defined".
Also please note that the visitor code has some assumptions which doesn't hold in general. For instance
var xOriginal = Expression.PropertyOrField(currentParameter, x.Name);
won't work inside nested new, because there you need access to a member of the x parameter like x.Sub.Id rather than x.Id. Which is basically the corersonding expression from NewExpression.Arguments.
Processing nested lambda expressions or collection type members and LINQ methods with expression visitors requires much more state control. While converting simple nested anonymous new expression like in the sample does not even need a ExpressionVisitor, because it could easily be achieved with simple recursive method like this:
public static Expression<Func<Tin, Tout>> Transform<Tin, Tout>(this Expression<Func<Tin, object>> source)
{
return Expression.Lambda<Func<Tin, Tout>>(
Transform(source.Body, typeof(Tout)),
source.Parameters);
}
static Expression Transform(Expression source, Type type)
{
if (source.Type != type && source is NewExpression newExpr && newExpr.Members.Count > 0)
{
return Expression.MemberInit(Expression.New(type), newExpr.Members
.Select(m => type.GetProperty(m.Name))
.Zip(newExpr.Arguments, (m, e) => Expression.Bind(m, Transform(e, m.PropertyType))));
}
return source;
}

How to create lambda expressions for every property of a class

I have a following problem. I have to iterate over all the properties of the class to configure some builder. A class has a lot of properties, so the code is cumbersome. It looks like this:
var b = builder<MyTypeWith1000Properties>
.WithProperty(x=>x.Property1)
.WithProperty(x=>x.Property2)
...
.WithProperty(x=>x.Property1000);
The code is repeated in many places for many differend types, not only MyTypeWith1000Properties. I was thinking about creating some extension, like this:
var b = builder<MyTypeWith1000Properties>
.WithAllProperties();
and then in the WithAllProperties I could iterate over type properties using Reflection, like this:
public static IDataExtractor<T> WithAllProperties(this IDataExtractor<T> extractor)
{
var properties = typeof(T).GetProperties();
foreach (var property in properties)
{
extractor = extractor.WithProperty(/*the problem is here/*);
}
return extractor;
}
How to convert the property variable in the loop to a corresponding expression
Expression<Func<TRow, TValue>> propertyExpression
as this is what WithProperty expects

Update - Setting the correct parameter value in Expression.Lambda>
You could try something like this
public static class BuilderExtension
{
public static IDataExtractor<T> WithAllProperties<T>(this IDataExtractor<T> extractor)
{
var properties = typeof(T).GetProperties();
foreach (var propertyInfo in properties)
{
var parameter = Expression.Parameter(typeof(T), "x");
var property = Expression.Property(parameter, propertyInfo);
var lambda = Expression.Lambda<Func<T, object>>(property,parameter);
extractor = extractor.WithProperty(lambda);
}
return extractor;
}
}
Suppose you have the following class structures
public class MyTypeWith100Properties
{
public string Property1 { get; set; }
public string Property2 { get; set; }
public string Property3 { get; set; }
public string Property4 { get; set; }
}
public interface IDataExtractor<T>
{
IDataExtractor<T> WithProperty(Expression<Func<T, object>> expr);
}
public class DataExtractor<T> : IDataExtractor<T>
{
public List<Expression<Func<T, object>>> Expressions { get; private set; }
public DataExtractor() {
Expressions = new List<Expression<Func<T, object>>>();
}
public IDataExtractor<T> WithProperty(Expression<Func<T, object>> expr)
{
Expressions.Add(expr);
return this;
}
}
Then if you run this
var builder = new DataExtractor<MyTypeWith100Properties>();
var b = builder.WithAllProperties<MyTypeWith100Properties>()
as DataExtractor<MyTypeWith100Properties>;
var instance = new MyTypeWith100Properties() {
Property1 = "This is property 1",
Property2 = "This is property 2",
Property3 = "This is property 3",
Property4 = "This is property 4"
};
foreach (var current in b.Expressions)
{
var compiled = current.Compile();
var result = compiled.Invoke(instance);
Console.WriteLine(result);
}
Your output will be
This is property 1
This is property 2
This is property 3
This is property 4

WithProperty is a generic method that takes a type parameter implied by the result type of the property member access expression, which is what TValue represents in its declaration. Since you want to use reflection to generate the lambdas, you have to do the WithProperty call dynamically as well so you can call the one with the proper type (e.g. WithProperty<String> for a String property).
Here is an extension method that generates a lambda that consists of all the chained WithProperty calls for all the properties in a class, and then compiles and calls the lambda on the IDataExtractor. I chained all the calls together and then compiled because there might be some overhead for compiling, so I didn't want to compile and call code for each property individually.
public static class IDataExtractorExt {
public static IDataExtractor<TRow> WithAllProperties<TRow>(this IDataExtractor<TRow> extractor) {
var p = Expression.Parameter(typeof(IDataExtractor<TRow>), "p"); // final lambda parameter
Expression ansBody = p; // start with p => p
var withPropGenericMI = typeof(IDataExtractor<TRow>).GetMethod("WithProperty"); // lookup WithProperty<> generic method
var properties = typeof(TRow).GetProperties();
foreach (var property in properties) {
var withPropMI = withPropGenericMI.MakeGenericMethod(property.PropertyType); // instantiate generic WithProperty<> to property type
var pxp = Expression.Parameter(typeof(TRow), "x"); // property accessor lambda parameter
var pxb = Expression.PropertyOrField(pxp, property.Name); // property accessor expression x.property
Expression propExpr = Expression.Lambda(pxb, pxp); // x => x.property
ansBody = Expression.Call(ansBody, withPropMI, propExpr); // build up p => p.WithProperty(x => x.property)...
}
return ((IDataExtractor<TRow>)(Expression.Lambda(ansBody, p).Compile().DynamicInvoke(extractor)));
}
}

Get the parameter value from a Linq Expression

I have the following class
public class MyClass
{
public bool Delete(Product product)
{
// some code.
}
}
Now I have a helper class that looks like this
public class Helper<T, TResult>
{
public Type Type;
public string Method;
public Type[] ArgTypes;
public object[] ArgValues;
public Helper(Expression<Func<T, TResult>> expression)
{
var body = (System.Linq.Expressions.MethodCallExpression)expression.Body;
this.Type = typeof(T);
this.Method = body.Method.Name;
this.ArgTypes = body.Arguments.Select(x => x.Type).ToArray();
this.ArgValues = ???
}
}
The idea ist to use this code from somewhere:
// I am returning a helper somewhere
public Helper<T> GetMethod<T>()
{
var product = GetProduct(1);
return new Helper<MyClass>(x => x.Delete(product));
}
// some other class decides, when to execute the helper
// Invoker already exists and is responsible for executing the method
// that is the main reason I don't just comile and execute my Expression
public bool ExecuteMethod<T>(Helper<T> helper)
{
var instance = new MyClass();
var Invoker = new Invoker(helper.Type, helper.Method, helper.ArgTypes, helper.ArgValues);
return (bool)Invoker.Invoke(instance);
}
The point where I am stuck is how to extract the arguments from the expression itself.
I found this way
((ConstantExpression)((MemberExpression)body.Arguments[0]).Expression).Value
which seems to be an object type with a field "product" but I believe there must be a simpler solution.
Any suggestions.
Update
Just to clarify, I modified my code according to what I want to achive. In my real word application I already have a class that does the same but without an expression tree:
var helper = new Helper(typeof(MyClass), "Delete",
new Type[] { typeof(Product) }, new object[] {product}));
The main reason for my Helper<T> is to have Compile-Time checking if the method signature is valid.
Update 2
This is my current implementation, is there a better way to acces the values, without using reflection?
public Helper(Expression<Func<T, TResult>> expression)
{
var body = (System.Linq.Expressions.MethodCallExpression)expression.Body;
this.Type = typeof(T);
this.Method = body.Method.Name;
this.ArgTypes = body.Arguments.Select(x => x.Type).ToArray();
var values = new List<object>();
foreach(var arg in body.Arguments)
{
values.Add(
(((ConstantExpression)exp.Expression).Value).GetType()
.GetField(exp.Member.Name)
.GetValue(((ConstantExpression)exp.Expression).Value);
);
}
this.ArgValues = values.ToArray();
}

This method works pretty well. It returns the argument types and values for an Expression>
private static KeyValuePair<Type, object>[] ResolveArgs<T>(Expression<Func<T, object>> expression)
{
var body = (System.Linq.Expressions.MethodCallExpression)expression.Body;
var values = new List<KeyValuePair<Type, object>>();
foreach (var argument in body.Arguments)
{
var exp = ResolveMemberExpression(argument);
var type = argument.Type;
var value = GetValue(exp);
values.Add(new KeyValuePair<Type, object>(type, value));
}
return values.ToArray();
}
public static MemberExpression ResolveMemberExpression(Expression expression)
{
if (expression is MemberExpression)
{
return (MemberExpression)expression;
}
else if (expression is UnaryExpression)
{
// if casting is involved, Expression is not x => x.FieldName but x => Convert(x.Fieldname)
return (MemberExpression)((UnaryExpression)expression).Operand;
}
else
{
throw new NotSupportedException(expression.ToString());
}
}
private static object GetValue(MemberExpression exp)
{
// expression is ConstantExpression or FieldExpression
if (exp.Expression is ConstantExpression)
{
return (((ConstantExpression)exp.Expression).Value)
.GetType()
.GetField(exp.Member.Name)
.GetValue(((ConstantExpression)exp.Expression).Value);
}
else if (exp.Expression is MemberExpression)
{
return GetValue((MemberExpression)exp.Expression);
}
else
{
throw new NotImplementedException();
}
}

You can compile the argument expression and then invoke it to calculate the value:
var values = new List<object>();
foreach(var arg in body.Arguments)
{
var value = Expression.Lambda(argument).Compile().DynamicInvoke();
values.Add(value);
}
this.ArgValues = values.ToArray();

Here is an example of creation of a delegate using a lambda. The object instance is encapsulated into the delegate using a C# feature called closure.
MyClass instance = new MyClass();
//This following line cannot be changed to var declaration
//since C# can't infer the type.
Func<Product, bool> deleteDelegate = p => instance.Delete(p);
Product product = new Product();
bool deleted = deleteDelegate(product);
Alternatively you are trying to create a Helper that automagically Currys.
public class Helper<T>
where T : new()
{
public TResult Execute<TResult>(Func<T, TResult> methodLambda)
{
var instance = new T();
return methodLamda(instance);
}
}
public void Main()
{
var helper = new Helper<MyClass>();
var product = new Product();
helper.Execute(x => x.Delete(product));
}
However I have to say this problem looks suspiciously like the creation of a Helper class to handle the lifetime of a WCF proxy....You know...just say...in which case this ISN'T how I would approach this...simply because this approach leaks WCF specific code into your domain.

How to get the value of an expression

I have this function that uses Linq expressions:
private Expression GetFieldValueExpression(ParameterExpression parameter, string fieldName)
{
Expression properyIndexExpression = System.Linq.Expressions.Expression.Constant (fieldName, typeof(string));
IndexExpression fieldValueExpression = System.Linq.Expressions.Expression.Property(parameter, "Item", new Expression[] { properyIndexExpression });
return Expression.Property(fieldValueExpression, "Value");
}
The value returned by Expression.Property(fieldValueExpression, "Value") is of type string.
I don't know how to get it. I know that i must create a lambda and compile it, but i don't know how.
Thank you for your time.

Perhaps you are looking for code like this:
public void EvaluateAnExpression()
{
//Make the parameter
var parm = Expression.Parameter(typeof(TestClass),"parm");
//Use your method to build the expression
var exp = GetFieldValueExpression(parm, "testField");
//Build a lambda for the expression
var lambda = Expression.Lambda(exp, parm);
//Compile the lamda and cast the result to a Func<>
var compiled = (Func<TestClass, string>)lambda.Compile();
//We'll make up some object to test on
var obj = new TestClass();
//Get the result (it will be TESTFIELD)
var result = compiled(obj);
}
that code assumes some test classes that look like this (basically the indexer property just returns the input but in upper case - a trivial example but works for testing):
public class TestClass
{
public InnerClass this[string indexParameter]
{
get
{
return new InnerClass { Value = indexParameter.ToUpper() };
}
}
}
public class InnerClass
{
public string Value { get; set; }
}