Develop Reference

Handling null values building expressions with child items

We are using Expressions to build a refined select statement (being used by GraphQL .Net) that only queries the properties asked for.
We parse the raw query and use a recursive method to build an expression that we use to generate our select (via entity framework)
Our code is HEAVILY based on this answer provided by #Svyatoslav Danyliv.
Our expressionbuilder class looks like this:
public class ExpressionBuilder
{
public Expression<Func<TSource, TTarget>> BuildSelector<TSource, TTarget>(string members)
{
return BuildSelector<TSource, TTarget>(members.Split(',').Select(m => m.Trim()));
}
public Expression<Func<TSource, TTarget>> BuildSelector<TSource, TTarget>(IEnumerable<string> members)
{
var parameter = Expression.Parameter(typeof(TSource), "e");
var body = NewObject(typeof(TTarget), parameter, members.Select(m => m.Split('.')));
return Expression.Lambda<Func<TSource, TTarget>>(body, parameter);
}
private Expression NewObject(Type targetType, Expression source, IEnumerable<string[]> memberPaths, int depth = 0)
{
var bindings = new List<MemberBinding>();
var target = Expression.Constant(null, targetType);
foreach (var memberGroup in memberPaths.GroupBy(path => path[depth]))
{
var memberName = memberGroup.Key;
var targetMember = Expression.PropertyOrField(target, memberName);
var sourceMember = Expression.PropertyOrField(source, memberName);
var childMembers = memberGroup.Where(path => depth + 1 < path.Length).ToList();
Expression targetValue = null;
if (!childMembers.Any())
{
targetValue = sourceMember;
}
else
{
if (IsEnumerableType(targetMember.Type, out var sourceElementType) &&
IsEnumerableType(targetMember.Type, out var targetElementType))
{
var sourceElementParam = Expression.Parameter(sourceElementType, "e");
targetValue = NewObject(targetElementType, sourceElementParam, childMembers, depth + 1);
targetValue = Expression.Call(typeof(Enumerable), nameof(Enumerable.Select),
new[] { sourceElementType, targetElementType }, sourceMember,
Expression.Lambda(targetValue, sourceElementParam));
targetValue = CorrectEnumerableResult(targetValue, targetElementType, targetMember.Type);
}
else
{
targetValue = NewObject(targetMember.Type, sourceMember, childMembers, depth + 1);
}
}
bindings.Add(Expression.Bind(targetMember.Member, targetValue));
}
return Expression.MemberInit(Expression.New(targetType), bindings);
}
private bool IsEnumerableType(Type type, out Type elementType)
{
foreach (var intf in type.GetInterfaces())
{
if (intf.IsGenericType && intf.GetGenericTypeDefinition() == typeof(IEnumerable<>))
{
elementType = intf.GetGenericArguments()[0];
return true;
}
}
elementType = null;
return false;
}
private bool IsSameCollectionType(Type type, Type genericType, Type elementType)
{
var result = genericType.MakeGenericType(elementType).IsAssignableFrom(type);
return result;
}
private Expression CorrectEnumerableResult(Expression enumerable, Type elementType, Type memberType)
{
if (memberType == enumerable.Type)
return enumerable;
if (memberType.IsArray)
return Expression.Call(typeof(Enumerable), nameof(Enumerable.ToArray), new[] { elementType }, enumerable);
if (IsSameCollectionType(memberType, typeof(List<>), elementType)
|| IsSameCollectionType(memberType, typeof(ICollection<>), elementType)
|| IsSameCollectionType(memberType, typeof(IReadOnlyList<>), elementType)
|| IsSameCollectionType(memberType, typeof(IReadOnlyCollection<>), elementType))
return Expression.Call(typeof(Enumerable), nameof(Enumerable.ToList), new[] { elementType }, enumerable);
throw new NotImplementedException($"Not implemented transformation for type '{memberType.Name}'");
}
}
and the usage is as follows:
var builder = new ExpressionBuilder();
var statement = builder.BuildSelector<Payslip, Payslip>(selectStatement);
The select statement above looks like "id,enrollment.id" where we are selecting parentobject.id and parentobject.enrollment.id
This statement variable is then used in a EntityQueryable.Select(statement) against a filtered database collection there we have essentially done the 'where' aspect of the query.
When working correctly the sql generated is ONLY the fields requested, and it works great. However in this instance it is trying to access the .Id property of a null value and constantly fails.
It is currently failing because the "Enrollment" property of the parentObject is not set, thus null.
We get the following error
System.InvalidOperationException: Nullable object must have a value.
at System.Nullable`1.get_Value()
at lambda_method2198(Closure , QueryContext , DbDataReader , ResultContext , SingleQueryResultCoordinator )
at Microsoft.EntityFrameworkCore.Query.Internal.SingleQueryingEnumerable`1.Enumerator.MoveNext()
I've tried to implement a try catch expression but keep failing, and end up with the exact same error. At this time, we aren't hitting the database, so we don't know if the value will be null or not (it works fine if it's not null).
I am by no means an expert on expressions, and am really struggling to find just where I should be wrapping this value, or even how. Ideally I'd just like to return null, but worst case a default enrollment object could be detected client side and possibly handled ok.
** UPDATE!! **
Reading into this I think that this is a result of this issue. The workaround would involve building something like this null check into the expression builder.
.Select(r => new Something()
{
X1 = r.X1,
X2 = r.X2,
X = new X()
{
Name= r.X.Name
},
XX = r.XXNullableId == null ? null : new XX()
{
XNumber = r.XX.XNumber
}
})
I have created a VERY simple EF Core Application here that creates a simple three table database and recreates the problem.
I believe the workaround mentioned above will work, it's just a matter of figuring out how to add it to the expression builder.

So I figured this one out (sort of) by special casing the property that causes the issue.
Due to the nature of our model, this situation will only occur in a few places, it's easier to add some workarounds for them, that have the expression builder do some strange null check on every property.
I created a simple attribute that I put on the 'Object' Property and have it point to the property I need to check for null.
That looks like this:
[System.AttributeUsage(System.AttributeTargets.Property, Inherited = true, AllowMultiple = true)]
public class ExpressionBuilderNullCheckForPropertyAttribute : Attribute
{
public ExpressionBuilderNullCheckForPropertyAttribute(string propertyName)
{
this.PropertyName = propertyName;
}
public string PropertyName { get; set; }
}
So now, my 'Person' Class (from the linked example repo above) now looks like this.
public class Person
{
[Required]
[Key, DatabaseGenerated(DatabaseGeneratedOption.Identity)]
public Guid Id { get; set; }
public string Name { get; set; }
public int Age { get; set; }
public Nullable<Guid> VetId { get; set; }
[ExpressionBuilderNullCheckForProperty(nameof(VetId))]
public Vet Vet { get; set; }
public List<Pet> Pets { get; set; }
}
Then I added a check into the ExpressionBuilder class to check for that attribute, if it is present, lookup the property it has mentioned, then perform a null check. If that property is null, then return a default object for that type. Which, in this case, is null.
if (targetMember.Member.GetCustomAttributesData().Any(x => x.AttributeType == typeof(ExpressionBuilderNullCheckForPropertyAttribute)))
{
var att = (ExpressionBuilderNullCheckForPropertyAttribute)targetMember.Member.GetCustomAttributes(true).First(x => x.GetType() == typeof(ExpressionBuilderNullCheckForPropertyAttribute));
var propertyToCheck = att.PropertyName;
var valueToCheck = Expression.PropertyOrField(source, propertyToCheck);
var nullCheck = Expression.Equal(valueToCheck, Expression.Constant(null, targetMember.Type));
var checkForNull = Expression.Condition(nullCheck, Expression.Default(targetMember.Type), targetValue);
bindings.Add(Expression.Bind(targetMember.Member, checkForNull));
}
else
{
bindings.Add(Expression.Bind(targetMember.Member, targetValue));
}
I have updated the example repo if anyone wants to see the fix working.

C# order List of Parent objects by Child's property via Reflection

I have a method that orders a queryable by a property via reflection, like so:
public static IQueryable<TEntity> OrderBy<TEntity>(this IQueryable<TEntity> source, string orderByProperty, bool desc)
{
string command = desc ? "OrderByDescending" : "OrderBy";
var parts = orderByProperty.Split('.');
var parameter = Expression.Parameter(typeof(TEntity), "p");
Expression body = parameter;
if (parts.Length > 1)
{
for (var i = 0; i < parts.Length - 1; i++)
{
body = Expression.Property(body, parts[i]);
}
}
var property = body.Type.GetProperty(parts.LastOrDefault())
body = Expression.Property(body, property.Name);
var orderByExpression = Expression.Lambda(body, parameter);
resultExpression = Expression.Call(typeof(Queryable), command,
new Type[] { typeof(TEntity), property.PropertyType },
source.Expression, Expression.Quote(orderByExpression));
return source.Provider.CreateQuery<TEntity>(resultExpression);
}
And it works if I use it for the object's properties or its children's properties, but it doesn't work when a child property is a List.
// Success
db.Users.OrderBy("Name", true)
db.Users.OrderBy("CurrentAddress.City.Name", false)
// Fail
db.Users.OrderBy("PhoneLines.RenewalDate", true)
My model is like so:
public class User {
public string Name { get; set; }
public Address CurrentAddress{ get; set; }
public List<Phone> PhoneLines { get; set; }
}
And I am trying to do something like this work:
db.Users.OrderByDescending(x =>
x.PhoneLines
.OrderByDescending(y => y.RenewalDate)
.Select(y => y.RenewalDate).FirstOrDefault()
);
Right now I am only interested in making this work for first level List children, that is, I am not concerned about a cascading list scenario, just one child in the chain is a list.
So far my attempts have failed. I am checking if the last property is a collection and if so i tried creating a lambda for the list and another lambda for the class' element target order property.
if (body.Type.Namespace.Equals("System.Collections.Generic"))
{
var mainType = body.Type.GenericTypeArguments.FirstOrDefault();
var parameterInner = Expression.Parameter(mainType, "x");
var mainProperty = Expression.Property(parameterInner, property.Name);
var orderByExpressionInner = Expression.Lambda(mainProperty, parameterInner);
var orderByExpression = Expression.Lambda(body, parameter);
var selM = typeof(Enumerable).GetMethods()
.Where(x => x.Name == command)
.First().MakeGenericMethod(mainType, property.PropertyType);
var resultExpressionOuter = Expression.Call(typeof(Queryable),
command, new Type[] { typeof(TEntity), body.Type },
source.Expression,
Expression.Call(null, selM, parameterInner, orderByExpressionInner
));
}
My knowledge with reflection is limited, is it possible to achieve this via reflection?

One option is to place a new property on your parent class and make this responsible for returning the value which you require. Such as this;
public class User
{
public string Name { get; set; }
public Address CurrentAddress { get; set; }
public List<Phone> PhoneLines { get; set; }
private DateTime _dummyValue;
public DateTime GetRenewalDate
{
get
{
if (this.PhoneLines == null || this.PhoneLines.Count == 0)
return DateTime.MinValue;
else
{
return this.PhoneLines.OrderByDescending(y => y.RenewalDate).Select(y => y.RenewalDate).FirstOrDefault();
}
}
set
{
_dummyValue = value;
}
}
}
Make sure you have default handling in case your list is empty or null, here I return MinValue of DateTime.
Edit:
To get around the other issue you have put in the comments
The specified type member 'GetRenewalDate' is not supported in LINQ to
Entities. Only initializers, entity members, and entity navigation
properties are supported.
You could try adding a dummy set, this might also work as a workaround (but is not elegant).
I have edited the code above to show this

How to create lambda expressions for every property of a class

I have a following problem. I have to iterate over all the properties of the class to configure some builder. A class has a lot of properties, so the code is cumbersome. It looks like this:
var b = builder<MyTypeWith1000Properties>
.WithProperty(x=>x.Property1)
.WithProperty(x=>x.Property2)
...
.WithProperty(x=>x.Property1000);
The code is repeated in many places for many differend types, not only MyTypeWith1000Properties. I was thinking about creating some extension, like this:
var b = builder<MyTypeWith1000Properties>
.WithAllProperties();
and then in the WithAllProperties I could iterate over type properties using Reflection, like this:
public static IDataExtractor<T> WithAllProperties(this IDataExtractor<T> extractor)
{
var properties = typeof(T).GetProperties();
foreach (var property in properties)
{
extractor = extractor.WithProperty(/*the problem is here/*);
}
return extractor;
}
How to convert the property variable in the loop to a corresponding expression
Expression<Func<TRow, TValue>> propertyExpression
as this is what WithProperty expects

Update - Setting the correct parameter value in Expression.Lambda>
You could try something like this
public static class BuilderExtension
{
public static IDataExtractor<T> WithAllProperties<T>(this IDataExtractor<T> extractor)
{
var properties = typeof(T).GetProperties();
foreach (var propertyInfo in properties)
{
var parameter = Expression.Parameter(typeof(T), "x");
var property = Expression.Property(parameter, propertyInfo);
var lambda = Expression.Lambda<Func<T, object>>(property,parameter);
extractor = extractor.WithProperty(lambda);
}
return extractor;
}
}
Suppose you have the following class structures
public class MyTypeWith100Properties
{
public string Property1 { get; set; }
public string Property2 { get; set; }
public string Property3 { get; set; }
public string Property4 { get; set; }
}
public interface IDataExtractor<T>
{
IDataExtractor<T> WithProperty(Expression<Func<T, object>> expr);
}
public class DataExtractor<T> : IDataExtractor<T>
{
public List<Expression<Func<T, object>>> Expressions { get; private set; }
public DataExtractor() {
Expressions = new List<Expression<Func<T, object>>>();
}
public IDataExtractor<T> WithProperty(Expression<Func<T, object>> expr)
{
Expressions.Add(expr);
return this;
}
}
Then if you run this
var builder = new DataExtractor<MyTypeWith100Properties>();
var b = builder.WithAllProperties<MyTypeWith100Properties>()
as DataExtractor<MyTypeWith100Properties>;
var instance = new MyTypeWith100Properties() {
Property1 = "This is property 1",
Property2 = "This is property 2",
Property3 = "This is property 3",
Property4 = "This is property 4"
};
foreach (var current in b.Expressions)
{
var compiled = current.Compile();
var result = compiled.Invoke(instance);
Console.WriteLine(result);
}
Your output will be
This is property 1
This is property 2
This is property 3
This is property 4

WithProperty is a generic method that takes a type parameter implied by the result type of the property member access expression, which is what TValue represents in its declaration. Since you want to use reflection to generate the lambdas, you have to do the WithProperty call dynamically as well so you can call the one with the proper type (e.g. WithProperty<String> for a String property).
Here is an extension method that generates a lambda that consists of all the chained WithProperty calls for all the properties in a class, and then compiles and calls the lambda on the IDataExtractor. I chained all the calls together and then compiled because there might be some overhead for compiling, so I didn't want to compile and call code for each property individually.
public static class IDataExtractorExt {
public static IDataExtractor<TRow> WithAllProperties<TRow>(this IDataExtractor<TRow> extractor) {
var p = Expression.Parameter(typeof(IDataExtractor<TRow>), "p"); // final lambda parameter
Expression ansBody = p; // start with p => p
var withPropGenericMI = typeof(IDataExtractor<TRow>).GetMethod("WithProperty"); // lookup WithProperty<> generic method
var properties = typeof(TRow).GetProperties();
foreach (var property in properties) {
var withPropMI = withPropGenericMI.MakeGenericMethod(property.PropertyType); // instantiate generic WithProperty<> to property type
var pxp = Expression.Parameter(typeof(TRow), "x"); // property accessor lambda parameter
var pxb = Expression.PropertyOrField(pxp, property.Name); // property accessor expression x.property
Expression propExpr = Expression.Lambda(pxb, pxp); // x => x.property
ansBody = Expression.Call(ansBody, withPropMI, propExpr); // build up p => p.WithProperty(x => x.property)...
}
return ((IDataExtractor<TRow>)(Expression.Lambda(ansBody, p).Compile().DynamicInvoke(extractor)));
}
}

select from a list with field name and field value with linq (similar to select command)

I want to select from a dynamic list where the list field name is for example 'SecName'and the SecName value is equal to for example 'xxx',
I know how to create it in sql, but I want to create and use it with entity framework.
Like Executing String in sql.
How can I make it in entity framework and linq.
It should be something like this
var lst=_efmodel.Tables.where(x=>x.fieldname=="SecName" && x.value=="xxx").tolist();
I'm looking for syntax to filter my list by passing two string, 1st string should be property Name and the Other should be that property value.

After searching I found the Answer ,Answer is:
public class SearchField
{
public string Name { get; set; }
public string #Value { get; set; }
//public string Operator { get; set; }
public SearchField(string Name, string #Value)
{
this.Name = Name;
this.#Value = #Value;
//Operator = "=";
}
}
public class FilterLinq<T>
{
public static Expression<Func<T, Boolean>> GetWherePredicate(params SearchField[] SearchFieldList)
{
//the 'IN' parameter for expression ie T=> condition
ParameterExpression pe = Expression.Parameter(typeof(T), typeof(T).Name);
//combine them with and 1=1 Like no expression
Expression combined = null;
if (SearchFieldList != null)
{
foreach (var fieldItem in SearchFieldList)
{
//Expression for accessing Fields name property
Expression columnNameProperty = Expression.Property(pe, fieldItem.Name);
//the name constant to match
Expression columnValue = Expression.Constant(fieldItem.Value);
//the first expression: PatientantLastName = ?
Expression e1 = Expression.Equal(columnNameProperty, columnValue);
if (combined == null)
{
combined = e1;
}
else
{
combined = Expression.And(combined, e1);
}
}
}
//create and return the predicate
if (combined != null) return Expression.Lambda<Func<T, Boolean>>(combined, pe);
return null;
}
}
and use it like this :
var lst = _efmodel.Count(2015).AsQueryable()
.Where(
FilterLinq<PazTedad_Result>.GetWherePredicate(
new SearchField("FieldName", "FieldValue"))).ToList();

Code must be something like this?
var lst=_efmodel.Table.Where(x => x.fieldname == <SomeValue>).ToList();

Generic method to filter a list object

I am trying to create a generic method that takes three parameters.
1) List collection
2) String PropertyName
3) String FilterString
The idea is we pass a collection of Objects, the name of the property of the object
and a Filter Criteria and it returns back a list of Objects where the property contains
the FilterString.
Also, the PropertyName is optional so if its not supplied I would like to return
all objects that contain the FilterString in any property.
Any pointers on this would be really helpful.
I am trying to have a method signature like this:
public static List FilterList(List collection, String FilterString, String Property = "")
This way I can call this method from anywhere and pass it any List and it would return me a filtered list.

You could do what you want using LINQ, as such,
var collection = ...
var filteredCollection =
collection.Where(item => item.Property == "something").ToList();
Otherwise, you could try Reflection,
public List<T> Filter<T>(
List<T> collection,
string property,
string filterValue)
{
var filteredCollection = new List<T>();
foreach (var item in collection)
{
// To check multiple properties use,
// item.GetType().GetProperties(BindingFlags.Public | BindingFlags.Instance)
var propertyInfo =
item.GetType()
.GetProperty(property, BindingFlags.Public | BindingFlags.Instance);
if (propertyInfo == null)
throw new NotSupportedException("property given does not exists");
var propertyValue = propertyInfo.GetValue(item, null);
if (propertyValue == filterValue)
filteredCollection.Add(item);
}
return filteredCollection;
}
The problem with this solution is that changes to the name of the property or misspellings result in a runtime error, rather than a compilation error as would be using an actual property expression where the name is hard-typed.
Also, do note that based on the binding flags, this will work only on public, non-static properties. You can modify such behavior by passing different flags.

You can use a combination of reflection and dynamic expressions for this. I've put together a sample that might look a bit long at the first look. However, it matches your requirements and addresses these by
Using reflection to find the properties that are of type string and match the property name - if provided.
Creating an expression that calls string.Contains on all properties that have been identified. If several properties have been identified, the calls to string.Contains are combined by Or-expressions. This filter expression is compiled and handed to the Where extension method as a parameter. The provided list is filtered using the expression.
Follow this link to run the sample.
using System;
using System.Collections.Generic;
using System.Reflection;
using System.Linq;
using System.Linq.Expressions;
public class Test
{
public static IEnumerable<T> SelectItems<T>(IEnumerable<T> items, string propName, string value)
{
IEnumerable<PropertyInfo> props;
if (!string.IsNullOrEmpty(propName))
props = new PropertyInfo[] { typeof(T).GetProperty(propName) };
else
props = typeof(T).GetProperties();
props = props.Where(x => x != null && x.PropertyType == typeof(string));
Expression lastExpr = null;
ParameterExpression paramExpr = Expression.Parameter(typeof(T), "x");
ConstantExpression valueExpr = Expression.Constant(value);
foreach(var prop in props)
{
var propExpr = GetPropertyExpression(prop, paramExpr, valueExpr);
if (lastExpr == null)
lastExpr = propExpr;
else
lastExpr = Expression.MakeBinary(ExpressionType.Or, lastExpr, propExpr);
}
if (lastExpr == null)
return new T[] {};
var filterExpr = Expression.Lambda(lastExpr, paramExpr);
return items.Where<T>((Func<T, bool>) filterExpr.Compile());
}
private static Expression GetPropertyExpression(PropertyInfo prop, ParameterExpression paramExpr, ConstantExpression valueExpr)
{
var memberAcc = Expression.MakeMemberAccess(paramExpr, prop);
var containsMember = typeof(string).GetMethod("Contains");
return Expression.Call(memberAcc, containsMember, valueExpr);
}
class TestClass
{
public string SomeProp { get; set; }
public string SomeOtherProp { get; set; }
}
public static void Main()
{
var data = new TestClass[] {
new TestClass() { SomeProp = "AAA", SomeOtherProp = "BBB" },
new TestClass() { SomeProp = "BBB", SomeOtherProp = "CCC" },
new TestClass() { SomeProp = "CCC", SomeOtherProp = "AAA" },
};
var result = SelectItems(data, "", "A");
foreach(var item in result)
Console.WriteLine(item.SomeProp);
}
}
In contrast to a completely reflection based approach, this one assembles the filter expression only once and compiles it, so that I'd expect a (small) performance improvement.

You should use Dynamic LINQ, for example, given SomeClass:
public class SomeClass
{
public int SomeField { get; set; }
}
List<SomeClass> list = new List<SomeClass>() { new SomeClass() { SomeField = 2 } };
and then:
var temp = list.AsQueryable().Where("SomeField == 1").Select("it");
var result= temp .Cast<SomeClass>().ToList();
So your function would be even simpler, with property name and filter merged into one parameter:
public List<T> Filter<T>(List<T> list, string filter)
{
var temp = list.AsQueryable().Where(filter).Select("it");
return temp.Cast<T>().ToList();
}
and you can provide different filters, like for example "SomeField > 4 && SomeField < 10" etc.

When using Markus solution it'll only work when all String properties are not null.
To ensure you could do this:
class TestClass
{
private string _someProp { get; set; }
public string SomeProp {
get
{
if(string.IsNullOrEmpty(_someProp)
{
_someProp = "";
}
return _someProp;
}
set
{
_someProp = value;
}
}
private string _someOtherProp { get; set; }
public string SomeOtherProp {
get
{
if(string.IsNullOrEmpty(_someOtherProp)
{
_someOtherProp = "";
}
return _someOtherProp;
}
set
{
_someOtherProp = value;
}
}
}
Since my rep is less then 50 I cannot comment;)