Develop Reference

Entity Framework dynamic linq where from generic source with dynamic where clause

I am trying to query Entity Framework where clause statement with generic type when table name is not known.
I am trying to use dynamic linq:
public static IEnumerable<T> wherePost<T>(this IEnumerable<T> source,
string predicate) where T : class
{
using (var context = new dbEntities())
{
string exp = predicate;//driverId>3
var p = Expression.Parameter(typeof(T));
var e = System.Linq.Dynamic.DynamicExpression.ParseLambda(new[] { p }, null, exp);
}
}
This is the call example to wherePost
db.Drivers.wherePost("City == \"Paris\"");
Not sure how to continue and query the DbSet<T> from Entity Framework using the DynamicExpression.

I assume you do not need to change TableName on the fly, but just query table via filter.
public static IQueryable<T> wherePost<T>(this IQueryable<T> source,
string predicate) where T : class
{
string exp = predicate;//driverId>3
var p = Expression.Parameter(typeof(T));
var e = System.Linq.Dynamic.DynamicExpression.ParseLambda(new[] { p }, null, exp);
var whereExpr = Expression.Call(typeof(Queryable), nameof(Queryable.Where), new[] { typeof(T) }, source.Expression, e);
return source.Provider.CreateQuery<T>(whereExpr);
}
And usage:
db.Drivers.wherePost("City == \"Paris\"").ToList();

OrderBy Expression Tree in Net Core Linq for Extension Method

I want to create an Extension method which mimics this, https://dejanstojanovic.net/aspnet/2019/january/filtering-and-paging-in-aspnet-core-web-api/
However, I want to add an OrderBy (for ColumnName) after StartsWith, how would I conduct this?
tried adding following and did not work .OrderBy(parameter)
Example:
return persons.Where(p => p.Name.StartsWith(filterModel.Term ?? String.Empty, StringComparison.InvariantCultureIgnoreCase))
.OrderBy(c=>c.Name)
.Skip((filterModel.Page-1) * filter.Limit)
.Take(filterModel.Limit);
public static class PaginateClass
{
static readonly MethodInfo startsWith = typeof(string).GetMethod("StartsWith", new[] { typeof(string), typeof(System.StringComparison) });
public static IEnumerable<T> Paginate<T>(this IEnumerable<T> input, PageModel pageModel, string columnName) where T : class
{
var type = typeof(T);
var propertyInfo = type.GetProperty(columnName);
//T p =>
var parameter = Expression.Parameter(type, "p");
//T p => p.ColumnName
var name = Expression.Property(parameter, propertyInfo);
// filterModel.Term ?? String.Empty
var term = Expression.Constant(pageModel.Term ?? String.Empty);
//StringComparison.InvariantCultureIgnoreCase
var comparison = Expression.Constant(StringComparison.InvariantCultureIgnoreCase);
//T p => p.ColumnName.StartsWith(filterModel.Term ?? String.Empty, StringComparison.InvariantCultureIgnoreCase)
var methodCall = Expression.Call(name, startsWith, term, comparison);
var lambda = Expression.Lambda<Func<T, bool>>(methodCall, parameter);
return input.Where(lambda.Compile()) //tried adding this and did not work .OrderBy(parameter)
.Skip((pageModel.Page - 1) * pageModel.Limit)
.Take(pageModel.Limit);
}
Other items PageModel:
public class PageModel
{
public int Page { get; set; }
public int Limit { get; set; }
public string Term { get; set; }
public PageModel()
{
this.Page = 1;
this.Limit = 3;
}
public object Clone()
{
var jsonString = JsonConvert.SerializeObject(this);
return JsonConvert.DeserializeObject(jsonString, this.GetType());
}
}
Dynamic Linq to Entities Orderby with Pagination

Check the sample code for the solution:
void Main()
{
var queryableRecords = Product.FetchQueryableProducts();
Expression expression = queryableRecords.OrderBy("Name");
var func = Expression.Lambda<Func<IQueryable<Product>>>(expression)
.Compile();
func().Dump();
}
public class Product
{
public int Id { get; set; }
public string Name { get; set; }
public static IQueryable<Product> FetchQueryableProducts()
{
List<Product> productList = new List<Product>()
{
new Product {Id=1, Name = "A"},
new Product {Id=1, Name = "B"},
new Product {Id=1, Name = "A"},
new Product {Id=2, Name = "C"},
new Product {Id=2, Name = "B"},
new Product {Id=2, Name = "C"},
};
return productList.AsQueryable();
}
}
public static class ExpressionTreesExtesion
{
public static Expression OrderBy(this IQueryable queryable, string propertyName)
{
var propInfo = queryable.ElementType.GetProperty(propertyName);
var collectionType = queryable.ElementType;
var parameterExpression = Expression.Parameter(collectionType, "g");
var propertyAccess = Expression.MakeMemberAccess(parameterExpression, propInfo);
var orderLambda = Expression.Lambda(propertyAccess, parameterExpression);
return Expression.Call(typeof(Queryable),
"OrderBy",
new Type[] { collectionType, propInfo.PropertyType },
queryable.Expression,
Expression.Quote(orderLambda));
}
}
Result
How it Works:
Created an expression using extension method on the Queryable type, which internally calls OrderBy method of the Queryable type, expecting IQueryable to be the Input, along with the field name and thus runs the ordering function and Ordered collection is the final Output
Option 2:
This may fit your use case better, here instead of calling OrderBy method, we are creating the Expression<Func<T,string>> as an extension method to the IEnumerable<T>, which can then be compiled and supplied to the OrderBy Call, as shown in the example and is thus much more intuitive and simple solution:
Creating Expression:
public static class ExpressionTreesExtesion
{
public static Expression<Func<T,string>> OrderByExpression<T>(this IEnumerable<T> enumerable, string propertyName)
{
var propInfo = typeof(T).GetProperty(propertyName);
var collectionType = typeof(T);
var parameterExpression = Expression.Parameter(collectionType, "x");
var propertyAccess = Expression.MakeMemberAccess(parameterExpression, propInfo);
var orderExpression = Expression.Lambda<Func<T,string>>(propertyAccess, parameterExpression);
return orderExpression;
}
}
How to Call:
var ProductExpression = records.OrderByExpression("Name");
var result = records.OrderBy(ProductExpression.Compile());
ProductExpression.Compile() above will compile into x => x.Name, where column name is supplied at the run-time
Please note in case the ordering field can be other types beside string data type, then make that also generic and supply it while calling extension method, only condition being property being called shall have the same type as supplied value, else it will be a run-time exception, while creating Expression
Edit 1, how to make the OrderType field also generic
public static Expression<Func<T, TField>> OrderByFunc<T,TField>(this IEnumerable<T> enumerable, string propertyName)
{
var propInfo = typeof(T).GetProperty(propertyName);
var collectionType = typeof(T);
var parameterExpression = Expression.Parameter(collectionType, "x");
var propertyAccess = Expression.MakeMemberAccess(parameterExpression, propInfo);
var orderExpression = Expression.Lambda<Func<T, TField>>(propertyAccess, parameterExpression);
return orderExpression;
}
How to call:
Now both the types need to be explicitly supplied, earlier were using generic type inference from IEnumerable<T>:
// For Integer Id field
var ProductExpression = records.OrderByFunc<Product,int>("Id");
// For string name field
var ProductExpression = records.OrderByFunc<Product,string>("Name");

Store Static Filter By Key Expression

I've got an function which generates an expression to filter a table by it's primary key, when passed in an Object[], this is very similar to Find function except that it doesn't materialize so you can pass an IQueryable around afterwards
public static Expression<Func<T, bool>> FilterByPrimaryKeyPredicate<T>(this DbContext dbContext, object[] id)
{
var keyProperties = dbContext.GetPrimaryKeyProperties<T>();
var parameter = Expression.Parameter(typeof(T), "e");
var body = keyProperties
// e => e.{propertyName} == new {id = id[i]}.id
.Select((p, i) => Expression.Equal(
Expression.Property(parameter, p.Name),
Expression.Convert(
Expression.PropertyOrField(Expression.Constant(new { id = id[i] }), "id"),
p.ClrType)))
.Aggregate(Expression.AndAlso);
return Expression.Lambda<Func<T, bool>>(body, parameter);
}
This works by first getting the primary keys for a table, it creates binary expression foreach property, the Id is wrapped in an anonymous type to leverage the query cache. This is working fine. However, I'd like to take this a step further.
I'd like to preserve the Expression so I don't have to generate it each time I pass on a new set of ids, How can I store this Expression while still leveraging the Query Cache?
Edit TL;DR
So I'm attempt to cache it using array access in a static class as suggest, however I'm encountering an error:
public class PrimaryKeyFilterContainer<T>
{
const string ANON_ID_PROP = "id";
static Expression<Func<T, bool>> _filter;
Type ANON_TYPE = new { id = (object)0 }.GetType();
public object[] id { get; set; }
public PrimaryKeyFilterContainer()
{
}
public Expression<Func<T, bool>> GetFilter(DbContext dbContext, object[] id)
{
this.id = id;
if(null == _filter)
{
var keyProperties = dbContext.GetPrimaryKeyProperties<T>();
var parameter = Expression.Parameter(typeof(T), "e");
var body = keyProperties
// e => e.PK[i] == id[i]
.Select((p, i) => Expression.Equal(
Expression.Property(parameter, p.Name),
Expression.Convert(BuildNewExpression(i),
p.ClrType)))
.Aggregate(Expression.AndAlso);
_filter = Expression.Lambda<Func<T, bool>>(body, parameter);
}
return _filter;
}
NewExpression BuildNewExpression(int index)
{
var currentObject = Expression.Constant(this);
var fieldAccess = Expression.PropertyOrField(currentObject, nameof(id));
var arrayAccess = Expression.ArrayAccess(fieldAccess, Expression.Constant(index));
return Expression.New(ANON_TYPE.GetConstructor(new[] { typeof(object) }), arrayAccess);
}
}
No coercion operator is defined between types '<>f__AnonymousType0`1[System.Object]' and 'System.Int32'
I'm getting closer but I'm not sure if it's going to work still.

As I mentioned in the comments, the main problem is that we cannot use array index access inside the expression tree - EF6 throws not supported exception and EF Core turns it into client evaluation.
So we need to store the keys in a class with dynamic count of properties and property types. Fortunately the System.Tuple generic classes provide such functionality, and can be used in both EF6 and EF Core.
Following is a class that implements the above idea:
public class PrimaryKeyFilter<TEntity>
where TEntity : class
{
object valueBuffer;
Func<object[], object> valueArrayConverter;
public PrimaryKeyFilter(DbContext dbContext)
{
var keyProperties = dbContext.GetPrimaryKeyProperties<TEntity>();
// Create value buffer type (Tuple) from key properties
var valueBufferType = TupleTypes[keyProperties.Count - 1]
.MakeGenericType(keyProperties.Select(p => p.ClrType).ToArray());
// Build the delegate for converting value array to value buffer
{
// object[] values => new Tuple(values[0], values[1], ...)
var parameter = Expression.Parameter(typeof(object[]), "values");
var body = Expression.New(
valueBufferType.GetConstructors().Single(),
keyProperties.Select((p, i) => Expression.Convert(
Expression.ArrayIndex(parameter, Expression.Constant(i)),
p.ClrType)));
valueArrayConverter = Expression.Lambda<Func<object[], object>>(body, parameter).Compile();
}
// Build the predicate expression
{
var parameter = Expression.Parameter(typeof(TEntity), "e");
var valueBuffer = Expression.Convert(
Expression.Field(Expression.Constant(this), nameof(this.valueBuffer)),
valueBufferType);
var body = keyProperties
// e => e.{propertyName} == valueBuffer.Item{i + 1}
.Select((p, i) => Expression.Equal(
Expression.Property(parameter, p.Name),
Expression.Property(valueBuffer, $"Item{i + 1}")))
.Aggregate(Expression.AndAlso);
Predicate = Expression.Lambda<Func<TEntity, bool>>(body, parameter);
}
}
public Expression<Func<TEntity, bool>> Predicate { get; }
public void SetValues(params object[] values) =>
valueBuffer = valueArrayConverter(values);
static readonly Type[] TupleTypes =
{
typeof(Tuple<>),
typeof(Tuple<,>),
typeof(Tuple<,,>),
typeof(Tuple<,,,>),
typeof(Tuple<,,,,>),
typeof(Tuple<,,,,,>),
typeof(Tuple<,,,,,,>),
typeof(Tuple<,,,,,,,>),
};
}
You can create and store an instance of the class. Then use the expression returned by the Predicate property inside the query. And SetValues method to set the parameters.
The drawback is that the value storage is bound to the class instance, hence it cannot be used concurrently. The original approach works well in all scenarios, and the performance impact IMO should be negligible, so you might consider staying on it.

Select distinct column from Entity Framework with dynamic column name

I'm searching a way to get a list of distinct values, for one column of my table. I need to make a reusable method.
This is what I tried so far, but it doesn't work:
IEnumerable<string> GetDistinctValues<T>(string columnName)
{
T.Select(m => m.ColumnName).Distinct().ToList();
}
The desired solution should be an extension method for EF objects.
I've tried this post Dynamically select columns in runtime using entity framework but it works only for a single record not for a list.

The only problem I see with Linq.Dynamic is that there were no updates since 2013 and the project is pretty much dead
I would handle it via extensions, and improve reflection performance via caching (not elaborated here)
Extensions:
public static class QueryableExtensions
{
public static IReadOnlyCollection<TResult> GetDistinctValuesForProperty<T, TResult>(this IQueryable<T> query, Expression<Func<T, TResult>> propertyAccess)
{
return SelectDistinct(query, propertyAccess).ToList();
}
public static IReadOnlyCollection<object> GetDistinctValuesForProperty<TSource>(this IQueryable<TSource> query, string propertyName)
{
var unboundFuncType = typeof(Func<,>);
var unboundExprType = typeof(Expression<>);
var sourceType = typeof(TSource); // TSource
var resultType = typeof(TSource)
.GetProperty(propertyName)
.PropertyType; // TResult
// Func<TSource, TResult>
var funcType = unboundFuncType.MakeGenericType(new [] { sourceType, resultType });
// Expression<Func<TSource, TResult>>
var expressionType = unboundExprType.MakeGenericType(new [] { funcType });
// Instance of Expression<Func<TSource, TResult>>, for example x => x.Name
var propertyAccess = typeof(StringExtensions)
.GetMethod(nameof(StringExtensions.AsPropertyExpression), new[] { typeof(string) })
.MakeGenericMethod(new [] { sourceType, resultType })
.Invoke(null, new object[] { propertyName });
// SelectDistinct query transform
var selectDistinctMethod = typeof(QueryableExtensions)
.GetMethod(nameof(QueryableExtensions.SelectDistinct), BindingFlags.NonPublic | BindingFlags.Static)
.MakeGenericMethod(new [] { sourceType, resultType });
// IQueryable<TSource> ==> IQueryable<TResult>
var result = selectDistinctMethod.Invoke(null, new object[] { query, propertyAccess });
// Cast to object via IEnumerable and convert to list
return ((IEnumerable)result).Cast<object>().ToList();
}
private static IQueryable<TResult> SelectDistinct<TSource, TResult>(this IQueryable<TSource> query, Expression<Func<TSource, TResult>> propertyAccess)
{
return query.Select(propertyAccess).Distinct();
}
}
public static class StringExtensions
{
public static Expression<Func<T, TResult>> AsPropertyExpression<T, TResult>(this string propertyName)
{
var parameter = Expression.Parameter(typeof(T), "x");
var property = typeof(T).GetProperty(propertyName);
var body = Expression.MakeMemberAccess(parameter, property);
return Expression.Lambda<Func<T, TResult>>(body, parameter);
}
}
Usage:
public class Person
{
public string Name { get; }
public int Age { get; }
public Person(string name, int age)
{
Name = name;
Age = age;
}
}
var people = new Person[]
{
new Person("John", 25), new Person("Peter", 25), new Person("Sean", 25),
new Person("John", 32), new Person("Peter", 32),
};
var query = people.AsQueryable();
var namePropertyExpression = "Name".AsPropertyExpression<Person, string>();
var agePropertyExpression = "Age".AsPropertyExpression<Person, int>();
// When you know the result type
var names1 = query.GetDistinctValuesForProperty(x => x.Name);
var ages1 = query.GetDistinctValuesForProperty(x => x.Age);
// When you know the result type, but you may want to reuse the property expression
var names2 = query.GetDistinctValuesForProperty(namePropertyExpression);
var ages2 = query.GetDistinctValuesForProperty(agePropertyExpression);
// When you just know the property name
var names3 = query.GetDistinctValuesForProperty("Name");
var ages3 = query.GetDistinctValuesForProperty("Age");

Finally I found a solution. I need to include reference to System.Linq.Dynamic (downloaded by nuget), and use the "Select" method that accept String to reference column.
using System.Linq.Dynamic;
public static async Task<IEnumerable<Object>> GetDistinctValuesForProperty<T>(this IQueryable<T> query, String PropertyName)
{
return await query.Select(PropertyName).Distinct().ToListAsync();
}
and call as
String ColumnName = "DateTimeInsert";
DbSet<Log> oDbSet = _uow.DbContext.Set<Log>();
Array DistinctValues;
if (typeof(Log).GetProperty(ColumnName) != null)
{
DistinctValues = (await oDbSet.GetDistinctValuesForProperty(ColumnName)).ToArray();
}
else
{
DistinctValues = new object[0];
}
I need to use array vs ienumerable due to a cast problem in case of datetime types

You can create a generic selector method using Expressions
public static Func<T, T> SelectorFunc<T>(string[] columns) {
// input parameter "o"
var xParameter = Expression.Parameter(typeof(T), "o");
// new statement "new Data()"
var xNew = Expression.New(typeof(T));
// create initializers
var bindings = columns.Select(o => o.Trim())
.Select(o =>
{
// property "Field1"
var mi = typeof(T).GetProperty(o);
// original value "o.Field1"
var xOriginal = Expression.Property(xParameter, mi);
// set value "Field1 = o.Field1"
return Expression.Bind(mi, xOriginal);
}
);
// initialization "new Data { Field1 = o.Field1, Field2 = o.Field2 }"
var xInit = Expression.MemberInit(xNew, bindings);
// expression "o => new Data { Field1 = o.Field1, Field2 = o.Field2 }"
var lambda = Expression.Lambda<Func<T, T>>(xInit, xParameter);
// compile to Func<Data, Data>
return lambda.Compile();
}
using it would be
T.Select( SelectorFunc<T>( new string[]{ "Column" } ) ).Distinct().ToList();
You can also use it any other linq functions like
T.Select( SelectorFunc<T>( new string[]{ "Column" } ) ).Where();
T.Select( SelectorFunc<T>( new string[]{ "Column" } ) ).AsQueryable();
for additional reference you can see the full OP here
LINQ : Dynamic select

Dynamic linq expression tree with nested properties

I have a list which I must filter on child properties. The filter operator is dynamic and I'm using a predicate builder in order to combine several filters/lambdas.
For simplicity, let's say that I have two classes like this:
public class FirstClass
{
public int Id { get; set; }
public ICollection<SecondClass> MyList { get; set; }
}
public class SecondClass
{
public int ReferenceId { get; set; }
public int Value { get; set; }
}
My filter use a reference id, an operator type and a value, such that a pseudo code would be like this:
"list of FirstClass".Where(param1 =>
param1.MyList.Single(param2 =>
param2.ReferenceId == "reference id").Value "operatorType" "value")
The actual filter will be something like 123 eq 456, where the reference id is 123, operatorType is "eq" and value is 456.
If the operator just was equality, then the following works just fine:
Expression<Func<FirstClass, bool>> lambda =
param1 => param1.MyList.Single(param2 => param2.ReferenceId == id).Value == value;
Also, filtering only on FirstClass with dynamic expressions, works like a charm, e.g. filtering on Id (my ExpressionTypeDictionary is a dictionary for selecting an ExpressionType based on the provided operatorType):
var parameter = Expression.Parameter(typeof(FirstClass), "param1");
Expression body = parameter;
body = Expression.Property(body, "Id");
body = Expression.MakeBinary(ExpressionTypeDictionary[operatorType], body, value);
var lambda = Expression.Lambda<Func<FirstClass, bool>>(body, new[] { parameter });
I'm able to get the following to compile, but executing the filter on real data using EF Core returns an exception for querySource:
var parameter = Expression.Parameter(typeof(FirstClass), "param1");
Expression<Func<FirstClass, int>> left = param1 =>
param1.MyClass.Single(param2 => param2.ReferenceId == id).Value;
var body = Expression.MakeBinary(
ExpressionTypeDictionary[operatorType],
left.Body,
Expression.Constant(value));
var lambda = Expression.Lambda<Func<FirstClass, bool>>(body, new[] { parameter });
...
theList.Where(lambda);
Any suggestions are appreciated :)

I think rather than expression like this
Expression<Func<FirstClass, bool>> predicate =
x => x.MyList.Single(y => y.ReferenceId == id).Value [operator] value;
you'd better build an expression like this:
Expression<Func<FirstClass, bool>> predicate =
x => x.MyList.Any(y => y.ReferenceId == id && y.Value == value);
Here is how you can do that:
var innerParameter = Expression.Parameter(typeof(SecondClass), "y");
var innerPredicate = Expression.Lambda<Func<SecondClass, bool>>(
Expression.AndAlso(
Expression.Equal(Expression.Property(innerParameter, "ReferenceId"), Expression.Constant(id)),
Expression.MakeBinary(ExpressionTypeDictionary[operatorType], Expression.Property(innerParameter, "Value"), Expression.Constant(value))),
innerParameter);
var parameter = Expression.Parameter(typeof(FirstClass), "x");
var predicate = Expression.Lambda<Func<FirstClass, bool>>(
Expression.Call(
typeof(Enumerable), "Any", new Type[] { typeof(SecondClass) },
Expression.Property(parameter, "MyList"), innerPredicate),
parameter);