Using Expression trees, I would need to build a GroupBy in a generic way.
The static method I'm going to use is the following:
public static IQueryable<Result> GroupBySelector<TSource>(this IQueryable<TSource> source, String coloumn)
{
//Code here
}
The Result class has two property :
public string Value { get; set; }
public int Count { get; set; }
Basically I'd like to build the following Linq query via Expression trees:
query.GroupBy(s => s.Country).Select(p => new
{
Value = p.Key,
Count = p.Count()
}
)
How would you implement it?
Looking at:
query.GroupBy(s => s.Country).Select(p => new
{
Value = p.Key,
Count = p.Count()
}
);
To match the signature of IQueryable<Result> what you actually need here is:
query.GroupBy(s => s.Country).Select(p => new
Result{
Value = p.Key,
Count = p.Count()
}
);
Now, the Select can work with any IQueryable<IGrouping<string, TSource>> as is. It's only the GroupBy that needs us to use expression trees.
Our task here is to start with a type and a string that represents a property (that itself returns string) and create a Expression<Func<TSource, string>> that represents obtaining the value of that property.
So, let's produce the simple bit of the method first:
public static IQueryable<Result> GroupBySelector<TSource>(this IQueryable<TSource> source, string column)
{
Expression<Func<TSource, string>> keySelector = //Build tree here.
return source.GroupBy(keySelector).Select(p => new Result{Value = p.Key, Count = p.Count()});
}
Okay. How to build the tree.
We're going to need a lambda that has a paramter of type TSource:
var param = Expression.Parameter(typeof(TSource));
We're going to need to obtain the property whose name matches column:
Expression.Property(param, column);
And the only logic needed in the lambda is simply to access that property:
Expression<Func<TSource, string>> keySelector = Expression.Lambda<Func<TSource, string>>
(
Expression.Property(param, column),
param
);
Putting it all together:
public static IQueryable<Result> GroupBySelector<TSource>(this IQueryable<TSource> source, String column)
{
var param = Expression.Parameter(typeof(TSource));
Expression<Func<TSource, string>> keySelector = Expression.Lambda<Func<TSource, string>>
(
Expression.Property(param, column),
param
);
return source.GroupBy(keySelector).Select(p => new Result{Value = p.Key, Count = p.Count()});
}
About the only thing left is the exception-handling, which I normally don't include in an answer, but one part of this is worth paying attention to.
First the obvious null and empty checks:
public static IQueryable<Result> GroupBySelector<TSource>(this IQueryable<TSource> source, String column)
{
if (source == null) throw new ArgumentNullException("source");
if (column == null) throw new ArgumentNullException("column");
if (column.Length == 0) throw new ArgumentException("column");
var param = Expression.Parameter(typeof(TSource));
Expression<Func<TSource, string>> keySelector = Expression.Lambda<Func<TSource, string>>
(
Expression.Property(param, column),
param
);
return source.GroupBy(keySelector).Select(p => new Result{Value = p.Key, Count = p.Count()});
}
Now, let's consider what happens if we pass a string for column that doesn't match a property of TSource. We get an ArgumentException with the message Instance property '[Whatever you asked for]' is not defined for type '[Whatever the type is]'. That's pretty much what we want in this case, so no issue.
If however we passed a string that did identify a property but where that property wasn't of type string we'd get something like "Expression of type 'System.Int32' cannot be used for return type 'System.String'". That's not dreadful, but it's not great either. Let's be more explicit:
public static IQueryable<Result> GroupBySelector<TSource>(this IQueryable<TSource> source, String column)
{
if (source == null) throw new ArgumentNullException("source");
if (column == null) throw new ArgumentNullException("column");
if (column.Length == 0) throw new ArgumentException("column");
var param = Expression.Parameter(typeof(TSource));
var prop = Expression.Property(param, column);
if (prop.Type != typeof(string)) throw new ArgumentException("'" + column + "' identifies a property of type '" + prop.Type + "', not a string property.", "column");
Expression<Func<TSource, string>> keySelector = Expression.Lambda<Func<TSource, string>>
(
prop,
param
);
return source.GroupBy(keySelector).Select(p => new Result{Value = p.Key, Count = p.Count()});
}
If this method was internal the above would perhaps be over-kill, but if it was public the extra info would be well worth it if you came to debug it.
Related
I have a table in the frontend with multiple columns and pagination. The user should be able to sort which columns he wants, and if he moves to the next page, that page should also be sorted by the given column.
My approach with a single sort key and pagination:
public List<Person> GetPersons(int page, int numPerPage)
{
return _context.Person
.OrderByDescending(p => p.FirstName)
.Skip(page*numPerPage)
.Take(numPerPage)
}
This is limited to only sorting by FirstName, but how can I do this with e.g. LastName, Address etc...?
OrderByDescending takes a function with 2 type parameters: Expression<Func<TSource, TKey>> where TKey represents the type of the key returned by the expression.
Therefore, you could pass an expression to your method and use in the OrderBy:
public List<Person> GetPersons<TKey>(int page, int numPerPage,
Expression<Func<Person, TKey>> orderByDesc)
{
return _context.Person
.OrderByDescending(orderByDesc)
.Skip(page*numPerPage)
.Take(numPerPage);
}
The TKey type parameter is inferred by the expression. So, it can be used as below:
var byName = GetPersons(page, pageSize, p => p.FirstName);
var byDate = GetPersons(page, pageSize, p => p.DateOfBirth);
var byHeight = GetPersons(page, pageSize, p => p.Height);
If you want to apply a filter/search then add a new expression to the method and use Where:
public List<Person> GetPersons<TKey>(int page, int numPerPage,
Expression<Func<Person, bool>> filter,
Expression<Func<Person, TKey>> orderByDesc)
{
return _context.Person
.Where(filter)
.OrderByDescending(orderByDesc)
.Skip(page*numPerPage)
.Take(numPerPage);
}
// filter by FirstName, order by LastName
var filtered = GetPersons(page, pageSize,
p => p.FirstName == "Bob", // filter
p => p.LastName); // orderBy
Try this extension method:
public static IQueryable<T> OrderByField<T>(this IQueryable<T> q, string SortField, bool Ascending)
{
var param = Expression.Parameter(typeof(T), "p");
var prop = Expression.Property(param, SortField);
var exp = Expression.Lambda(prop, param);
string method = Ascending ? "OrderBy" : "OrderByDescending";
Type[] types = new Type[] { q.ElementType, exp.Body.Type };
var mce = Expression.Call(typeof(Queryable), method, types, q.Expression, exp);
return q.Provider.CreateQuery<T>(mce);
}
And you can pass property name as a string like FirstName, LastName or Address etc.. Thus you can order Persons dynamically.
public List<Person> GetPersons(int page, int numPerPage, string orderFieldName)
{
return _context.Person
.OrderByField(orderFieldName, false)
.Skip(page*numPerPage)
.Take(numPerPage)
}
See: How to use expression trees to build dynamic queries
EDIT:
Another possible solution is using System.Linq.Dynamic see: https://stackoverflow.com/a/8660293/5519709
I've got an function which generates an expression to filter a table by it's primary key, when passed in an Object[], this is very similar to Find function except that it doesn't materialize so you can pass an IQueryable around afterwards
public static Expression<Func<T, bool>> FilterByPrimaryKeyPredicate<T>(this DbContext dbContext, object[] id)
{
var keyProperties = dbContext.GetPrimaryKeyProperties<T>();
var parameter = Expression.Parameter(typeof(T), "e");
var body = keyProperties
// e => e.{propertyName} == new {id = id[i]}.id
.Select((p, i) => Expression.Equal(
Expression.Property(parameter, p.Name),
Expression.Convert(
Expression.PropertyOrField(Expression.Constant(new { id = id[i] }), "id"),
p.ClrType)))
.Aggregate(Expression.AndAlso);
return Expression.Lambda<Func<T, bool>>(body, parameter);
}
This works by first getting the primary keys for a table, it creates binary expression foreach property, the Id is wrapped in an anonymous type to leverage the query cache. This is working fine. However, I'd like to take this a step further.
I'd like to preserve the Expression so I don't have to generate it each time I pass on a new set of ids, How can I store this Expression while still leveraging the Query Cache?
Edit TL;DR
So I'm attempt to cache it using array access in a static class as suggest, however I'm encountering an error:
public class PrimaryKeyFilterContainer<T>
{
const string ANON_ID_PROP = "id";
static Expression<Func<T, bool>> _filter;
Type ANON_TYPE = new { id = (object)0 }.GetType();
public object[] id { get; set; }
public PrimaryKeyFilterContainer()
{
}
public Expression<Func<T, bool>> GetFilter(DbContext dbContext, object[] id)
{
this.id = id;
if(null == _filter)
{
var keyProperties = dbContext.GetPrimaryKeyProperties<T>();
var parameter = Expression.Parameter(typeof(T), "e");
var body = keyProperties
// e => e.PK[i] == id[i]
.Select((p, i) => Expression.Equal(
Expression.Property(parameter, p.Name),
Expression.Convert(BuildNewExpression(i),
p.ClrType)))
.Aggregate(Expression.AndAlso);
_filter = Expression.Lambda<Func<T, bool>>(body, parameter);
}
return _filter;
}
NewExpression BuildNewExpression(int index)
{
var currentObject = Expression.Constant(this);
var fieldAccess = Expression.PropertyOrField(currentObject, nameof(id));
var arrayAccess = Expression.ArrayAccess(fieldAccess, Expression.Constant(index));
return Expression.New(ANON_TYPE.GetConstructor(new[] { typeof(object) }), arrayAccess);
}
}
No coercion operator is defined between types '<>f__AnonymousType0`1[System.Object]' and 'System.Int32'
I'm getting closer but I'm not sure if it's going to work still.
As I mentioned in the comments, the main problem is that we cannot use array index access inside the expression tree - EF6 throws not supported exception and EF Core turns it into client evaluation.
So we need to store the keys in a class with dynamic count of properties and property types. Fortunately the System.Tuple generic classes provide such functionality, and can be used in both EF6 and EF Core.
Following is a class that implements the above idea:
public class PrimaryKeyFilter<TEntity>
where TEntity : class
{
object valueBuffer;
Func<object[], object> valueArrayConverter;
public PrimaryKeyFilter(DbContext dbContext)
{
var keyProperties = dbContext.GetPrimaryKeyProperties<TEntity>();
// Create value buffer type (Tuple) from key properties
var valueBufferType = TupleTypes[keyProperties.Count - 1]
.MakeGenericType(keyProperties.Select(p => p.ClrType).ToArray());
// Build the delegate for converting value array to value buffer
{
// object[] values => new Tuple(values[0], values[1], ...)
var parameter = Expression.Parameter(typeof(object[]), "values");
var body = Expression.New(
valueBufferType.GetConstructors().Single(),
keyProperties.Select((p, i) => Expression.Convert(
Expression.ArrayIndex(parameter, Expression.Constant(i)),
p.ClrType)));
valueArrayConverter = Expression.Lambda<Func<object[], object>>(body, parameter).Compile();
}
// Build the predicate expression
{
var parameter = Expression.Parameter(typeof(TEntity), "e");
var valueBuffer = Expression.Convert(
Expression.Field(Expression.Constant(this), nameof(this.valueBuffer)),
valueBufferType);
var body = keyProperties
// e => e.{propertyName} == valueBuffer.Item{i + 1}
.Select((p, i) => Expression.Equal(
Expression.Property(parameter, p.Name),
Expression.Property(valueBuffer, $"Item{i + 1}")))
.Aggregate(Expression.AndAlso);
Predicate = Expression.Lambda<Func<TEntity, bool>>(body, parameter);
}
}
public Expression<Func<TEntity, bool>> Predicate { get; }
public void SetValues(params object[] values) =>
valueBuffer = valueArrayConverter(values);
static readonly Type[] TupleTypes =
{
typeof(Tuple<>),
typeof(Tuple<,>),
typeof(Tuple<,,>),
typeof(Tuple<,,,>),
typeof(Tuple<,,,,>),
typeof(Tuple<,,,,,>),
typeof(Tuple<,,,,,,>),
typeof(Tuple<,,,,,,,>),
};
}
You can create and store an instance of the class. Then use the expression returned by the Predicate property inside the query. And SetValues method to set the parameters.
The drawback is that the value storage is bound to the class instance, hence it cannot be used concurrently. The original approach works well in all scenarios, and the performance impact IMO should be negligible, so you might consider staying on it.
I'm searching a way to get a list of distinct values, for one column of my table. I need to make a reusable method.
This is what I tried so far, but it doesn't work:
IEnumerable<string> GetDistinctValues<T>(string columnName)
{
T.Select(m => m.ColumnName).Distinct().ToList();
}
The desired solution should be an extension method for EF objects.
I've tried this post Dynamically select columns in runtime using entity framework but it works only for a single record not for a list.
The only problem I see with Linq.Dynamic is that there were no updates since 2013 and the project is pretty much dead
I would handle it via extensions, and improve reflection performance via caching (not elaborated here)
Extensions:
public static class QueryableExtensions
{
public static IReadOnlyCollection<TResult> GetDistinctValuesForProperty<T, TResult>(this IQueryable<T> query, Expression<Func<T, TResult>> propertyAccess)
{
return SelectDistinct(query, propertyAccess).ToList();
}
public static IReadOnlyCollection<object> GetDistinctValuesForProperty<TSource>(this IQueryable<TSource> query, string propertyName)
{
var unboundFuncType = typeof(Func<,>);
var unboundExprType = typeof(Expression<>);
var sourceType = typeof(TSource); // TSource
var resultType = typeof(TSource)
.GetProperty(propertyName)
.PropertyType; // TResult
// Func<TSource, TResult>
var funcType = unboundFuncType.MakeGenericType(new [] { sourceType, resultType });
// Expression<Func<TSource, TResult>>
var expressionType = unboundExprType.MakeGenericType(new [] { funcType });
// Instance of Expression<Func<TSource, TResult>>, for example x => x.Name
var propertyAccess = typeof(StringExtensions)
.GetMethod(nameof(StringExtensions.AsPropertyExpression), new[] { typeof(string) })
.MakeGenericMethod(new [] { sourceType, resultType })
.Invoke(null, new object[] { propertyName });
// SelectDistinct query transform
var selectDistinctMethod = typeof(QueryableExtensions)
.GetMethod(nameof(QueryableExtensions.SelectDistinct), BindingFlags.NonPublic | BindingFlags.Static)
.MakeGenericMethod(new [] { sourceType, resultType });
// IQueryable<TSource> ==> IQueryable<TResult>
var result = selectDistinctMethod.Invoke(null, new object[] { query, propertyAccess });
// Cast to object via IEnumerable and convert to list
return ((IEnumerable)result).Cast<object>().ToList();
}
private static IQueryable<TResult> SelectDistinct<TSource, TResult>(this IQueryable<TSource> query, Expression<Func<TSource, TResult>> propertyAccess)
{
return query.Select(propertyAccess).Distinct();
}
}
public static class StringExtensions
{
public static Expression<Func<T, TResult>> AsPropertyExpression<T, TResult>(this string propertyName)
{
var parameter = Expression.Parameter(typeof(T), "x");
var property = typeof(T).GetProperty(propertyName);
var body = Expression.MakeMemberAccess(parameter, property);
return Expression.Lambda<Func<T, TResult>>(body, parameter);
}
}
Usage:
public class Person
{
public string Name { get; }
public int Age { get; }
public Person(string name, int age)
{
Name = name;
Age = age;
}
}
var people = new Person[]
{
new Person("John", 25), new Person("Peter", 25), new Person("Sean", 25),
new Person("John", 32), new Person("Peter", 32),
};
var query = people.AsQueryable();
var namePropertyExpression = "Name".AsPropertyExpression<Person, string>();
var agePropertyExpression = "Age".AsPropertyExpression<Person, int>();
// When you know the result type
var names1 = query.GetDistinctValuesForProperty(x => x.Name);
var ages1 = query.GetDistinctValuesForProperty(x => x.Age);
// When you know the result type, but you may want to reuse the property expression
var names2 = query.GetDistinctValuesForProperty(namePropertyExpression);
var ages2 = query.GetDistinctValuesForProperty(agePropertyExpression);
// When you just know the property name
var names3 = query.GetDistinctValuesForProperty("Name");
var ages3 = query.GetDistinctValuesForProperty("Age");
Finally I found a solution. I need to include reference to System.Linq.Dynamic (downloaded by nuget), and use the "Select" method that accept String to reference column.
using System.Linq.Dynamic;
public static async Task<IEnumerable<Object>> GetDistinctValuesForProperty<T>(this IQueryable<T> query, String PropertyName)
{
return await query.Select(PropertyName).Distinct().ToListAsync();
}
and call as
String ColumnName = "DateTimeInsert";
DbSet<Log> oDbSet = _uow.DbContext.Set<Log>();
Array DistinctValues;
if (typeof(Log).GetProperty(ColumnName) != null)
{
DistinctValues = (await oDbSet.GetDistinctValuesForProperty(ColumnName)).ToArray();
}
else
{
DistinctValues = new object[0];
}
I need to use array vs ienumerable due to a cast problem in case of datetime types
You can create a generic selector method using Expressions
public static Func<T, T> SelectorFunc<T>(string[] columns) {
// input parameter "o"
var xParameter = Expression.Parameter(typeof(T), "o");
// new statement "new Data()"
var xNew = Expression.New(typeof(T));
// create initializers
var bindings = columns.Select(o => o.Trim())
.Select(o =>
{
// property "Field1"
var mi = typeof(T).GetProperty(o);
// original value "o.Field1"
var xOriginal = Expression.Property(xParameter, mi);
// set value "Field1 = o.Field1"
return Expression.Bind(mi, xOriginal);
}
);
// initialization "new Data { Field1 = o.Field1, Field2 = o.Field2 }"
var xInit = Expression.MemberInit(xNew, bindings);
// expression "o => new Data { Field1 = o.Field1, Field2 = o.Field2 }"
var lambda = Expression.Lambda<Func<T, T>>(xInit, xParameter);
// compile to Func<Data, Data>
return lambda.Compile();
}
using it would be
T.Select( SelectorFunc<T>( new string[]{ "Column" } ) ).Distinct().ToList();
You can also use it any other linq functions like
T.Select( SelectorFunc<T>( new string[]{ "Column" } ) ).Where();
T.Select( SelectorFunc<T>( new string[]{ "Column" } ) ).AsQueryable();
for additional reference you can see the full OP here
LINQ : Dynamic select
My issue is I need to query on the value of a property in a generic class. The property is tagged with an attribute.
See the following code:
var rowKeyProperty = EFUtil.GetClassPropertyForRowKey<T>();
var tenantKeyProperty = EFUtil.GetClassPropertyForTenantKey<T>();
var queryResult =
objContext.CreateObjectSet<T>().Single(l => (((int) tenantKeyProperty.GetValue(l, null)) == tenantKey) &&
(((int)rowKeyProperty.GetValue(l, null)) == KeyValue));
The rowKeyProperty and tenantKeyProperty are of type System.Reflection.PropertyInfo.
I understand why I am getting the error. When the linq query is translated to SQL, it can't understand the property.GetValue.
However, I'm completely stumped as to a work around here. Does anyone have any ideas how to achieve this? Thx.
You need to actually build up the Expression objects to represent the expression that you want this to mimic, in this case the expression you want to represent is:
l => l.SomeProperty == SomeValue
So you need to build up each component of that bit by bit, from creating the parameter, defining the equality operator, the property access, the constant value, etc.
public static Expression<Func<TItem, bool>> PropertyEquals<TItem, TValue>(
PropertyInfo property, TValue value)
{
var param = Expression.Parameter(typeof(TItem));
var body = Expression.Equal(Expression.Property(param, property),
Expression.Constant(value));
return Expression.Lambda<Func<TItem, bool>>(body, param);
}
Once you have all of that you can call it using the data that you have:
var queryResult = objContext.CreateObjectSet<T>()
.Where(PropertyEquals<T, int>(tenantKeyProperty, tenantKey))
.Where(PropertyEquals<T, int>(rowKeyProperty, KeyValue))
.Single();
Appendix here... Following #Servy answer and based on this topic with a nice answer by #TomBrothers, you can use the same logic to make a StartsWith (or similar) function:
public static Expression<Func<TItem, bool>> PropertyStartsWith<TItem>(PropertyInfo propertyInfo, string value)
{
var param = Expression.Parameter(typeof(TItem));
var m = Expression.MakeMemberAccess(param, propertyInfo);
var c = Expression.Constant(value, typeof(string));
var mi = typeof(string).GetMethod("StartsWith", new Type[] { typeof(string) });
var body = Expression.Call(m, mi, c);
return Expression.Lambda<Func<TItem, bool>>(body, param);
}
In this case, it forces value to be a string.
It is more correct to specify the type in Expression.Constant(value, typeof(TValue)))
public static Expression<Func<TItem, bool>> PropertyEquals<TItem, TValue>(
string property, TValue value)
{
var xParameter = Expression.Parameter(typeof(TItem));
var body = Expression.Equal(Expression.Property(xParameter, property), Expression.Constant(value, typeof(TValue)));
return Expression.Lambda<Func<TItem, bool>>(body, xParameter);
}
Or, like this, to check the property. ChangeType
public static Expression<Func<TItem, bool>> PropertyEquals<TItem, TValue>(
string property, TValue value)
{
var xParameter = Expression.Parameter(typeof(TItem));
var type = typeof(TItem).GetProperty(property).PropertyType;
value = ChangeType<TValue>(value);
BinaryExpression body = Expression.Equal(Expression.Property(xParameter, property), Expression.Constant(value, type));
return Expression.Lambda<Func<TItem, bool>>(body, xParameter);
}
What is it for. I check all class references to classes, I look for "..ID" entries. Somewhere I have a type "int" and "int?".
public class BudgetLimit : BaseRecord
{
[Required]
public int DepartmentID { get; set; }
public virtual Department Department { get; set;}
public int? ProjectID { get; set; }
public virtual Project Project { get; set; }
}
You add .AsEnableable after the LINQ statement.
e.g objectdata.AsEnumerable()
enter link description here
I want to create a method like this:
var result = database.Search<EntityType>(x=>x.Name, "Entity Name field value");
result = database.Search<EntityType>(x=>x.Id, "Entity Id field value");
result = database.Search<EntityType2>(x=>x.Id, "Entity2 Id field value");
result = database.Search<EntityTypeAny>(x=>x.FieldAny, "EntityAny FieldAny value");
How can I implement this method?
You can turn a selector and value into a predicate using Expression.Equal:
static IQueryable<TSource> Search<TSource, TValue>(
this IQueryable<TSource> source,
Expression<Func<TSource,TValue>> selector,
TValue value)
{
var predicate = Expression.Lambda<Func<TSource,bool>>(
Expression.Equal(
selector.Body,
Expression.Constant(value, typeof(TValue))
), selector.Parameters);
return source.Where(predicate);
}
Then you just need to do something like:
var result = database.SomeEntities.Search(x => x.SomeProp, "value");
If you want to do it from the database, then that depends on what the database is; for example, with LINQ-to-SQL you could add an additional method:
static IQueryable<TSource> Search<TSource, TValue>(
this System.Data.Linq.DataContext database,
Expression<Func<TSource, TValue>> selector,
TValue value) where TSource : class
{
IQueryable<TSource> source = database.GetTable<TSource>();
return Search(source, selector, value);
}
and use:
var result = database.Search<SomeEntity, string>(x => x.SomeProp, "value");
frankly I think it is clearer to use the database.SomeEntities version, though.
I can only think of this (with 2 generic arguments)
public static IEnumerable<TModel> Search<TModel, TValue>(
Expression<Func<TModel, TValue>> expression,
TValue value
)
{
return new List<TModel>();
}
usage
var result = Search<EntityType, int>(x => x.Id, 1);
var result2 = Search<EntityType, string>(x => x.Name, "The name");
you can replace TValue with object to avoid the second generic argument, but I would stick with this.
Btw. this works great in conjunction with this little helper
public static class ExpressionHelpers
{
public static string MemberName<T, V>(this Expression<Func<T, V>> expression)
{
var memberExpression = expression.Body as MemberExpression;
if (memberExpression == null)
throw new InvalidOperationException("Expression must be a member expression");
return memberExpression.Member.Name;
}
}
Now you can get the Name of the Property (Id oder Name) in this example by calling
var name = expression.MemberName();
do you want types to dynamic
public ReturnType Read<ReturnType>(string FieldName, object dfVal)
{
if (Res.IsDBNull(Res.GetOrdinal(FieldName)))
return dfVal;
try {
return (ReturnType)Res.GetValue(Res.GetOrdinal(FieldName));
} catch (Exception ex) {
return dfVal;
}
}