I am trying to create a query based on some JSON, I currently have the JSON parsed into a set of rules, each rule contains the name of the field, the type of comparison (=, > etc) and the value to compare.
The issue I am having is getting it from that rule, to an IQueryable object, I am guessing I need to use reflection and somehow build the expression tree, but I'm not sure on the right approach...
Assuming I have:
public class Order : BaseEntity
{
public int OrderID{ get; set; }
}
and I have the rule which is:
public class Rule
{
public string field { get; set; }
public Operations op { get; set; }
public string data { get; set; }
}
Running it I get:
field = "OrderID"
op = "eq"
data = "123"
I have the method to parse it with the signature:
public IQueryable<T> FilterObjectSet<T>(IQueryable<T> inputQuery) where T : class
As part of this method I want to do:
inputQuery = inputQuery.Where(o => propertyInfo.Name == rule1.data);
This doesn't work because it basically just generates the sql "OrderID" = "123" which is obviously wrong, I need it to take the column name from inputQuery that has the same name as propertyInfo.Name and build the query that way...
Hope that made sense? Any suggestions?
Edit: I guess what I am asking is to convert a string (Because I can build one pretty simply from the rule) to an expression, maybe using Dynamic LINQ?
Something like this:
public static IQueryable<T> FilterObjectSet<T>(IQueryable<T> inputQuery,
Rule rule) where T : class
{
var par = Expression.Parameter(typeof(T));
var prop = Expression.PropertyOrField(par, rule.field);
var propType = prop.Member.MemberType == System.Reflection.MemberTypes.Field ?
((FieldInfo)prop.Member).FieldType :
((PropertyInfo)prop.Member).PropertyType);
// I convert the data that is a string to the "correct" type here
object data2 = Convert.ChangeType(rule.data,
propType,
CultureInfo.InvariantCulture);
var eq = Expression.Equal(prop, Expression.Constant(data2));
var lambda = Expression.Lambda<Func<T, bool>>(eq, par);
return inputQuery.Where(lambda);
}
If you need some explanation, you can ask. Note that this won't work on types that have special implicit conversions (like a MyString type that has an implicit conversion from string). This because Convert.ChangeType uses only the IConvertible interface.
Null handling for data is perhaps something else that should be handled.
Be aware that I'm not sure the Expression.PropertyOrField is handled by the various IQueryable<T> engines (LINQ-to-SQL and EF). I have only tested it with the AsQueryable() engine. If they don't "accept" it, you must split it in a Expression.Property or Expression.Field depending on what rule.field is.
A nearly equivalent version that doesn't use Expression.PropertyOrField:
public static IQueryable<T> FilterObjectSet<T>(IQueryable<T> inputQuery,
Rule rule) where T : class
{
Type type = typeof(T);
var par = Expression.Parameter(type);
Type fieldPropertyType;
Expression fieldPropertyExpression;
FieldInfo fieldInfo = type.GetField(rule.field);
if (fieldInfo == null)
{
PropertyInfo propertyInfo = type.GetProperty(rule.field);
if (propertyInfo == null)
{
throw new Exception();
}
fieldPropertyType = propertyInfo.PropertyType;
fieldPropertyExpression = Expression.Property(par, propertyInfo);
}
else
{
fieldPropertyType = fieldInfo.FieldType;
fieldPropertyExpression = Expression.Field(par, fieldInfo);
}
object data2 = Convert.ChangeType(rule.data, fieldPropertyType);
var eq = Expression.Equal(fieldPropertyExpression,
Expression.Constant(data2));
var lambda = Expression.Lambda<Func<T, bool>>(eq, par);
return inputQuery.Where(lambda);
}
In the end I used a Dynamic Linq library I found on Guthrie's Blog:
http://weblogs.asp.net/scottgu/archive/2008/01/07/dynamic-linq-part-1-using-the-linq-dynamic-query-library.aspx
Using this I was able to properly parse out and use the parameters I had built into the rules
Related
So, I was wondering if it possible to do the next thing in c#:
I have a DB model - let's say it is Car:
public class Car {
public string Id {get;set;}
public string Name {get;set}
}
And a DbSet for this type in someDbContext:
public DbSet<Car> Cars {get;set;}
And also I have a CarDto
public class CarDto {
public string Id {get;set;}
public string Name {get;set}
}
And as result we get something like this:
var select = new Func<CarDto, bool>(car => car.Name == "BMW");
// And somehow use this expression for other type Car
someDbContext.Cars.Where(select);
Maybe there is an approach in which I could map these Funcs like this:
var newFunc = mapper.Map<Func<Car, bool>>(select);
Any thoughts?
If you just want to handle rewriting property accesses, you can use an ExpressionVisitor which looks a bit like this:
public class Program
{
public static void Main()
{
Expression<Func<Car, bool>> expr = x => x.Name == "BMW";
var replaced = ReplaceParameter<CarDto>(expr);
}
private static Expression<Func<T, bool>> ReplaceParameter<T>(LambdaExpression expr)
{
if (expr.Parameters.Count != 1)
throw new ArgumentException("Expected 1 parameter", nameof(expr));
var newParameter = Expression.Parameter(typeof(T), expr.Parameters[0].Name);
var visitor = new ParameterReplaceVisitor()
{
Target = expr.Parameters[0],
Replacement = newParameter,
};
var rewrittenBody = visitor.Visit(expr.Body);
return Expression.Lambda<Func<T, bool>>(rewrittenBody, newParameter);
}
}
public class ParameterReplaceVisitor : ExpressionVisitor
{
public ParameterExpression Target { get; set; }
public ParameterExpression Replacement { get; set; }
protected override Expression VisitMember(MemberExpression node)
{
if (node.Expression == this.Target)
{
// Try and find a property with the same name on the target type
var members = this.Replacement.Type.GetMember(node.Member.Name, node.Member.MemberType, BindingFlags.Public | BindingFlags.Instance);
if (members.Length != 1)
{
throw new ArgumentException($"Unable to find a single member {node.Member.Name} of type {node.Member.MemberType} on {this.Target.Type}");
}
return Expression.MakeMemberAccess(this.Replacement, members[0]);
}
return base.VisitMember(node);
}
}
We need to deconstruct the LambdaExpression into its body and parameters. We need to create a new parameter which has the correct type, and replace all usages of the old parameter with the new one. This is where the visitor comes in: whenever it sees you access a member on the old parameter, it tries to find the corresponding member on the new parameter, and access that instead.
We then construct a new LambdaExpression, using the rewritten body and the new parameter.
You have a whole bunch of options:
Derive your Dto class from the context class. That way you can use polymorphism as normal.
Extract an interface and implement it in both your Dto and context classes. Same as above then, use polymorphism.
Use duck-typing. In C#, that's done with the dynamic keyword. You lose Intellisense and compile-time error checking, but your code will work.
Reflection. It's a lot of code, it's slow, it's practically a much worse version of #3, but you can cobble it together if you really try.
Something like Automapper will help you map your context to your Dto piece-wise, but it won't help you translate your lambda function filters.
I have list of dynamic objects where I want to query by custom property. In other words, it would look like this if I wasn't going for reflection:
IEnumerable<User> FilterUsers(IEnumerable<User> users, string selectedValue)
{
users.Where(user => user.Name == selectedValue);
}
So far I've come up with the following implementation that works if users is typed:
IEnumerable<User> FilterUsers(IEnumerable<User> users, string selectedField, string selectedValue)
{
LabelTarget returnTarget = Expression.Label(typeof(bool));
ParameterExpression userParameter = Expression.Parameter(typeof(User));
MemberExpression userSelectedField = Expression.Property(userParameter, selectedField);
Expression test = Expression.Equal(userSelectedField, Expression.Constant(selectedValue));
Expression iftrue = Expression.Return(returnTarget, Expression.Constant(true));
Expression iffalse = Expression.Return(returnTarget, Expression.Constant(false));
var ex = Expression.Block(
Expression.IfThenElse(test, iftrue, iffalse),
Expression.Label(returnTarget, Expression.Constant(false)));
var whereClause = Expression.Lambda<Func<User, bool>>(
ex,
new ParameterExpression[] { userParameter }
).Compile();
return users.Where(user => whereClause(user));
}
What I am really trying to do is to make users dynamic object:
IEnumerable<dynamic> FilterUsers(IEnumerable<dynamic> users, string selectedField, string selectedValue) {
// ...
ParameterExpression userParameter = Expression.Parameter(typeof(object)); // ???
MemberExpression userSelectedField = Expression.Property(userParameter, selectedField); // throws
// ...
}
This throws the following exception: Instance property 'Name' is not defined for type 'System.Object' (Parameter 'propertyName'). What am I missing?
Alternatively, how can I use Dictionary<string, object>?
Using dynamic here doesn't get you much: you'd be better off using generics if you can:
IEnumerable<T> FilterUsers<T>(IEnumerable<T> users, string selectedField, string selectedValue)
{
var userParameter = Expression.Parameter(typeof(T));
var userSelectedField = Expression.Property(userParameter, selectedField);
// etc...
}
If you do need to use dynamic, then you'll need to get the runtime type of each user, using .GetType(). However bear in mind that there's nothing stopping someone from passing in an IEnumerable containing lots of different types of object, and they don't all have to have a property called selectedField!
Or, they might pass in lots of different types of object, each of one has a property called selectedField, but they're distinct properties (e.g. class A { public string Foo { get; set; } } and class B { public string Foo { get; set; } } -- those two Foo properties are distinct).
So you'll have to call .GetType() on each one of them, which means you won't be able to get the performance benefits of using compiled expressions.
If you can guarantee that all elements have the same type, you can do something like:
private static IEnumerable<dynamic> FilterCollection(IEnumerable<dynamic> collection, string property, string value)
{
if (!collection.Any()) return collection;
var collectionItemType = collection.First().GetType();
var userParameter = Expression.Parameter(typeof(object));
var convertedUser = Expression.Convert(userParameter, collectionItemType);
var userSelectedField = Expression.Property(convertedUser, selectedField);
...
}
Beware however that you're enumerating users twice, which is probably a bad thing. You might do better to get the IEnumerator yourself and work with it explicitly.
As #canton7 said you should be using a generic method. I also see in your question you specified you're looking for properties, why not use regular old reflection?
public static IEnumerable<T> FilterItems<T>(IEnumerable<T> items, string property, string value)
{
var prop = typeof(T).GetProperties().First(p => p.Name == property);
return items.Where(i => prop.GetValue(i).ToString().Contains(value));
}
Of course that code should be enhanced to handle different errors....
How would I get the JSON PropertyName of the following class & Property? Something like a "nameof()" equivilent for JSON Properties?
ie, something like
var jsonName = GetJSONPropertyName(SampleClass.SampleClassID); //should return "jsoniD"
public class SampleClass
{
public SampleClass() { }
[JsonProperty(PropertyName = "jsoniD")]
public string SampleClassID { get; set; }
}
A good question would be, how do you pass a property in a type-safe way. Properties are not first-class objects in .NET.
One of the ways would be this:
using System.Linq.Expressions;
// ...
static string GetJsonPropertyName<TC, TP>(Expression<Func<TC, TP>> expr)
{
if (expr.Body is MemberExpression body)
return body.Member.GetCustomAttribute<JsonPropertyAttribute>()?.PropertyName;
else
throw new ArgumentException("expect field access lambda");
}
You'll need to call the function like this:
var jsonName = GetJsonPropertyName<SampleClass, string>(x => x.SampleClassID);
Yes, it doesn't feel very natural. Sorry for that.
Thanks to #elgonzo, the code can be simplified like this:
static string GetJsonPropertyName<TC>(Expression<Func<TC, object>> expr)
{
// in case the property type is a value type, the expression contains
// an outer Convert, so we need to remove it
var body = (expr.Body is UnaryExpression unary) ? unary.Operand : expr.Body;
if (body is System.Linq.Expressions.MemberExpression memberEx)
return memberEx.Member.GetCustomAttribute<JsonPropertyAttribute>()?.PropertyName;
else
throw new ArgumentException("expect field access lambda");
}
var jsonName = GetJsonPropertyName<SampleClass>(x => x.SampleClassID);
Working Value-Type Expression support based on #Vlad's solution
(with UnaryExpression pattern lifted from this SO POST)
public static string GetJsonPropertyName<T>(Expression<Func<T, object>> expr)
{
if (((expr.Body as UnaryExpression)?.Operand ?? expr.Body) is MemberExpression body)
return body.Member.GetCustomAttribute<JsonPropertyAttribute>()?.PropertyName;
else
throw new ArgumentException("expect field access lambda");
}
I want to execute linq method on iqueryable with an expression tree from function where I'm passing name of linq method and name of property. But my sample method works only with mapped properties. It throws an exception when I try to for example to find max of calculated property.
My classes:
public partial class Something
{
public int a { get; set; }
public int b { get; set; }
}
public partial class Something
{
public int calculated { get { return a * b; } }
}
Sample method:
public static object ExecuteLinqMethod(IQueryable<T> q, string Field, string Method)
{
var param = Expression.Parameter(typeof(T), "p");
Expression prop = Expression.Property(param, Field);
var exp = Expression.Lambda(prop, param);
Type[] types = new Type[] { q.ElementType, exp.Body.Type };
var mce = Expression.Call(typeof(Queryable),Method,types,q.Expression,exp);
return q.Provider.Execute(mce);
}
To be able to query on calculated properties, you have at least 2 options:
1) you store the calculated values in the db with the rows (or in a different table), and use them in your queries of course this requires datamodel change, and redundancy in data, but is the most performant way. But is not that exciting, so lets move on to
2) you need to be able to express the way you "calculate" the properties in a way that sql will understand, meaning the property needs to be replaced with a linq expression in the final query. I found in 2009 an amazing article from Eric Lippert on registering inline such properties, but I cannot find it anymore. As such here is a link to another, that has the same idea. Basically you define your calculation as an expression tree, and use the compiled version in your code.
To make it more convenient, you would attribute your property with a
[AttributeUsage(AttributeTargets.Property)]
class CalculatedByAttribute: Attribute
{
public string StaticMethodName {get; private set;}
public CalculatedByAttribute(string staticMethodName)
{
StaticMethodName = staticMethodName;
}
}
Like:
public partial class Something
{
[CalculatedBy("calculatedExpression")]
public int calculated { get { return calculatedExpression.Compile()(this); } }
public static Expression<Func<Something, int>> calculatedExpression = s => s.a * s.b;
}
(of course you can cache the compilation) :)
Then in your method, if the property has your attribute, you get the static property value, and use that in your queries. Something along:
public static object ExecuteLinqMethod<T>(IQueryable<T> q, string Field, string Method)
{
var propInfo = typeof(T).GetProperty(Field);
LambdaExpression exp;
var myAttr = propInfo.GetCustomAttributes(typeof(CalculatedByAttribute), true).OfType<CalculatedByAttribute>().FirstOrDefault();
if (myAttr != null)
exp = (LambdaExpression)typeof(T).GetField(myAttr.StaticMethodName, BindingFlags.Static | BindingFlags.Public).GetValue(null);
else
{
var param = Expression.Parameter(typeof(T), "p");
Expression prop = Expression.Property(param, Field);
exp = Expression.Lambda(prop, param);
}
Type[] types = new Type[] { q.ElementType, exp.Body.Type };
var mce = Expression.Call(typeof(Queryable),Method,types,q.Expression,exp);
return q.Provider.Execute(mce);
}
Here are my interfaces and enum, dumbed down slightly. :
public interface IExpression
{
ExpressionType ExpressionType { get; }
}
public interface ILiteralExpression : IExpression
{
object Value { get; set; }
}
public interface IOperatorExpression : IExpression
{
IExpression[] Operands { get; set; }
string OperatorUniqueName { get; set; }
IOperatorExpression SetOperand(int index, IExpression expression);
}
public enum ExpressionType
{
Operator,
Literal
}
To create an expression, I can do something like this:
var expression = ExpressionManager.Engines["Default"].Parser.Parse("1 + 3 * 4 + \"myVariable\"");
Which is equivalent to something like this:
var expression1 = ExpressionManager.CreateOperator("Add", 2).
SetOperand(0, ExpressionManager.CreateOperator("Add", 2).
SetOperand(0, ExpressionManager.CreateLiteral(1)).
SetOperand(1, ExpressionManager.CreateOperator("Multiply", 2).
SetOperand(0, ExpressionManager.CreateLiteral(3)).
SetOperand(1, ExpressionManager.CreateLiteral(4)))).
SetOperand(1, ExpressionManager.CreateLiteral("myVariable"));
I'd like to be able to do something like this (efficiently):
(from e in expression
where e is ILiteralExpression && "myVariable".Equals(((ILiteralExpression)e).Value)
select (ILiteralExpression)e).ToList().
ForEach(e => e.Value = 2);
I think I need to do some implementing of IQueryable or something, but I'm not sure where to start. Any suggestions?
Walk your expression tree, and convert each element to an object from the System.Linq.Expressions namespace, using the factory methods of the Expression class.
If you cannot modify your IExpression class to add methods that let you implement the visitor pattern, you can rely on type checking the old style:
private static Expression ConvertExpression(IExpression expr) {
if (expr is ILiteralExpression) {
return Expression.Constant(((ILiteralExpression)expr).Value);
}
if (expr is IOperatorExpression) {
var ops = ((IOperatorExpression)expr)
.Operands
.Select(ConvertExpression)
.ToList();
var res = ops[0];
for (int i = 1 ; i != ops.Length ; i++) {
if (((IOperatorExpression)expr).OperatorUniqueName == "+") {
res = Expressions.Add(res, ops[i]);
} else if (((IOperatorExpression)expr).OperatorUniqueName == "-") {
res = Expressions.Subtract(res, ops[i]);
} else if (...) {
}
}
return res;
}
}
Of course you'll need a lot more logic in this method. The crucial part is passing parameters: you need to figure out where your variables are and what is their type, use Expression.ParameterExpression to create it, and then compile your converted expression into a Func<...> of some sort using the LambdaExpression.Compile method. With a compiled lambda in hand, you can plug in your expressions into LINQ's in-memory framework.
If you can make your IExpressions visitable, add a visitor that walks the expression and converts it to a LINQ expression using a stack.
What you're looking to do is build your own LINQ Query Provider. You can choose what LINQ operations you want to allow (WHERE, OrderBy, etc.)
This is the blog post series that helped me the most when I wrote one:
http://blogs.msdn.com/b/mattwar/archive/2007/07/30/linq-building-an-iqueryable-provider-part-i.aspx