I am trying to use Expression Trees because based on description, that seems to be the most correct (performant, configurable) approach.
I expect to be able to craft a statement that gets the first item from the existingItems collection that matches the propertyNameToCompareOn value of the incomingItem.
I have a method with the following signature and simulated code body...
DetectDifferences<T>(List<T> incomingItems, List<T> existingItems)
{
var propertyNameToCompareOn = GetThisValueFromAConfigFile(T.FullName());
//does this belong outside of the loop?
var leftParam = Expression.Parameter(typeof(T), "left");
var leftProperty = Expression.Property(leftParam, identField);
var rightParam = Expression.Parameter(typeof(T), "right");
var rightProperty = Expression.Property(rightParam, identField);
//this throws the error
var condition = Expression.Lambda<Func<T, bool>>(Expression.Equal(leftProperty, rightProperty));
foreach (var incomingItem in incomingItems) //could be a parallel or something else.
{
// also, where am I supposed to provide incomingItem to this statement?
var existingItem = existingItems.FirstOrDefault(expression/condition/idk);
// the statement for Foo would be something like
var existingFoos = exsistingItems.FirstOrDefault(f => f.Bar.Equals(incomingItem.Bar);
//if item does not exist, consider it new for persistence
//if item does exist, compare a configured list of the remaining properties between the
// objects. If they are all the same, report no changes. If any
// important property is different, capture the differences for
// persistence. (This is where precalculating hashes seems like the
// wrong approach due to expense.)
}
}
At the marked line above, I get an "Incorrect number of parameters supplied for lambda declaration" InvalidOperationException. At this point I am just hacking crap together from the web and I really dont know what this wants. There are a bunch of overloads that VS can full my screen with, and none of the examples make sense from the articles on MSDN/SO.
PS - I dont really want an IComparer or similar implementation if it can be helped. I can do that with reflection. I do need to make this as rapid as possible, but allow it to be called for multiple types, hence the choice of expression trees.
When working with expression trees, it's important to first understand, in real code, what you want to do.
I always begin by first writing out (in static code) what the resulting expression looks like with real C# lambda syntax.
Based on your description, your stated goal is that you should be able to (dynamically) look up some property of the type T that gives some sort of quick comparison. How would you write this if both T and TProperty were both known at compile time?
I suspect it would look something like this:
Func<Foo, Foo, bool> comparer = (Foo first, Foo second) =>
first.FooProperty == second.FooProperty;
Right away we can see that your Expression is wrong. You don't need one input T, you need two!
It should also be obvious why you're getting the InvalidOperationException as well. You never supplied any parameters to your lambda expression, only the body. Above, 'first' and 'second' are the parameters provided to the lambda. You'll need to provide them to the Expression.Lambda()call as well.
var condition = Expression.Lambda<Func<T,T, bool>>(
Expression.Equal(leftProperty, rightProperty),
leftParam,
rightParam);
This simply uses the Expression.Lambda(Expression, ParameterExpression[]) overload for Expression.Lambda. Each ParameterExpression is the parameter that is used in the body. That's it. Don't forget to .Compile() your expression into a delegate if you want to actually invoke it.
Of course this doesn't mean that your technique will be necessarily fast. If you're using fancy expression trees to compare two lists with a naive O(n^2) approach, it won't matter.
Here's a method to make a property access expression;
public static Expression<Func<T, object>> MakeLambda<T>(string propertyName)
{
var param = Expression.Parameter(typeof(T));
var propertyInfo = typeof(T).GetProperty(propertyName);
var expr = Expression.MakeMemberAccess(param, propertyInfo);
var lambda = Expression.Lambda<Func<T, object>>(expr, param);
return lambda;
}
which you can use like this;
var accessor = MakeLambda<Foo>("Name").Compile();
accessor(myFooInstance); // returns name
Making your missing line
var existingItem = existingItems.FirstOrDefault(e => accessor(e) == accessor(incomingItem));
Be aware the == only works well for value types like ints; careful of comparing objects.
Here's proof the lambda approach is much faster;
static void Main(string[] args)
{
var l1 = new List<Foo> { };
for(var i = 0; i < 10000000; i++)
{
l1.Add(new Foo { Name = "x" + i.ToString() });
}
var propertyName = nameof(Foo.Name);
var lambda = MakeLambda<Foo>(propertyName);
var f = lambda.Compile();
var propertyInfo = typeof(Foo).GetProperty(nameof(Foo.Name));
var sw1 = Stopwatch.StartNew();
foreach (var item in l1)
{
var value = f(item);
}
sw1.Stop();
var sw2 = Stopwatch.StartNew();
foreach (var item in l1)
{
var value = propertyInfo.GetValue(item);
}
sw2.Stop();
Console.WriteLine($"{sw1.ElapsedMilliseconds} vs {sw2.ElapsedMilliseconds}");
}
As someone's also pointed out, though, the double-loop in the OP is O(N^2) and that should probably be the next consideration if efficiency is the driver here.
Related
I am working on a project to filter a list in a generic way. I am retrieving an IEnumerable<T> at runtime but I don't know what is T. I need to cast the list I am retrieving to IEnumerable<T> and not IEnumerable, because I need extension methods like ToList and Where. Here is my code.
private IList<object> UpdateList(KeyValuePair<string, string>[] conditions)
{
// Here I want to get the list property to update
// The List is of List<Model1>, but of course I don't know that at runtime
// casting it to IEnumerable<object> would give Invalid Cast Exception
var listToModify = (IEnumerable<object>)propertyListInfoToUpdate.GetValue(model);
foreach (var condition in conditions)
{
// Filter is an extension method defined below
listToModify = listToModify.Filter(condition .Key, condition .Value);
}
// ToList can only be invoked on IEnumerable<T>
return listToModify.ToList();
}
public static IEnumerable<T> Filter<T>(this IEnumerable<T> source, string propertyName, object value)
{
var parameter = Expression.Parameter(typeof(T), "x");
var property = Expression.Property(parameter, propertyName);
var propertyType = ((PropertyInfo)property.Member).PropertyType;
Expression constant = Expression.Constant(value);
if (((ConstantExpression)constant).Type != propertyType)
{ constant = Expression.Convert(constant, propertyType); }
var equality = Expression.Equal(property, constant);
var predicate = Expression.Lambda<Func<T, bool>>(equality, parameter);
var compiled = predicate.Compile();
// Where can only be invoked on IEnumerable<T>
return source.Where(compiled);
}
Also please note that I cannot retrieve the List like this
((IEnumerable)propertyListInfoToUpdate.GetValue(model)).Cast<object>()
since it will generate the below exception in the Filter extention
ParameterExpression of type 'Model1' cannot be used for delegate parameter of type 'System.Object'
Use GetGenericArguments and MakeGenericMethod to interface generic signatures.
private IList<object> UpdateList(KeyValuePair<string, string> conditions)
{
var rawList = (IEnumerable)propertyListInfoToUpdate.GetValue(model);
var listItemType = propertyListInfoToUpdate.PropertyType.GetGenericArguments().First();
var filterMethod = this.GetType().GetMethod("Filter").MakeGenericMethod(genericType);
object listToModify = rawList;
foreach (var condition in conditions)
{
listToModify = filterMethod.Invoke(null, new object[] { listToModify, condition.Key, condition.Value })
}
return ((IEnumerable)listToModify).Cast<object>().ToList();
}
Assuming your propertyListInfoToUpdate is a PropertyInfo and the property type is List<T>.
Why are you using Expression at all? It's hard to understand your question without a good Minimal, Complete, and Verifiable code example. Even if we could solve the casting question, you're still returning IList<object>. It's not like the consumer of the code will benefit from the casting.
And it's not really possible to solve the casting problem, at least not in the way you seem to want. A different approach would be to call the Filter() dynamically. In the olden days, we'd have to do this by using reflection, but the dynamic type gives us runtime support. You could get it to work something like this:
private IList<object> UpdateList(KeyValuePair<string, string>[] conditions)
{
dynamic listToModify = propertyListInfoToUpdate.GetValue(model);
foreach (var condition in conditions)
{
// Filter is an extension method defined below
listToModify = Filter(listToModify, condition.Key, condition.Value);
}
// ToList can only be invoked on IEnumerable<T>
return ((IEnumerable<object>)Enumerable.Cast<object>(listToModify)).ToList();
}
NOTE: your original code isn't valid; I made the assumption that conditions is supposed to be an array, but of course if you change it to anything that has a GetEnumerator() method, that would be fine.
All that said, it seems to me that given the lack of a compile-time type parameter, it would be more direct to just change your Filter() method so that it's not generic, and so that you use object.Equals() to compare the property value to the condition value. You seem to be jumping through a lot of hoops to use generics, without gaining any of the compile-time benefit of generics.
Note that if all this was about was executing LINQ query methods, that could be addressed easily simply by using Enumerable.Cast<object>() and using object.Equals() directly. It's the fact that you want to use expressions to access the property value (a reasonable goal) that is complicating the issue. But even there, you can stick with IEnumerable<object> and just build the object.Equals() into your expression.
Creating an expression and compiling it every time is very expensive. You should either use Reflection directly or a library like FastMember (or cache the expressions). In addition, your code uses Expression.Equal which translates into the equality operator (==), which is not a good way of comparing objects. You should be using Object.Equals.
Here is the code using FastMember:
private IList<object> UpdateList(KeyValuePair<string, string>[] conditions)
{
var listToModify = ((IEnumerable)propertyListInfoToUpdate.GetValue(model)).Cast<object>();
foreach (var condition in conditions)
{
listToModify = listToModify.Where(o => Equals(ObjectAccessor.Create(o)[condition.Key], condition.Value));
}
return listToModify.ToList();
}
Side note - your Filter method doesn't really need generics. It can be changed to accept and return an IEnumerable<object> by tweaking the expression, which also would've solved your problem.
I am really struggling to write a generic class, that needs to check if all members of a passed expression are not null, by invoking the actual object.
I am calling the method like:
new TestClass<Class1, Class2>.IsAnyMemberNull(x => x.Property1.List1, object1);
and the Method looks like:
public bool IsAnyMemberNull(Expression<Func<T, IList<TM>>> listExpression, T entityDb)
{
var expressionsToCheck = new List<MemberExpression>();
var expression = listExpression.Body as MemberExpression;
while (expression != null)
{
expressionsToCheck.Add(expression);
expression = expression.Expression as MemberExpression;
}
for (var i = expressionsToCheck.Count - 1; i >= 0; i--)
{
var objectMember = Expression.Convert(expressionsToCheck[i], typeof (object));
var lambda = Expression.Lambda<Func<T, object>>(objectMember);
var value = lambda.Compile().Invoke(entityDb);
if (value == null)
return true;
}
return false;
}
When executing, I get the exception:
incorrect number of parameters supplied for lambda declaration
Any ideas, what I have done wrong?
While it is possible to get your code working, so that correct lambdas are built and compiled, using repeatedly compiled lambdas to achieve null-checking is a costly overkill.
Generally, using lambdas so casually (compiling a lambda for each property in chain and a single object) comes with a notable performance hit. I've run some tests, and on my computer executing this method 1000 times for a given object yielded ~300-700 ms time (depending on the number of properties in the chain). Dunno how many entities you deal with, but it's not a good sign, and better replacements are available. Please read further...
The question is, what are you using this for? That method of yours reminds me of null-conditional operators quite a lot. Generally, if you:
just want to check if any property in the property chain is a null
use C# 6
have all parameter chain lambdas (like x => x.Property1.List1) known at runtime
Then you might scrap all that IsAnyMemberNull method altogether in favour of something like:
object1.Property1.List1 == null
Much more concise and no additional methods are required. I ran it 1 million times and it was still within ~23ms time. It means it's dozens thousands faster than creating all those lambdas.
If you can't use null-coalescing operators for whatever reason (especially when the expression is built dynamically), you might instead decide to use Field/Property reflection.
I took the liberty of removing all that generic class wrapping in favour of a generic method. From your usage example, it seemed the only purpose of the generic class was to access a specific method with class' generic type parameters. It means one new class would have to be made and stored for each variation of the method, for no apparent reason, if I'm not mistaken, for the rest of application's lifetime. Generic methods in specific classes are generally preferred to specific methods in generic classes, in cases like these.
Also, I removed IList, because I see no reason how requiring the last parameter to be of IList type serves the purpose of the function; it only limits its applicability with no apparent gain.
Overall, the result is the following:
public bool IsAnyMemberNull<TEntity, TMember>(Expression<Func<TEntity, TMember>> paramChain, TEntity entityDb)
{
var expressionsToCheck = new List<MemberExpression>();
var expression = paramChain.Body as MemberExpression;
while (expression != null)
{
expressionsToCheck.Add(expression);
expression = expression.Expression as MemberExpression;
}
object value = entityDb;
for (var i = expressionsToCheck.Count - 1; i >= 0; i--)
{
var member = expressionsToCheck[i].Member;
if (member is PropertyInfo) value = (member as PropertyInfo).GetValue(value);
else if (member is FieldInfo) value = (member as FieldInfo).GetValue(value);
else throw new Exception(); // member generally should be a property or field, shouldn't it?
if (value == null)
return true;
}
return false;
}
After running this ~1000 times, it took about 4-6ms; 50-100 times better than lambdas, though null-propagation still reigns supreme.
Called as following (assuming it still resides in TestClass, which it doesn't need to):
new TestClass().IsAnyMemberNull<Class1,Class2>(x => x.Property1.List1, object1);
(Class1 and Class2 might not be necessary, thanks to type inference)
Hope this helps. It's not exactly what you asked for, but I'm afraid with all this lambda-spawning you could run into serious performance issues; especially if this code was to be used many times per request.
You have a problem in lambda expression creation - it is simpler than you thought. You should build lambda for each expressionToCheck with original expression parameter:
for (var i = expressionsToCheck.Count - 1; i >= 0; i--)
{
var lambda = Expression.Lambda<Func<T, object>>(expressionsToCheck[i], listExpression.Parameters);
var value = lambda.Compile().Invoke(entityDb);
if (value == null)
return true;
}
I almost got what I need but only one place where I stucked. I need to build fileCount = c.CPNDocs.Count() Dynamically in Lambda Expression. Code is below with comments what I am using to build Dynamic Lambda Expression.
var dColDefaultList = new List<String>() { "Download", "I_ID", "C_TYP", "C_LST_ACT" }; // <------- Columns I need in Lambdas Expression
ParameterExpression cParam = Expression.Parameter(typeof(CPNDBase), "c");
NewExpression newExp = Expression.New(typeof(DTDataModel));
List<MemberBinding> bindings = new List<MemberBinding>();
foreach (String sCol in dColDefaultList)
{
if (!String.Equals(sCol, "Download")) {
bindings.Add(GetMemberBinding(sCol, cParam, sCol));
}
else
{
bindings.Add(GetMemberBinding("fileCount", cParam, "CPNDocs.Count()")); // <-------need count of rows return from CPNDocs(Different Table) is a Object I recieved from Entity Relatioship
}
}
MemberInitExpression memberInitExpression = System.Linq.Expressions.Expression.MemberInit(newExp, bindings);
Expression<Func<CPNDBase, DTDataModel>> selector = (Expression<Func<CPNDBase, DTDataModel>>)BinaryExpression.Lambda(memberInitExpression, cParam);
// selector will be selector = {c => new DTDataModel() {fileCount = c.CPNDocs, I_ID = c.I_ID, C_TYP = c.C_TYP, C_LST_ACT = c.C_LST_ACT }}
// but I Need selector = {c => new DTDataModel() {fileCount = c.CPNDocs.Count(), I_ID = c.I_ID, C_TYP = c.C_TYP, C_LST_ACT = c.C_LST_ACT }}
// Question is How can I make fileCount = c.CPNDocs.Count() ?
var resultLm = finalFilteredCPNData.AsQueryable<CPNDBase>().Select(selector);
Above method is defined here :
static MemberBinding GetMemberBinding(string property, ParameterExpression param, string column)
{
MemberInfo memberInfo = typeof(DTDataModel).GetMember(property)[0];
MemberExpression memberExpression = LambdaExpression.PropertyOrField(param, column);
return System.Linq.Expressions.Expression.Bind(memberInfo, memberExpression);
}
Does anybody know how can I do this?
The Count() is not a property. It is an extension method implemented in a static class. This extension method is implemented at several places. Correct place depends on what are your classes inheriting from. To find the correct place you use the "go to definition" feature of Visual Studio.
e.g. for IQueryable.Count() the extension methods are implemented by System.Linq.Queryable static class as can be seen here → http://referencesource.microsoft.com/#System.Core/System/Linq/IQueryable.cs
So in order to encode the expression you need to encode a call to the extension method.
Much simpler way to generate expression trees from strings was shown quite early in a prototype published by Microsoft. Introductory article is available e.g. in Dynamic Expressions and Queries in LINQ
We use modified version of the original source of the automatic "string to linq" engine with success and it simplifies development a lot. By inspecting the source code of the System.Linq.Dynamic you can find exact way how to encode the expression. Link to the original source code available through NuGet is mentioned e.g. in Stack Overflow article Dynamic LINQ - Is There A .NET 4 Version?
I am not very familiar with lambda expressions. So I have the following expression:
EnabledPropertySelector = l => l.FranchiseInfo.ExternalSystemType == ExternalSystemTypes.Master
And two properties:
public string VisibilityPropertyName { get; set; }
public object VisibilityPropertyValue { get; set; }
I want to extract some data from the expression so in the end I can get the values of the two properties:
VisibilityPropertyName == 'FranchiseInfo.ExternalSystemType';
VisibilityPropertyValue == ExternalSystemTypes.Master;
VisibilityPropertyName is always a string. This is the name of the property.
VisibilityPropertyValue can be of any type.
EDIT:
I have a lot of properties. Some of them are dependent on other properties. For every single property I have to manually write the name and the value of the parent property:
{ VisibilityPropertyName = 'FranchiseInfo.ExternalSystemType', VisibilityPropertyValue = ExternalSystemTypes.Master, EnabledPropertySelector = l => l.FranchiseInfo.ExternalSystemType == ExternalSystemTypes.Master}
Instead of writing all this I want to write only the expression and populate the properties from it.
This is the declaration of the expresion:
Expression<Func<TEntity, bool?>> EnabledPropertySelector
First off all, you need an Expression. What's the type of EnabledPropertySelector? It'll need to be something like Expression<Func<T, bool>> where T is whatever the type of "l" in your example is.
If you already have an Expression then you can use the Expression API to extract whatever you need:-
var body = EnabledPropertySelector.Body as BinaryExpression;
var left = body.Left as PropertyExpression;
var outerMemberName = left.Member.Name;
var innerMemberName = (left.Expression as PropertyExpression).Member.Name
VisibilityPropertyName = innerMemberName + "." + outerMemberName;
var right = body.Right as PropertyExpression;
var rightValueDelegate = Expression.Lambda<Func<object>>(right).Compile();
VisibilityPropertyValue = rightValueDelegate();
etc.
I really recommend doing some reading to properly grok the expression API before diving in though; there are a lot of corner cases depending on how flexible you need to be. E.g. is the expression always of the form parameter.Property.Property == constant? It gets really complicated really quickly, so you'll want a solid understanding of the fundamentals before trying to handle any real-world cases.
There's a reasonable introduction to expression trees on MSDN, but some focused googling might get you a better understanding quicker.
You can use Funciton and Action class, I'm not very sure of what you want be able to do, but I can give an tip.
Functions returns a value:
Function<InputType1,InputType2,ResultType> functionVariableName;
Usage:
functionVariableName = (param1, param2) => {
//...process both params
return result;
};
Actions, do not return values:
Action<InputType1,InputType2> actionVariableName;
Usage:
actionVariableName= (param1, param2) => {
//...process both params
};
If the lambda expression is simple (one line, with out if expression) you can make the lambda with out {}:
functionVariableName = (param1, param2) => //process values and result;
Hope this helps...
if you want to create an IEnumerable where the two properties are equal:
var results = EnabledPropertySelector.Where(l => l.FranchiseInfo.ExternalSystemType ==
ExternalSystemTypes.Master.ToString());
I want to provide a set of filters for a user to pick from, and each filter will correspond to an Expression<Func<X, bool>>. So, I might want to take a dynamic list of available items ('Joe', 'Steve', 'Pete', etc), and create a collection of "hard-coded" filters based on those names, and let the user select which filter(s) he wants to use. My problem is that even when I try to "hard-code" my expression based on a string value from the dynamic list, the expression is still storing the value as, what looks to me, a property hanging off of an anonymous type (and I don't know how to serialize the anon. type). Sorry if this is confusing, I'm not quite sure how to articulate this.
Here's my sample code:
public class Foo
{
public string Name { get; set; }
}
static void Main(string[] args)
{
Foo[] source = new Foo[]
{
new Foo() { Name = "Steven" } ,
new Foo() { Name = "John" } ,
new Foo() { Name = "Pete" },
};
List<Expression<Func<Foo, bool>>> filterLst = new List<Expression<Func<Foo, bool>>>();
foreach (Foo f in source)
{
Expression<Func<Foo, bool>> exp = x => x.Name == f.Name;
filterLst.Add(exp);
}
}
}
My problem is that when I look at when I look at the body of my expression, it reads as follows:
(x.Name = value(ConsoleApplication1.Program+<>c__DisplayClass3).value)
When what I really want is for the first one to read like this:
(x.Name = "Steven")
(if I change my code to this, instead, that's exactly what I get:
Expression<Func<Foo, bool>> exp = x => x.Name == "Steven";
)
I've tried forcing my value to a local string value before sticking it into the Expression, but it doesn't seem to help:
List<Expression<Func<Foo, bool>>> filterLst = new List<Expression<Func<Foo, bool>>>();
foreach (Foo f in source)
{
string value = f.Name;
Expression<Func<Foo, bool>> exp = x => x.Name == value;
filterLst.Add(exp);
}
I don't understand why (or really even how) it's still looking at some anonymous type even once I'm using a local variable that is declared to a string. Is there a way to make this work as I want it to?
The anon-type is actually the compiler-generated type it is using to perform capture of the variables. With delegates you can hack around this by implementing the capture by hand, but not with lambda expressions compiled to expression-trees.
Two choices:
build the expression tree explicitely on code via Expression.Constant etc
learn how to handle the anon types
The latter isn't too bad actually; they are just MemberExpression typically, although I have some code kicking around that covers this in full detail. I can also provide examples of buildin the expression tree, but I'm not at a PC at the moment and it doesn't lend itself well to iPod typing...
From a brief read of the question I'd look at the first option more than the second.
Oh, and watch out; the first foreach code in the question looks susceptible to the notorious l-value capture issue ;)
Edit: I found a PC ;p
var param = Expression.Parameter(typeof(Foo), "x");
var body = Expression.Equal(
Expression.PropertyOrField(param, "Name"),
Expression.Constant(f.Name, typeof(string)));
var exp = Expression.Lambda<Func<Foo, bool>>(body, param);
filterLst.Add(exp);
Marc Gravell's answer is correct, and here's how you'd implement his first choice:
var filters =
from f in source
let param = Expression.Parameter(typeof(Foo),"x")
select Expression.Lambda<Func<Foo, bool>>(
Expression.Equal(
Expression.Property(param, "Name"),
Expression.Constant(f.Name)), param);
But this typically isn't necessary. Your second example of the for loop should work with all major LINQ providers. Is there a reason you need the expression to use constants?