Develop Reference

Dynamically generate LINQ (member-)expressions

I am using FluentValidation to validate my form items in Xamarin.Forms. The definition of these items comes from the outside. Therefore I don't know, what properties I need to validate on my viewmodel.
RuleFor(viewmodel => viewmodel.Description).NotEmpty();
My idea was, to dynamically generate these expressions at runtime.
I created a List in the validator to store these expressions.
public List<Expression<Func<IViewModel, object>>> RequiredFieldExpressions
= new List<Expression<Func<IViewModel, object>>>();
Before validating my viewmodel, I generate the expressions.
var tmpMethod = typeof(TypeHelper).GetRuntimeMethod("GetExpression", new Type[] { typeof(string) });
var tmpGeneric = tmpMethod.MakeGenericMethod(myViewModel.GetType(), typeof(string));
var tmpInvokeResult = tmpGeneric.Invoke(null, new object[] {coreObjectPropertyName});
The method which creates the expression:
public static Expression<Func<T, TProperty>> GetExpression<T, TProperty>(string inPropertyName) where T : IViewModel
{
var tmpPropertyInfo = typeof(T).GetRuntimeProperties().First(p => p.Name == inPropertyName);
var tmpEntityParam = Expression.Parameter(typeof(T), "type");
Expression tmpExpression = Expression.Property(tmpEntityParam, tmpPropertyInfo);
if (tmpPropertyInfo.PropertyType != typeof(TProperty))
{
tmpExpression = Expression.Convert(tmpExpression, typeof(TProperty));
}
return Expression.Lambda<Func<T, TProperty>>(tmpExpression, tmpEntityParam);
}
Now the line which should create the validation rule throws an invalid cast exception.
// Cast not valid
RuleFor((Expression<Func<IViewModel, object>>) tmpInvokeResult).NotEmpty();
What am I missing?

I had to change the method call from
var tmpGeneric = tmpMethod.MakeGenericMethod(myViewModel.GetType(), typeof(string));
to
var tmpGeneric = tmpMethod.MakeGenericMethod(myViewModel.GetType(), typeof(object));
My guess
Xamarin.Forms uses a portable class library (PCL). It seems like the functionality to cast generic expressions isn't implemented.
It would be nice if somebody could verify this.
Update
I am not able to cast generic expressions. Seems like it's just not possible. You need to explicitly convert the expression before returning it.
https://dotnetfiddle.net/ufNId4

using expression trees to compare objects by a single property nets InvalidOperationException

I am trying to use Expression Trees because based on description, that seems to be the most correct (performant, configurable) approach.
I expect to be able to craft a statement that gets the first item from the existingItems collection that matches the propertyNameToCompareOn value of the incomingItem.
I have a method with the following signature and simulated code body...
DetectDifferences<T>(List<T> incomingItems, List<T> existingItems)
{
var propertyNameToCompareOn = GetThisValueFromAConfigFile(T.FullName());
//does this belong outside of the loop?
var leftParam = Expression.Parameter(typeof(T), "left");
var leftProperty = Expression.Property(leftParam, identField);
var rightParam = Expression.Parameter(typeof(T), "right");
var rightProperty = Expression.Property(rightParam, identField);
//this throws the error
var condition = Expression.Lambda<Func<T, bool>>(Expression.Equal(leftProperty, rightProperty));
foreach (var incomingItem in incomingItems) //could be a parallel or something else.
{
// also, where am I supposed to provide incomingItem to this statement?
var existingItem = existingItems.FirstOrDefault(expression/condition/idk);
// the statement for Foo would be something like
var existingFoos = exsistingItems.FirstOrDefault(f => f.Bar.Equals(incomingItem.Bar);
//if item does not exist, consider it new for persistence
//if item does exist, compare a configured list of the remaining properties between the
// objects. If they are all the same, report no changes. If any
// important property is different, capture the differences for
// persistence. (This is where precalculating hashes seems like the
// wrong approach due to expense.)
}
}
At the marked line above, I get an "Incorrect number of parameters supplied for lambda declaration" InvalidOperationException. At this point I am just hacking crap together from the web and I really dont know what this wants. There are a bunch of overloads that VS can full my screen with, and none of the examples make sense from the articles on MSDN/SO.
PS - I dont really want an IComparer or similar implementation if it can be helped. I can do that with reflection. I do need to make this as rapid as possible, but allow it to be called for multiple types, hence the choice of expression trees.

When working with expression trees, it's important to first understand, in real code, what you want to do.
I always begin by first writing out (in static code) what the resulting expression looks like with real C# lambda syntax.
Based on your description, your stated goal is that you should be able to (dynamically) look up some property of the type T that gives some sort of quick comparison. How would you write this if both T and TProperty were both known at compile time?
I suspect it would look something like this:
Func<Foo, Foo, bool> comparer = (Foo first, Foo second) =>
first.FooProperty == second.FooProperty;
Right away we can see that your Expression is wrong. You don't need one input T, you need two!
It should also be obvious why you're getting the InvalidOperationException as well. You never supplied any parameters to your lambda expression, only the body. Above, 'first' and 'second' are the parameters provided to the lambda. You'll need to provide them to the Expression.Lambda()call as well.
var condition = Expression.Lambda<Func<T,T, bool>>(
Expression.Equal(leftProperty, rightProperty),
leftParam,
rightParam);
This simply uses the Expression.Lambda(Expression, ParameterExpression[]) overload for Expression.Lambda. Each ParameterExpression is the parameter that is used in the body. That's it. Don't forget to .Compile() your expression into a delegate if you want to actually invoke it.
Of course this doesn't mean that your technique will be necessarily fast. If you're using fancy expression trees to compare two lists with a naive O(n^2) approach, it won't matter.

Here's a method to make a property access expression;
public static Expression<Func<T, object>> MakeLambda<T>(string propertyName)
{
var param = Expression.Parameter(typeof(T));
var propertyInfo = typeof(T).GetProperty(propertyName);
var expr = Expression.MakeMemberAccess(param, propertyInfo);
var lambda = Expression.Lambda<Func<T, object>>(expr, param);
return lambda;
}
which you can use like this;
var accessor = MakeLambda<Foo>("Name").Compile();
accessor(myFooInstance); // returns name
Making your missing line
var existingItem = existingItems.FirstOrDefault(e => accessor(e) == accessor(incomingItem));
Be aware the == only works well for value types like ints; careful of comparing objects.
Here's proof the lambda approach is much faster;
static void Main(string[] args)
{
var l1 = new List<Foo> { };
for(var i = 0; i < 10000000; i++)
{
l1.Add(new Foo { Name = "x" + i.ToString() });
}
var propertyName = nameof(Foo.Name);
var lambda = MakeLambda<Foo>(propertyName);
var f = lambda.Compile();
var propertyInfo = typeof(Foo).GetProperty(nameof(Foo.Name));
var sw1 = Stopwatch.StartNew();
foreach (var item in l1)
{
var value = f(item);
}
sw1.Stop();
var sw2 = Stopwatch.StartNew();
foreach (var item in l1)
{
var value = propertyInfo.GetValue(item);
}
sw2.Stop();
Console.WriteLine($"{sw1.ElapsedMilliseconds} vs {sw2.ElapsedMilliseconds}");
}
As someone's also pointed out, though, the double-loop in the OP is O(N^2) and that should probably be the next consideration if efficiency is the driver here.

LINQ Building Dynamic Expression in C#

I would like to dynamically build the linq expression used in the following Select statement on a DataTable filled from a DB:
IQueryable iq = myDataTable
.AsEnumerable()
.AsQueryable()
.Select(
d => new {
Field1 = d.Field<int>(37),
Field2 = d.Field<string>(0),
Field3 = d.Field<DateTime>(1)
}
);
The fields are not known before runtime. I will get the list of fields ("Field1", "Field2", "Field3"...), their types (int, string, datetime...) and their indexes (37, 0, 1...) at runtime.
Is there any way to achieve that?
[UPDATE]
I started to write the following code:
Type type = typeof(DataRow);
MethodInfo mi1 = typeof(DataRowExtensions).GetMethod("Field", new Type[] { typeof(DataRow), typeof(int) });
mi1 = mi1.MakeGenericMethod(type);
List<Expression> list1 = new List<Expression>();
ParameterExpression datarow1 = Expression.Parameter(type, "datarow");
ConstantExpression constant1 = Expression.Constant(37, typeof(int));
list1.Add(datarow1);
list1.Add(constant1);
ReadOnlyCollection<Expression> collection1 = new ReadOnlyCollection<Expression>(list1);
MethodCallExpression right1 = Expression.Call(null, mi1, datarow1, constant1);
Expression left1 = Expression.Parameter(type, "Field1");
var expression1 = Expression.Assign(left1, right1);
MethodInfo mi2 = typeof(DataRowExtensions).GetMethod("Field", new Type[] { typeof(DataRow), typeof(int) });
mi2 = mi2.MakeGenericMethod(type);
List<Expression> list2 = new List<Expression>();
ParameterExpression datarow2 = Expression.Parameter(type, "datarow");
ConstantExpression constant2 = Expression.Constant(0, typeof(int));
list2.Add(datarow2);
list2.Add(constant2);
ReadOnlyCollection<Expression> collection2 = new ReadOnlyCollection<Expression>(list2);
MethodCallExpression right2 = Expression.Call(null, mi2, datarow2, constant2);
Expression left2 = Expression.Parameter(type, "Field2");
var expression2 = Expression.Assign(left2, right2);
List<Expression> bindings = new List<Expression>();
bindings.Add(expression1);
bindings.Add(expression2);
var newInit = Expression.New(typeof(object));
var init = Expression.NewArrayInit(type, bindings);
var dr = Expression.Parameter(type, "dr");
var linq = Expression.Lambda<Func<DataRow, DataRow>>(init, dr);
It compiles, however, there are several problems:
expression1 is (Field1 = datarow.Field(0)) instead of Field1 = datarow.Field<int>(0)
same for expression 2
it throws a runtime exception: 'System.Data.DataRow[]' cannot be used for return type 'System.Data.DataRow' on the line var linq = Expression.Lambda...
Could you please help?
#George
I think I do not need a Type dictionary as I have a set of predefined types.

I strongly recommend for this kind of situations to build a query string and run it against your database.
But ... for the sake of art, you can do this. To achieve what you want you need more than just creating an expression tree. First, you will need a "pool" of anonymous types for each projection case found at runtime. Id est, you need to have a Dictionary < somekey, sometype> where somekey can be a string that represents a serialized combination of columns (column indexes, etc) and sometype will be a real System.Type. But wait those types does not exists at compilation time. The question is: How to create these types?. The ONLY solution is to use Reflection.Emit and AssemblyBuilder, TypeBuilder, ModuleBuilder, etc ... in order to create new types at runtime. Belive me or not, anonymous types are just normal types but C# compiler does this magic for us. You would also need a mapping for each column found in your datatable and each property defined in your new type.
1) So ... you eventually request an "anonymous type" for your particular situation. If that type is not currently built, just built it before, store it in your dictionary and then ... use it.
2) The remaining job is to emit the select body wich is actually simple. Use Expression.MemberInit and Expression.Bind API's. Search for them on google, you will find plenty of informations. To find out what to "Bind" using Expression.Bind, you search in the second dictionary (the mapping between column and property).
Hope this helps. I cannot give you code because I am writing from a device where i don't have visual studio.

C# Expression to return TSQL Contains keyword

I have the following code:
public Expression FilterString(string property, string Value, ParameterExpression parameter)
{
var getname = Expression.Property(parameter, property);
var toLower = Expression.Call(getname, "ToLower", null, null);
var contains = Expression.Call(toLower, "Contains", null, new[] { Expression.Constant(Value.ToString().ToLower()) });
//This will result in "LOWER(Body) LIKE '%abc123% " but I need "CONTAINS(Body, 'abc123')"
return contains;
}
var parameter = Expression.Parameter(typeof(Message), "message");
var expressionFilter = myclass.FilterString("Body","abc123", parameter);
var lambda = Expression.Lambda(expressionFilter, parameter);
//Apply lambda to EF query object
query = query.Where((Expression<Func<Message, bool>>)lambda);
This produces a TSQL LIKE statement but I'd like it to do a CONTAINS statement.
Is there anyway that I can modify the above FilterString method to make it do that?

It doesn't look like you can using vanilla EF, at least according to the source here: http://entityframework.codeplex.com/SourceControl/latest#src/EntityFramework/Core/Common/EntitySql/AST/BuiltInKind.cs
You'd have to add Contains to the AST and generate it. Here's a basic tutorial on DBProviders.

I realize this is not exactly what you want, but you should be able to get the results you want pretty easily by allowing yourself to step out of LINQ and querying the database directly:
MyDbContext.Database
.SqlQuery<MyTable>("select * from [MyTable] where contains(Body, #p0), "abc123")

Getting Count() property in Dynamic Lambda Expression

I almost got what I need but only one place where I stucked. I need to build fileCount = c.CPNDocs.Count() Dynamically in Lambda Expression. Code is below with comments what I am using to build Dynamic Lambda Expression.
var dColDefaultList = new List<String>() { "Download", "I_ID", "C_TYP", "C_LST_ACT" }; // <------- Columns I need in Lambdas Expression
ParameterExpression cParam = Expression.Parameter(typeof(CPNDBase), "c");
NewExpression newExp = Expression.New(typeof(DTDataModel));
List<MemberBinding> bindings = new List<MemberBinding>();
foreach (String sCol in dColDefaultList)
{
if (!String.Equals(sCol, "Download")) {
bindings.Add(GetMemberBinding(sCol, cParam, sCol));
}
else
{
bindings.Add(GetMemberBinding("fileCount", cParam, "CPNDocs.Count()")); // <-------need count of rows return from CPNDocs(Different Table) is a Object I recieved from Entity Relatioship
}
}
MemberInitExpression memberInitExpression = System.Linq.Expressions.Expression.MemberInit(newExp, bindings);
Expression<Func<CPNDBase, DTDataModel>> selector = (Expression<Func<CPNDBase, DTDataModel>>)BinaryExpression.Lambda(memberInitExpression, cParam);
// selector will be selector = {c => new DTDataModel() {fileCount = c.CPNDocs, I_ID = c.I_ID, C_TYP = c.C_TYP, C_LST_ACT = c.C_LST_ACT }}
// but I Need selector = {c => new DTDataModel() {fileCount = c.CPNDocs.Count(), I_ID = c.I_ID, C_TYP = c.C_TYP, C_LST_ACT = c.C_LST_ACT }}
// Question is How can I make fileCount = c.CPNDocs.Count() ?
var resultLm = finalFilteredCPNData.AsQueryable<CPNDBase>().Select(selector);
Above method is defined here :
static MemberBinding GetMemberBinding(string property, ParameterExpression param, string column)
{
MemberInfo memberInfo = typeof(DTDataModel).GetMember(property)[0];
MemberExpression memberExpression = LambdaExpression.PropertyOrField(param, column);
return System.Linq.Expressions.Expression.Bind(memberInfo, memberExpression);
}
Does anybody know how can I do this?

The Count() is not a property. It is an extension method implemented in a static class. This extension method is implemented at several places. Correct place depends on what are your classes inheriting from. To find the correct place you use the "go to definition" feature of Visual Studio.
e.g. for IQueryable.Count() the extension methods are implemented by System.Linq.Queryable static class as can be seen here → http://referencesource.microsoft.com/#System.Core/System/Linq/IQueryable.cs
So in order to encode the expression you need to encode a call to the extension method.
Much simpler way to generate expression trees from strings was shown quite early in a prototype published by Microsoft. Introductory article is available e.g. in Dynamic Expressions and Queries in LINQ
We use modified version of the original source of the automatic "string to linq" engine with success and it simplifies development a lot. By inspecting the source code of the System.Linq.Dynamic you can find exact way how to encode the expression. Link to the original source code available through NuGet is mentioned e.g. in Stack Overflow article Dynamic LINQ - Is There A .NET 4 Version?