Develop Reference

C# using ExpressionTree to map DataTable to List<T>

I have written a ToList(); extension Method to convert a DataTable to List. This just works under some circumstances but we have much old code which uses DataTables and sometimes it's needed. My Problem is that this method works with reflection what is ok but not that performant. I need about 1,2sek for 100.000 DataRows.
So i decided to build this with Expression Trees. At first i want to replace the Setter Call of Properties. Up to this time i could easily get the value:
var exactType = Nullable.GetUnderlyingType(propType) ?? propType;
var wert = Convert.ChangeType(zeile[spaltenname], exactType);
and set it:
propertyInfo.SetValue(tempObjekt, wert, null);
Now i searched StackOverflow and found this:
var zielExp = Expression.Parameter(typeof(T));
var wertExp = Expression.Parameter(propType);
var propertyExp = Expression.Property(zielExp, matchProp);
var zuweisungExp = Expression.Assign(propertyExp, wertExp);
var setter = Expression.Lambda<Action<T, int>>(zuweisungExp, zielExp, wertExp).Compile();
setter(tempObjekt, wert);
My big Problem is that the Lambda Action expects an integer. But i need this expecting the type of my Property. I have the Type of my Property via PropertyInfo. But can't get this to work. Thought i can easily make:
Action<T, object>
but this results in following excepion:
ArgumentException The ParameterExpression from Type "System.Int32"
cannot be used as Delegateparameter from Type "System.Object".
Someone out there knows a possible solution?

Instead of the generic Expression.Lambda method you can use this overload which takes a type:
public static LambdaExpression Lambda(
Type delegateType,
Expression body,
params ParameterExpression[] parameters
)
Then you can use the Type.MakeGenericType method to create the type for your action:
var actionType = typeof(Action<,>).MakeGenericType(typeof(T), proptype);
var setter = Expression.Lambda(actionType, zuweisungExp, zielExp, wertExp).Compile();
Edit following the comments regarding performance:
You can also just build the expression runtime to map the DataTable to your class of type T with a select, so there's only need to use reflection once, which should greatly improve performance. I wrote the following extension method to convert a DataTable to List<T> (note that this method will throw a runtime exception if you don't plan to map all datacolumns to a property in the class, so be sure to take care of that if that might happen):
public static class LocalExtensions
{
public static List<T> DataTableToList<T>(this DataTable table) where T : class
{
//Map the properties in a dictionary by name for easy access
var propertiesByName = typeof(T)
.GetProperties(BindingFlags.Public | BindingFlags.Instance)
.ToDictionary(p => p.Name);
var columnNames = table.Columns.Cast<DataColumn>().Select(dc => dc.ColumnName);
//The indexer property to access DataRow["columnName"] is called "Item"
var property = typeof(DataRow).GetProperties().First(p => p.Name == "Item"
&& p.GetIndexParameters().Length == 1
&& p.GetIndexParameters()[0].ParameterType == typeof(string));
var paramExpr = Expression.Parameter(typeof(DataRow), "r");
var newExpr = Expression.New(typeof(T));
//Create the expressions to map properties from your class to the corresponding
//value in the datarow. This will throw a runtime exception if your class
//doesn't contain properties for all columnnames!
var memberBindings = columnNames.Select(columnName =>
{
var pi = propertiesByName[columnName];
var indexExpr = Expression.MakeIndex(paramExpr, property,
new[] { Expression.Constant(columnName) });
//Datarow["columnName"] is of type object, cast to the right type
var convert = Expression.Convert(indexExpr, pi.PropertyType);
return Expression.Bind(pi, convert);
});
var initExpr = Expression.MemberInit(newExpr, memberBindings);
var func = Expression.Lambda<Func<DataRow, T>>(initExpr,paramExpr).Compile();
return table.Rows.Cast<DataRow>().Select(func).ToList();
}
}
Then I wrote a small testclass and some code which creates a datatable of 1,000,000 rows that get mapped to a list. Building the expression + converting to a list now only takes 486ms on my pc (granted it is a very small class of course):
class Test
{
public string TestString { get; set; }
public int TestInt { get; set; }
}
class Program
{
static void Main()
{
DataTable table = new DataTable();
table.Columns.Add(new DataColumn("TestString", typeof(string)));
table.Columns.Add(new DataColumn("TestInt", typeof(int)));
for(int i = 0; i < 1000000; i++)
{
var row = table.NewRow();
row["TestString"] = $"String number: {i}";
row["TestInt"] = i;
table.Rows.Add(row);
}
var stopwatch = Stopwatch.StartNew();
var myList = table.DataTableToList<Test>();
stopwatch.Stop();
Console.WriteLine(stopwatch.Elapsed.ToString());
}
}

I think I've understood you correctly. I cannot translate your variables so I'm taking my best guess here based on what I'm seeing in your question:
For an Action<object,object> where the first parameter is the Entity itself and the second is the type of the property you can use
var instance = Expression.Parameter(typeof(object), "i");
var argument = Expression.Parameter(typeof(object), "a");
var convertObj = Expression.TypeAs(instance, propertyInfo.DeclaringType);
var convert = Expression.Convert(argument, propertyInfo.PropertyType);
var setterCall = Expression.Call(convertObj, propertyInfo.GetSetMethod(), convert);
var compiled = ((Expression<Action<object, object>>) Expression.Lambda(setterCall, instance, argument)).Compile();
If you know T (ie, the type of the Entity), you can do this instead:
var instance = Expression.Parameter(typeof(T), "i");
var argument = Expression.Parameter(typeof(object), "a");
var convert = Expression.Convert(argument, propertyInfo.PropertyType);
var setterCall = Expression.Call(instance , propertyInfo.GetSetMethod(), convert);
var compiled = ((Expression<Action<T, object>>) Expression.Lambda(setterCall, instance, argument)).Compile();

I comment here because I do not have the necessary reputation to comment on the response of #Alexander Derek
var memberBindings = columnNames.Select(columnName =>
{
var pi = propertiesByName[columnName];
var indexExpr = Expression.MakeIndex(paramExpr, property,
new[] { Expression.Constant(columnName) });
//Datarow["columnName"] is of type object, cast to the right type
var convert = Expression.Convert(indexExpr, pi.PropertyType);
return Expression.Bind(pi, convert);
});
in order to avoid runtime exception i added a try-catch and .where()
var memberBindings = columnNames.Select(columnName =>
{
try
{
var pi = propertiesByName[columnName];
var indexExpr = Expression.MakeIndex(paramExpr, property,
new[] { Expression.Constant(columnName) });
var convert = Expression.Convert(indexExpr, pi.PropertyType);
return Expression.Bind(pi, convert);
}
catch(Exception e)
{
return null;
}
});
var initExpr = Expression.MemberInit(newExpr, memberBindings.Where(obj => obj != null));

How to create linq WHERE IN statement without using Contains()

I have a situation where I am using a linq provider that does not support the .Contains method to generate a WHERE IN clause in the query. I am looking for a way to generate the (Value = X OR Value = Y OR Value = Z) statement dynamically from the list of items to match. I haven't found a good example of building expression trees to do this.
Normally I would query this way:
var names = new string[] { "name1", "name2", "name3" }
var matches = query.Where(x => names.Contains(x.Name));
So far the closest thing I could find was to use the
Dynamic Linq Library and build a string to be interpreted but it feels a bit too hacky.

You don't even need an external library for something like this:
var names = new string[] { "name1", "name2", "name3" };
// Where MyClass is the type of your class
ParameterExpression par = Expression.Parameter(typeof(MyClass));
MemberExpression prop = Expression.Property(par, "Name");
Expression expression = null;
foreach (string name in names)
{
Expression expression2 = Expression.Equal(prop, Expression.Constant(name));
if (expression == null)
{
expression = expression2;
}
else
{
expression = Expression.OrElse(expression, expression2);
}
}
var query = ...; // Your query
if (expression != null)
{
// Where MyClass is the type of your class
var lambda = Expression.Lambda<Func<MyClass, bool>>(expression, par);
query = query.Where(lambda);
}
You can build a concatenation of Expression.OrElse, with the comparisons between the property Name and one of the strings.
In this particular case (3 strings), the resulting Expression, when looked from the debugger, is:
(((Param_0.Name == "name1") OrElse (Param_0.Name == "name2")) OrElse (Param_0.Name == "name3"))

Just improving the answer by xanatos:
var names = new string[] { "name1", "name2", "name3" };
var query = ...; // Your query
if (names.Any())
{
// Where MyClass is the type of your class
ParameterExpression par = Expression.Parameter(typeof(MyClass));
MemberExpression prop = Expression.Property(par, "Name");
var expression=names
.Select(v => Expression.Equal(prop, Expression.Constant(v)))
.Aggregate(Expression.OrElse);
// Where MyClass is the type of your class
var lambda = Expression.Lambda<Func<MyClass, bool>>(expression, par);
query = query.Where(lambda);
}

LINQ Delegate Lambda Select Statement Bind Child Property

I am creating a delegate for a Select statement in LINQ. Some of the property bindings are to child properties on the object I'm selecting from.
This is the LINQ statement I want to put in my delegate:
var list = dataSet.Select(x => new ViewModel()
{
Name = x.Name,
ClassType = x.ClassType.Description
};
I can get the Name no worries with my code, but I do not know how to get the ClassType.Description.
Here is my current code:
protected Func<Student, ManagerStudentListViewModel> GetSelectStatement()
{
var studentType = typeof(Student);
var viewModelType = typeof(ManagerStudentListViewModel);
var parameterExpression = Expression.Parameter(studentType, "x");
var newInstantiationExpression = Expression.New(viewModelType);
// Name Binding
var viewModelProperty = viewModelType.GetProperty("Name");
var studentProperty = studentType.GetProperty("Name");
var nameMemberExpression = Expression.Property(parameterExpression, studentProperty);
var nameBinding = Expression.Bind(viewModelProperty, nameMemberExpression);
// ClassType.Description Binding
// ???
var bindings = new List<MemberAssignment>() { nameBinding, classTypeBinding };
var memberInitExpression = Expression.MemberInit(newInstantiationExpression, bindings);
var lambda = Expression.Lambda<Func<Student, ManagerStudentListViewModel>>(memberInitExpression, parameterExpression);
return lambda.Compile();
}

Accessing deeply nested members is no different than accessing any other properties, provided you know the name of the members. Just create an expression to get the first property, then add the expression to get the second.
Expression<Func<Student, ManagerStudentListViewModel>> GetSelectStatement()
{
var studentType = typeof(Student);
var viewModelType = typeof(ManagerStudentListViewModel);
var param = Expression.Parameter(studentType, "x");
var nameValue = Expression.Property(param, "Name");
var classTypeValue = Expression.Property(
Expression.Property(param, "ClassType"), // get the class type
"Description"); // get the description of the class type
var nameMemberBinding = Expression.Bind(
viewModelType.GetProperty("Name"),
nameValue);
var classTypeMemberBinding = Expression.Bind(
viewModelType.GetProperty("ClassType"),
classTypeValue);
var initializer = Expression.MemberInit(
Expression.New(viewModelType),
nameMemberBinding,
classTypeMemberBinding);
return Expression.Lambda<Func<Student, ManagerStudentListViewModel>>(initializer, param);
}

Dynamic LINQ for RUntime Column Projection

Im trying to implement Dynamic LINQ Query for selected columns to show has Output columns in LINQ Query.
Here is the error:
/Property 'System.String CompanyCode' is not defined for type
'System.String'"
public static void SelectProjection()
{
DataMovementDataContext dbMovement = new DataMovementDataContext();
var entity = dbMovement.ListofAccountingDocs2_1075s.AsQueryable();
Type type = entity.ElementType;
var entityParam = Expression.Parameter(entity.ElementType, "row");
Expression expr = entityParam;
string[] props = "AccountingDocumentNbr,CompanyCode,FiscalYearNbr".Split(',');
foreach (string prop in props)
{
// use reflection (not ComponentModel) to mirror LINQ
PropertyInfo pi = type.GetProperty(prop);
expr = Expression.Property(expr, pi);
type = pi.PropertyType;
}
// row => row.Property
// var columnLambda = Expression.Lambda( Expression.Property(entityParam, "GLCompanyCode"), entityParam);
var columnLambda = Expression.Lambda(Expression.Property(expr, "AccountingDocumentNbr,GLCompanyCode"), entityParam);
// Items.Select(row => row.Property)
var selectCall = Expression.Call(typeof(Queryable), "Select", new Type[] { entity.ElementType, columnLambda.Body.Type }, entity.Expression, columnLambda);
// Items.Select(row => row.Property).Distinct
var distinctCall = Expression.Call(typeof(Queryable), "Distinct", new Type[] { typeof(string) }, selectCall);
// colvalue => colvalue
var sortParam = Expression.Parameter(typeof(string), "AccountingDocumentNbr");
var columnResultLambda = Expression.Lambda(sortParam, sortParam);
// Items.Select(row => row.Property).Distinct.OrderBy(colvalue => colvalue)
var ordercall = Expression.Call(typeof(Queryable), "OrderBy",
new Type[] { typeof(string), columnResultLambda.Body.Type },
distinctCall, columnResultLambda);
var result = entity.Provider.CreateQuery(ordercall);
foreach (var item in result)
{
Console.Write(item);
}
}
Can any one provide me help in solving above error?

If we look at this code:
// I chnaged this part:
string[] props = new string[]
{
"AccountingDocumentNbr",
"CompanyCode",
"FiscalYearNbr"
};
Expression expr;
Type type = entity.ElementType;
foreach (string prop in props)
{
PropertyInfo pi = type.GetProperty(prop);
expr = Expression.Property(expr, pi);
type = pi.PropertyType; // This line may cause your loop
// to do something you dont want to do?
}
It seems to me that your loop going down the properties, gets a string and then searches for a prop called CompanyCode, which the class String obviously doesnt have.
Your loop does the following at the moment:
from type get the property named AccountingDocumentNbr
from the return type of property AccountingDocumentNbr get the property called CompanyCode
form the return type of property CompanyCode get the type of the property called FiscalYearNbr
I hardly doubt this is what you really want to do.

How do i expand on this expression?

I have this bit of code as an example, basically it spits out
p => p.fieldname.StartsWith("123")
But who would i expand on this to do something like this:
p => p.anotherentity.fieldname.StartsWith("123")
Here is a sample of the code i am have refactored for own needs:
string propertyName = "FirstName";
string methodName = "StartsWith";
string keyword = "123";
Type t = typeof (Person);
ParameterExpression paramExp = Expression.Parameter(t, "p");
// the parameter: p
MemberExpression memberExp = Expression.MakeMemberAccess(paramExp,
t.GetMember(propertyName).FirstOrDefault());
// part of the body: p.FirstName
MethodCallExpression callExp = Expression.Call(memberExp,
typeof (string).GetMethod(methodName,
new Type[] {typeof (string)}),
Expression.Constant(keyword));
// the body: p.FirstName.StartsWith("123")
Expression<Func<Person, bool>> whereExp = Expression.Lambda<Func<Person, bool>>(callExp, paramExp);
Expression<Func<Person, string>> selectExp = Expression.Lambda<Func<Person, string>>(memberExp, paramExp);
Console.WriteLine(whereExp); // p => p.FirstName.StartsWith("123")
Console.WriteLine(selectExp); // p => p.FirstName
To further explain let me show you what i would like to do:
public class Person
{
public string IdentityCode {get;set;}
public Loans Loans {get;set;}
}
public class Loans
{
public int Id {get;set;}
public Asset Assets {get;set;}
public Person person {get;set;}
}
public class Asset
{
public string SerialNumber {get;set;}
}
Then using an expression build something like this:
p => p.Loans.Asset.SerialNumber.StartsWith("123)
Or
p => p.Loans.Person.IdentityCode.StartsWith("123")

untested, but...
ParameterExpression paramExp = Expression.Parameter(t, "p"); // the parameter: p
MemberExpression memberExp =
Expression.MakeMemberAccess(paramExp, t.GetMember(propertyName).FirstOrDefault());
would become something like:
ParameterExpression paramExp = Expression.Parameter(t, "p"); // the parameter: p
MemberExpression otherEntityExp =
Expression.MakeMemberAccess(paramExp, t.GetMember("anotherentity").FirstOrDefault());
MemberExpression memberExp =
Expression.MakeMemberAccess(otherEntityExp, t.GetMember(propertyName).FirstOrDefault());

I'm not sure what you're asking for, updating an expression or building one from scratch...
If you already have the existing, old expression and want to update it, it would be very easy to create a new one. The idea is to dig through the expression tree down to the expression you want to replace. Then update all parent expressions with the newly replaced one.
Expression<Func<Obj, bool>> expr = p => p.fieldname.StartsWith("123");
var body = expr.Body as MethodCallExpression; // *.StartsWith()
var obj = body.Object as MemberExpression; // p.fieldname
var param = expr.Parameters.First(); // p
var newAccess = Expression.PropertyOrField(param, "anotherentity"); // p.anotherentity
var newObj = obj.Update(newAccess); // update obj
var newBody = body.Update(newObj, body.Arguments); // update body
var newExpr = expr.Update(newBody, expr.Parameters);// update expr
Otherwise to build up the expression tree:
Expression<Func<Person, bool>> expr =
p => p.Loans.Asset.SerialNumber.StartsWith("123");
Work it out from the beginning.
var p = Expression.Parameter(typeof(Person), "p");
var accessLoans = Expression.PropertyOrField(p, "Loans");
var accessAsset = Expression.PropertyOrField(accessLoans, "Asset");
var accessSerialNumber = Expression.PropertyOrField(accessAsset, "SerialNumber");
var callArgs = new Expression[] { Expression.Constant("123", typeof(string)) };
var callStartsWith = Expression.Call(accessSerialNumber, "StartsWith", null, callArgs);
var newExpr = Expression.Lambda<Func<Person, bool>>(callStartsWith, p);
I'll leave the last one as an exercise for you.