Develop Reference

C# using ExpressionTree to map DataTable to List<T>

I have written a ToList(); extension Method to convert a DataTable to List. This just works under some circumstances but we have much old code which uses DataTables and sometimes it's needed. My Problem is that this method works with reflection what is ok but not that performant. I need about 1,2sek for 100.000 DataRows.
So i decided to build this with Expression Trees. At first i want to replace the Setter Call of Properties. Up to this time i could easily get the value:
var exactType = Nullable.GetUnderlyingType(propType) ?? propType;
var wert = Convert.ChangeType(zeile[spaltenname], exactType);
and set it:
propertyInfo.SetValue(tempObjekt, wert, null);
Now i searched StackOverflow and found this:
var zielExp = Expression.Parameter(typeof(T));
var wertExp = Expression.Parameter(propType);
var propertyExp = Expression.Property(zielExp, matchProp);
var zuweisungExp = Expression.Assign(propertyExp, wertExp);
var setter = Expression.Lambda<Action<T, int>>(zuweisungExp, zielExp, wertExp).Compile();
setter(tempObjekt, wert);
My big Problem is that the Lambda Action expects an integer. But i need this expecting the type of my Property. I have the Type of my Property via PropertyInfo. But can't get this to work. Thought i can easily make:
Action<T, object>
but this results in following excepion:
ArgumentException The ParameterExpression from Type "System.Int32"
cannot be used as Delegateparameter from Type "System.Object".
Someone out there knows a possible solution?

Instead of the generic Expression.Lambda method you can use this overload which takes a type:
public static LambdaExpression Lambda(
Type delegateType,
Expression body,
params ParameterExpression[] parameters
)
Then you can use the Type.MakeGenericType method to create the type for your action:
var actionType = typeof(Action<,>).MakeGenericType(typeof(T), proptype);
var setter = Expression.Lambda(actionType, zuweisungExp, zielExp, wertExp).Compile();
Edit following the comments regarding performance:
You can also just build the expression runtime to map the DataTable to your class of type T with a select, so there's only need to use reflection once, which should greatly improve performance. I wrote the following extension method to convert a DataTable to List<T> (note that this method will throw a runtime exception if you don't plan to map all datacolumns to a property in the class, so be sure to take care of that if that might happen):
public static class LocalExtensions
{
public static List<T> DataTableToList<T>(this DataTable table) where T : class
{
//Map the properties in a dictionary by name for easy access
var propertiesByName = typeof(T)
.GetProperties(BindingFlags.Public | BindingFlags.Instance)
.ToDictionary(p => p.Name);
var columnNames = table.Columns.Cast<DataColumn>().Select(dc => dc.ColumnName);
//The indexer property to access DataRow["columnName"] is called "Item"
var property = typeof(DataRow).GetProperties().First(p => p.Name == "Item"
&& p.GetIndexParameters().Length == 1
&& p.GetIndexParameters()[0].ParameterType == typeof(string));
var paramExpr = Expression.Parameter(typeof(DataRow), "r");
var newExpr = Expression.New(typeof(T));
//Create the expressions to map properties from your class to the corresponding
//value in the datarow. This will throw a runtime exception if your class
//doesn't contain properties for all columnnames!
var memberBindings = columnNames.Select(columnName =>
{
var pi = propertiesByName[columnName];
var indexExpr = Expression.MakeIndex(paramExpr, property,
new[] { Expression.Constant(columnName) });
//Datarow["columnName"] is of type object, cast to the right type
var convert = Expression.Convert(indexExpr, pi.PropertyType);
return Expression.Bind(pi, convert);
});
var initExpr = Expression.MemberInit(newExpr, memberBindings);
var func = Expression.Lambda<Func<DataRow, T>>(initExpr,paramExpr).Compile();
return table.Rows.Cast<DataRow>().Select(func).ToList();
}
}
Then I wrote a small testclass and some code which creates a datatable of 1,000,000 rows that get mapped to a list. Building the expression + converting to a list now only takes 486ms on my pc (granted it is a very small class of course):
class Test
{
public string TestString { get; set; }
public int TestInt { get; set; }
}
class Program
{
static void Main()
{
DataTable table = new DataTable();
table.Columns.Add(new DataColumn("TestString", typeof(string)));
table.Columns.Add(new DataColumn("TestInt", typeof(int)));
for(int i = 0; i < 1000000; i++)
{
var row = table.NewRow();
row["TestString"] = $"String number: {i}";
row["TestInt"] = i;
table.Rows.Add(row);
}
var stopwatch = Stopwatch.StartNew();
var myList = table.DataTableToList<Test>();
stopwatch.Stop();
Console.WriteLine(stopwatch.Elapsed.ToString());
}
}

I think I've understood you correctly. I cannot translate your variables so I'm taking my best guess here based on what I'm seeing in your question:
For an Action<object,object> where the first parameter is the Entity itself and the second is the type of the property you can use
var instance = Expression.Parameter(typeof(object), "i");
var argument = Expression.Parameter(typeof(object), "a");
var convertObj = Expression.TypeAs(instance, propertyInfo.DeclaringType);
var convert = Expression.Convert(argument, propertyInfo.PropertyType);
var setterCall = Expression.Call(convertObj, propertyInfo.GetSetMethod(), convert);
var compiled = ((Expression<Action<object, object>>) Expression.Lambda(setterCall, instance, argument)).Compile();
If you know T (ie, the type of the Entity), you can do this instead:
var instance = Expression.Parameter(typeof(T), "i");
var argument = Expression.Parameter(typeof(object), "a");
var convert = Expression.Convert(argument, propertyInfo.PropertyType);
var setterCall = Expression.Call(instance , propertyInfo.GetSetMethod(), convert);
var compiled = ((Expression<Action<T, object>>) Expression.Lambda(setterCall, instance, argument)).Compile();

I comment here because I do not have the necessary reputation to comment on the response of #Alexander Derek
var memberBindings = columnNames.Select(columnName =>
{
var pi = propertiesByName[columnName];
var indexExpr = Expression.MakeIndex(paramExpr, property,
new[] { Expression.Constant(columnName) });
//Datarow["columnName"] is of type object, cast to the right type
var convert = Expression.Convert(indexExpr, pi.PropertyType);
return Expression.Bind(pi, convert);
});
in order to avoid runtime exception i added a try-catch and .where()
var memberBindings = columnNames.Select(columnName =>
{
try
{
var pi = propertiesByName[columnName];
var indexExpr = Expression.MakeIndex(paramExpr, property,
new[] { Expression.Constant(columnName) });
var convert = Expression.Convert(indexExpr, pi.PropertyType);
return Expression.Bind(pi, convert);
}
catch(Exception e)
{
return null;
}
});
var initExpr = Expression.MemberInit(newExpr, memberBindings.Where(obj => obj != null));

add where clauses to linq query with generic column name

I want to add where clauses to a linq query only if there is the column on the generic type table.
code example :
this function is generic for all tables in model. I want to add where condition for all the tabels that have "AccountId" column.
public IQueryable RetrieveAll(params Expression>[] eagerProperties) {
var entitySet = ResolveEntitySet(typeof(T));
var query = context.CreateQuery<T>(entitySet);
foreach (var e in eagerProperties)
{
query = query.Expand(e);
}
var type = typeof(T);
var account = type.GetProperty("AccountId");
if(account!=null)
{
query = query.where(x=>x...)
}
return query
I need something like
Guid g = new Guid("3252353h....")
query.where(x=>x.AccountId == g)
Thanks

You must create the Where predicate dynamically using Expression Tree, look code below:
public static IQueryable<T> RetrieveAll<T>(params Expression[] eagerProperties)
{
var type = typeof(T);
var entitySet = ResolveEntitySet(type);
var query = context.CreateQuery<T>(entitySet);
foreach (var e in eagerProperties)
{
query = query.Expand(e);
}
var account = type.GetProperty("AccountId");
if (account != null)
{
Guid g = new Guid("3252353h....");
var parameter = Expression.Parameter(type);
var property = Expression.Property(parameter, account);
var guidValue = Expression.Constant(g);
var lambdaPredicate = Expression.Lambda<Func<T, bool>>(Expression.Equal(property, guidValue), parameter);
return query.Where(lambdaPredicate);
}
return query;
}

Dynamic LINQ for RUntime Column Projection

Im trying to implement Dynamic LINQ Query for selected columns to show has Output columns in LINQ Query.
Here is the error:
/Property 'System.String CompanyCode' is not defined for type
'System.String'"
public static void SelectProjection()
{
DataMovementDataContext dbMovement = new DataMovementDataContext();
var entity = dbMovement.ListofAccountingDocs2_1075s.AsQueryable();
Type type = entity.ElementType;
var entityParam = Expression.Parameter(entity.ElementType, "row");
Expression expr = entityParam;
string[] props = "AccountingDocumentNbr,CompanyCode,FiscalYearNbr".Split(',');
foreach (string prop in props)
{
// use reflection (not ComponentModel) to mirror LINQ
PropertyInfo pi = type.GetProperty(prop);
expr = Expression.Property(expr, pi);
type = pi.PropertyType;
}
// row => row.Property
// var columnLambda = Expression.Lambda( Expression.Property(entityParam, "GLCompanyCode"), entityParam);
var columnLambda = Expression.Lambda(Expression.Property(expr, "AccountingDocumentNbr,GLCompanyCode"), entityParam);
// Items.Select(row => row.Property)
var selectCall = Expression.Call(typeof(Queryable), "Select", new Type[] { entity.ElementType, columnLambda.Body.Type }, entity.Expression, columnLambda);
// Items.Select(row => row.Property).Distinct
var distinctCall = Expression.Call(typeof(Queryable), "Distinct", new Type[] { typeof(string) }, selectCall);
// colvalue => colvalue
var sortParam = Expression.Parameter(typeof(string), "AccountingDocumentNbr");
var columnResultLambda = Expression.Lambda(sortParam, sortParam);
// Items.Select(row => row.Property).Distinct.OrderBy(colvalue => colvalue)
var ordercall = Expression.Call(typeof(Queryable), "OrderBy",
new Type[] { typeof(string), columnResultLambda.Body.Type },
distinctCall, columnResultLambda);
var result = entity.Provider.CreateQuery(ordercall);
foreach (var item in result)
{
Console.Write(item);
}
}
Can any one provide me help in solving above error?

If we look at this code:
// I chnaged this part:
string[] props = new string[]
{
"AccountingDocumentNbr",
"CompanyCode",
"FiscalYearNbr"
};
Expression expr;
Type type = entity.ElementType;
foreach (string prop in props)
{
PropertyInfo pi = type.GetProperty(prop);
expr = Expression.Property(expr, pi);
type = pi.PropertyType; // This line may cause your loop
// to do something you dont want to do?
}
It seems to me that your loop going down the properties, gets a string and then searches for a prop called CompanyCode, which the class String obviously doesnt have.
Your loop does the following at the moment:
from type get the property named AccountingDocumentNbr
from the return type of property AccountingDocumentNbr get the property called CompanyCode
form the return type of property CompanyCode get the type of the property called FiscalYearNbr
I hardly doubt this is what you really want to do.

Expression to create an instance with object initializer

Is there any way to create an instance of an object with object initializer with an Expression Tree? I mean create an Expression Tree to build this lambda:
// my class
public class MyObject {
public bool DisplayValue { get; set; }
}
// my lambda:
var lambda = (Func<bool, MyObject>)
(displayValue => new MyObject { DisplayValue = displayValue });
How can I create this lambda with an Expression Tree?
UPDATE:
I tryed myself and write following code:
public static Func<bool, dynamic> Creator;
static void BuildLambda() {
var expectedType = typeof(MyObject);
var displayValueParam = Expression.Parameter(typeof(bool), "displayValue");
var ctor = Expression.New(expectedType);
var local = Expression.Parameter(expectedType, "obj");
var displayValueProperty = Expression.Property(ctor, "DisplayValue");
var returnTarget = Expression.Label(expectedType);
var returnExpression = Expression.Return(returnTarget,local, expectedType);
var returnLabel = Expression.Label(returnTarget, Expression.Default(expectedType));
var block = Expression.Block(
new[] { local },
Expression.Assign(local, ctor),
Expression.Assign(displayValueProperty, displayValueParam),
Expression.Return(Expression.Label(expectedType), local, expectedType),
returnExpression,
returnLabel
);
Creator =
Expression.Lambda<Func<bool, dynamic>>(block, displayValueParam)
.Compile();
}
But it throws the following error:
Cannot jump to undefined label ''.
Can everybody help me please?

To represent object initializers in an Expression, you should use Expression.MemberInit():
Expression<Func<bool, MyObject>> BuildLambda() {
var createdType = typeof(MyObject);
var displayValueParam = Expression.Parameter(typeof(bool), "displayValue");
var ctor = Expression.New(createdType);
var displayValueProperty = createdType.GetProperty("DisplayValue");
var displayValueAssignment = Expression.Bind(
displayValueProperty, displayValueParam);
var memberInit = Expression.MemberInit(ctor, displayValueAssignment);
return
Expression.Lambda<Func<bool, MyObject>>(memberInit, displayValueParam);
}
To verify this actually does what you want, you can call ToString() on the created expression. In this case, the output is as expected:
displayValue => new MyObject() {DisplayValue = displayValue}

Finally I found my answer:
public static Func<bool, dynamic> Creator;
static void BuildLambda() {
var expectedType = typeof(MyObject);
var displayValueParam = Expression.Parameter(typeof(bool), "displayValue");
var ctor = Expression.New(expectedType);
var local = Expression.Parameter(expectedType, "obj");
var displayValueProperty = Expression.Property(local, "DisplayValue");
var returnTarget = Expression.Label(expectedType);
var returnExpression = Expression.Return(returnTarget,local, expectedType);
var returnLabel = Expression.Label(returnTarget, Expression.Default(expectedType));
var block = Expression.Block(
new[] { local },
Expression.Assign(local, ctor),
Expression.Assign(displayValueProperty, displayValueParam),
/* I forgot to remove this line:
* Expression.Return(Expression.Label(expectedType), local, expectedType),
* and now it works.
* */
returnExpression,
returnLabel
);
Creator =
Expression.Lambda<Func<bool, dynamic>>(block, displayValueParam)
.Compile();
}
UPDATE:
While it works fine, but #svick provide a better and shorter way in his answer that is actuallt wath I was looking for: MemberInit. Please see #svick's answer.

How do i expand on this expression?

I have this bit of code as an example, basically it spits out
p => p.fieldname.StartsWith("123")
But who would i expand on this to do something like this:
p => p.anotherentity.fieldname.StartsWith("123")
Here is a sample of the code i am have refactored for own needs:
string propertyName = "FirstName";
string methodName = "StartsWith";
string keyword = "123";
Type t = typeof (Person);
ParameterExpression paramExp = Expression.Parameter(t, "p");
// the parameter: p
MemberExpression memberExp = Expression.MakeMemberAccess(paramExp,
t.GetMember(propertyName).FirstOrDefault());
// part of the body: p.FirstName
MethodCallExpression callExp = Expression.Call(memberExp,
typeof (string).GetMethod(methodName,
new Type[] {typeof (string)}),
Expression.Constant(keyword));
// the body: p.FirstName.StartsWith("123")
Expression<Func<Person, bool>> whereExp = Expression.Lambda<Func<Person, bool>>(callExp, paramExp);
Expression<Func<Person, string>> selectExp = Expression.Lambda<Func<Person, string>>(memberExp, paramExp);
Console.WriteLine(whereExp); // p => p.FirstName.StartsWith("123")
Console.WriteLine(selectExp); // p => p.FirstName
To further explain let me show you what i would like to do:
public class Person
{
public string IdentityCode {get;set;}
public Loans Loans {get;set;}
}
public class Loans
{
public int Id {get;set;}
public Asset Assets {get;set;}
public Person person {get;set;}
}
public class Asset
{
public string SerialNumber {get;set;}
}
Then using an expression build something like this:
p => p.Loans.Asset.SerialNumber.StartsWith("123)
Or
p => p.Loans.Person.IdentityCode.StartsWith("123")

untested, but...
ParameterExpression paramExp = Expression.Parameter(t, "p"); // the parameter: p
MemberExpression memberExp =
Expression.MakeMemberAccess(paramExp, t.GetMember(propertyName).FirstOrDefault());
would become something like:
ParameterExpression paramExp = Expression.Parameter(t, "p"); // the parameter: p
MemberExpression otherEntityExp =
Expression.MakeMemberAccess(paramExp, t.GetMember("anotherentity").FirstOrDefault());
MemberExpression memberExp =
Expression.MakeMemberAccess(otherEntityExp, t.GetMember(propertyName).FirstOrDefault());

I'm not sure what you're asking for, updating an expression or building one from scratch...
If you already have the existing, old expression and want to update it, it would be very easy to create a new one. The idea is to dig through the expression tree down to the expression you want to replace. Then update all parent expressions with the newly replaced one.
Expression<Func<Obj, bool>> expr = p => p.fieldname.StartsWith("123");
var body = expr.Body as MethodCallExpression; // *.StartsWith()
var obj = body.Object as MemberExpression; // p.fieldname
var param = expr.Parameters.First(); // p
var newAccess = Expression.PropertyOrField(param, "anotherentity"); // p.anotherentity
var newObj = obj.Update(newAccess); // update obj
var newBody = body.Update(newObj, body.Arguments); // update body
var newExpr = expr.Update(newBody, expr.Parameters);// update expr
Otherwise to build up the expression tree:
Expression<Func<Person, bool>> expr =
p => p.Loans.Asset.SerialNumber.StartsWith("123");
Work it out from the beginning.
var p = Expression.Parameter(typeof(Person), "p");
var accessLoans = Expression.PropertyOrField(p, "Loans");
var accessAsset = Expression.PropertyOrField(accessLoans, "Asset");
var accessSerialNumber = Expression.PropertyOrField(accessAsset, "SerialNumber");
var callArgs = new Expression[] { Expression.Constant("123", typeof(string)) };
var callStartsWith = Expression.Call(accessSerialNumber, "StartsWith", null, callArgs);
var newExpr = Expression.Lambda<Func<Person, bool>>(callStartsWith, p);
I'll leave the last one as an exercise for you.