Develop Reference

C# using ExpressionTree to map DataTable to List<T>

I have written a ToList(); extension Method to convert a DataTable to List. This just works under some circumstances but we have much old code which uses DataTables and sometimes it's needed. My Problem is that this method works with reflection what is ok but not that performant. I need about 1,2sek for 100.000 DataRows.
So i decided to build this with Expression Trees. At first i want to replace the Setter Call of Properties. Up to this time i could easily get the value:
var exactType = Nullable.GetUnderlyingType(propType) ?? propType;
var wert = Convert.ChangeType(zeile[spaltenname], exactType);
and set it:
propertyInfo.SetValue(tempObjekt, wert, null);
Now i searched StackOverflow and found this:
var zielExp = Expression.Parameter(typeof(T));
var wertExp = Expression.Parameter(propType);
var propertyExp = Expression.Property(zielExp, matchProp);
var zuweisungExp = Expression.Assign(propertyExp, wertExp);
var setter = Expression.Lambda<Action<T, int>>(zuweisungExp, zielExp, wertExp).Compile();
setter(tempObjekt, wert);
My big Problem is that the Lambda Action expects an integer. But i need this expecting the type of my Property. I have the Type of my Property via PropertyInfo. But can't get this to work. Thought i can easily make:
Action<T, object>
but this results in following excepion:
ArgumentException The ParameterExpression from Type "System.Int32"
cannot be used as Delegateparameter from Type "System.Object".
Someone out there knows a possible solution?

Instead of the generic Expression.Lambda method you can use this overload which takes a type:
public static LambdaExpression Lambda(
Type delegateType,
Expression body,
params ParameterExpression[] parameters
)
Then you can use the Type.MakeGenericType method to create the type for your action:
var actionType = typeof(Action<,>).MakeGenericType(typeof(T), proptype);
var setter = Expression.Lambda(actionType, zuweisungExp, zielExp, wertExp).Compile();
Edit following the comments regarding performance:
You can also just build the expression runtime to map the DataTable to your class of type T with a select, so there's only need to use reflection once, which should greatly improve performance. I wrote the following extension method to convert a DataTable to List<T> (note that this method will throw a runtime exception if you don't plan to map all datacolumns to a property in the class, so be sure to take care of that if that might happen):
public static class LocalExtensions
{
public static List<T> DataTableToList<T>(this DataTable table) where T : class
{
//Map the properties in a dictionary by name for easy access
var propertiesByName = typeof(T)
.GetProperties(BindingFlags.Public | BindingFlags.Instance)
.ToDictionary(p => p.Name);
var columnNames = table.Columns.Cast<DataColumn>().Select(dc => dc.ColumnName);
//The indexer property to access DataRow["columnName"] is called "Item"
var property = typeof(DataRow).GetProperties().First(p => p.Name == "Item"
&& p.GetIndexParameters().Length == 1
&& p.GetIndexParameters()[0].ParameterType == typeof(string));
var paramExpr = Expression.Parameter(typeof(DataRow), "r");
var newExpr = Expression.New(typeof(T));
//Create the expressions to map properties from your class to the corresponding
//value in the datarow. This will throw a runtime exception if your class
//doesn't contain properties for all columnnames!
var memberBindings = columnNames.Select(columnName =>
{
var pi = propertiesByName[columnName];
var indexExpr = Expression.MakeIndex(paramExpr, property,
new[] { Expression.Constant(columnName) });
//Datarow["columnName"] is of type object, cast to the right type
var convert = Expression.Convert(indexExpr, pi.PropertyType);
return Expression.Bind(pi, convert);
});
var initExpr = Expression.MemberInit(newExpr, memberBindings);
var func = Expression.Lambda<Func<DataRow, T>>(initExpr,paramExpr).Compile();
return table.Rows.Cast<DataRow>().Select(func).ToList();
}
}
Then I wrote a small testclass and some code which creates a datatable of 1,000,000 rows that get mapped to a list. Building the expression + converting to a list now only takes 486ms on my pc (granted it is a very small class of course):
class Test
{
public string TestString { get; set; }
public int TestInt { get; set; }
}
class Program
{
static void Main()
{
DataTable table = new DataTable();
table.Columns.Add(new DataColumn("TestString", typeof(string)));
table.Columns.Add(new DataColumn("TestInt", typeof(int)));
for(int i = 0; i < 1000000; i++)
{
var row = table.NewRow();
row["TestString"] = $"String number: {i}";
row["TestInt"] = i;
table.Rows.Add(row);
}
var stopwatch = Stopwatch.StartNew();
var myList = table.DataTableToList<Test>();
stopwatch.Stop();
Console.WriteLine(stopwatch.Elapsed.ToString());
}
}

I think I've understood you correctly. I cannot translate your variables so I'm taking my best guess here based on what I'm seeing in your question:
For an Action<object,object> where the first parameter is the Entity itself and the second is the type of the property you can use
var instance = Expression.Parameter(typeof(object), "i");
var argument = Expression.Parameter(typeof(object), "a");
var convertObj = Expression.TypeAs(instance, propertyInfo.DeclaringType);
var convert = Expression.Convert(argument, propertyInfo.PropertyType);
var setterCall = Expression.Call(convertObj, propertyInfo.GetSetMethod(), convert);
var compiled = ((Expression<Action<object, object>>) Expression.Lambda(setterCall, instance, argument)).Compile();
If you know T (ie, the type of the Entity), you can do this instead:
var instance = Expression.Parameter(typeof(T), "i");
var argument = Expression.Parameter(typeof(object), "a");
var convert = Expression.Convert(argument, propertyInfo.PropertyType);
var setterCall = Expression.Call(instance , propertyInfo.GetSetMethod(), convert);
var compiled = ((Expression<Action<T, object>>) Expression.Lambda(setterCall, instance, argument)).Compile();

I comment here because I do not have the necessary reputation to comment on the response of #Alexander Derek
var memberBindings = columnNames.Select(columnName =>
{
var pi = propertiesByName[columnName];
var indexExpr = Expression.MakeIndex(paramExpr, property,
new[] { Expression.Constant(columnName) });
//Datarow["columnName"] is of type object, cast to the right type
var convert = Expression.Convert(indexExpr, pi.PropertyType);
return Expression.Bind(pi, convert);
});
in order to avoid runtime exception i added a try-catch and .where()
var memberBindings = columnNames.Select(columnName =>
{
try
{
var pi = propertiesByName[columnName];
var indexExpr = Expression.MakeIndex(paramExpr, property,
new[] { Expression.Constant(columnName) });
var convert = Expression.Convert(indexExpr, pi.PropertyType);
return Expression.Bind(pi, convert);
}
catch(Exception e)
{
return null;
}
});
var initExpr = Expression.MemberInit(newExpr, memberBindings.Where(obj => obj != null));

Looping through members of a defined concrete type?

I'm doing comparisons across databases (with about 20 fields) and I've defined a concrete type to handle the comparison.
If my comparison fails, I want to loop through the individual items in the catch block and provide the user a list errors. Below, I've done it manually through the first few variables. Is there a more efficient way to loop this through all 20 fields? I started with a foreach (Object objectItem .. but not sure if that's the right way to go.
Any thoughts or much needed guidance?
try {
CollectionAssert.IsSubsetOf(orgs, members, "Error Matching testlist Fields");
}
catch
{
//OrgID
var sourceOrgID = orgs.Select(o => o.OrgID);
var destOrgID = members.Select(o => o.OrgID);
var errorList1 = sourceOrgID.Except(destOrgID);
string failedTests = null;
failedTests = string.Join("\n", errorList1);
Assert.IsTrue(0 == failedTests.Length, "The following Org IDs are not contained in the source: \n" + failedTests);
//DealerCode
var sourceDealerCode = orgs.Select(o => o.DealerCode);
var destDealerCode = members.Select(o => o.DealerCode);
var errorList2 = sourceDealerCode.Except(destDealerCode);
failedTests = null;
failedTests = string.Join("\n", errorList2);
Assert.IsTrue(0 == failedTests.Length, "The following Dealer Codes are not contained in the source: \n" + failedTests);
//orgkey
var sourceOrgKey = orgs.Select(o => o.OrgKey);
var destOrgKey = members.Select(o => o.OrgKey);
var errorList3 = sourceOrgKey.Except(destOrgKey);
failedTests = null;
failedTests = string.Join("\n", errorList3);
Assert.IsTrue(0 == failedTests.Length, "The following Org Keys are not contained in the source: \n" + failedTests);

You need reflection to do this:
Type type = obj.GetType();
PropertyInfo[] properties = type.GetProperties();
foreach (PropertyInfo property in properties)
{
GetColumn(orgList,property.Name);
}
var names = items.Select(x => x.GetType().GetProperty("prpname").GetValue(x));
public IEnumerable<object> GetColumn(List<Item> items, string columnName)
{
var values = items.Select(x =>
x.GetType().GetProperty(columnName).GetValue(x));//u can put your test code heere and you can loop it through object properties
}
you can create a list to store rslt and add results to the list else you can write output in logs files

If I haven't misunderstood your question, you could do something like this using shouldly
[TestMethod]
public void UnitTestExample()
{
var orgs = new Organisation
{
Ids = new List<int>{ 1,3 },
DealerCodes = new List<string> { "foo","bar"}
};
var members = new Organisation
{
Ids = new List<int> { 1,2,3 },
DealerCodes = new List<string> { "foo", "bar", "buzz" }
};
orgs.ShouldSatisfyAllConditions(
() => orgs.Ids.ShouldBe(members.Ids, ignoreOrder: true),
() => orgs.DealerCodes.ShouldBe(members.DealerCodes, ignoreOrder: true)
);
}
The output produced is:
You would still have to specify every property you want to check inside ShouldSatisfyAllConditions but shouldly does all the heavy lifting around comparing the lists and output the differences.

I'm assume that:
orgs is a IQueryable<T1>
members is a IQueryable<T2>
typeof(T1) == typeof(T2)
If so, that function could help:
private static void CompareRecords<TEntity>(IQueryable<TEntity> orgs, IQueryable<TEntity> members)
{
var entityType = typeof (TEntity);
var selectMethod = typeof (Queryable).GetMethods().First(x => x.Name == "Select");
var exceptMethod = typeof(Queryable).GetMethods().First(x => x.Name == "Except");
var toArrayMethod = typeof (Enumerable).GetMethods().First(x => x.Name == "ToArray");
foreach (var property in entityType.GetProperties())
{
var paramExpr = Expression.Parameter(entityType, "x");
var propExpr = Expression.Property(paramExpr, property.Name);
var delegateType = typeof (Func<,>).MakeGenericType(entityType, property.PropertyType);
var lambdaExpr = Expression.Lambda(delegateType, propExpr, paramExpr);
var parameterizedSelectMethod = selectMethod.MakeGenericMethod(entityType, property.PropertyType);
var parameterizedExceptMethod = exceptMethod.MakeGenericMethod(property.PropertyType);
var source = parameterizedSelectMethod.Invoke(null, new object[] {orgs, lambdaExpr});
var dest = parameterizedSelectMethod.Invoke(null, new object[] {members, lambdaExpr});
var errorList = parameterizedExceptMethod.Invoke(null, new[] {source, dest});
var errorListArray = toArrayMethod.MakeGenericMethod(property.PropertyType).Invoke(null, new[] {errorList});
var failedTests = string.Join("\n", ((IEnumerable)errorListArray).Cast<object>().Select(x => x.ToString()));
Assert.IsTrue(0 == failedTests.Length, $"The following {property.Name} are not contained in the source: \n{failedTests}");
}
}

filtering a tuple based on item 3 in the tuple

I need to print a list of countries that use dollars as a currency from a web service.
the data comes form a class called country services which contains this tuple:
public static IEnumerable<Tuple<string, string, string, string>> GetCountryData()
{
var countryService = new CountryServiceProxy.country();
var xmlStringResult = countryService.GetCurrencies();
var result = new List<Tuple<string, string, string, string>>();
var xPathDoc = new XPathDocument(new XmlTextReader(new StringReader(xmlStringResult)));
var navigator = xPathDoc.CreateNavigator();
var nodes = navigator.Select("//Table");
var nodeNames = new[] {"Name", "CountryCode", "Currency", "CurrencyCode"};
while (nodes.MoveNext())
{
var nodeValues = new[] {string.Empty, string.Empty, string.Empty, string.Empty};
for (var i = 0; i < nodeNames.Length; i++)
{
var node = nodes.Current.SelectSingleNode(nodeNames[i]);
if (node != null)
{
nodeValues[i] = node.Value;
}
}
result.Add(new Tuple<string, string, string, string>(nodeValues[0], nodeValues[1], nodeValues[2], nodeValues[3]));
}
return result;
}
I need to call that method and use it to print out a list with countries that use dollars:
private static IEnumerable<Country> Excercise4()
{
// var data = CountryService.GetCountryData().Where(x => x.Item3.Contains("Dollar"));
// ////var data = CountryService.GetCountryData().Where(x => x.Item3 == "Dollar");
// //Console.WriteLine(data);
return new List<Country>
{
new Country("da", "C1", "$", "Dollar"),
new Country("Country2", "C3", "$", "Dollar")
};
}
so far my method looks like this. i cant seem to figure out how to print out the tuple , havent used tuple before.
my write method is as follows:
ConsoleHelper.PrintCountryResults(Excercise4(), "Return a list of Country objects who's currency is Dollar");
ConsoleHelper.Pause(PauseMessage);
and the console helper class looks like this :
public static void PrintCountryResults(IEnumerable<Country> results, string title)
{
PrintHeader(title);
foreach (var result in results)
{
Console.WriteLine("{0, -30}{1, -5}{2, -15}{3, -5}", result.CountryName, result.CountryCode, result.CurrencyName, result.CurrencyCode);
}
}
any help would be appreciated as i said havent used tuple before first try at them ..
Thanx

You can filter the properties of the tuple by referencing the ItemX property with X being the 1-based index of the property that you're interested in. in you case if the currency is the third item, filtering the tuples would look like
var countriesUsingDollarAsACurrency = GetCountryData().Where(tuple => tuple.Item3 == "Dollar");

Generic DbDataReader to List<T> mapping

I am having a slight issue (more like an annoyance) with my property binding data access classes. The problem is that the mapping fails when there exists no column in the reader for corresponding property in class.
Code
Here is the mapper class:
// Map our datareader object to a strongly typed list
private static IList<T> Map<T>(DbDataReader dr) where T : new()
{
try
{
// initialize our returnable list
List<T> list = new List<T>();
// fire up the lamda mapping
var converter = new Converter<T>();
while (dr.Read())
{
// read in each row, and properly map it to our T object
var obj = converter.CreateItemFromRow(dr);
// add it to our list
list.Add(obj);
}
// reutrn it
return list;
}
catch (Exception ex)
{
return default(List<T>);
}
}
Converter class:
/// <summary>
/// Converter class to convert returned Sql Records to strongly typed classes
/// </summary>
/// <typeparam name="T">Type of the object we'll convert too</typeparam>
internal class Converter<T> where T : new()
{
// Concurrent Dictionay objects
private static ConcurrentDictionary<Type, object> _convertActionMap = new ConcurrentDictionary<Type, object>();
// Delegate action declaration
private Action<IDataReader, T> _convertAction;
// Build our mapping based on the properties in the class/type we've passed in to the class
private static Action<IDataReader, T> GetMapFunc()
{
var exps = new List<Expression>();
var paramExp = Expression.Parameter(typeof(IDataReader), "o7thDR");
var targetExp = Expression.Parameter(typeof(T), "o7thTarget");
var getPropInfo = typeof(IDataRecord).GetProperty("Item", new[] { typeof(string) });
var _props = typeof(T).GetProperties();
foreach (var property in _props)
{
var getPropExp = Expression.MakeIndex(paramExp, getPropInfo, new[] { Expression.Constant(property.Name, typeof(string)) });
var castExp = Expression.TypeAs(getPropExp, property.PropertyType);
var bindExp = Expression.Assign(Expression.Property(targetExp, property), castExp);
exps.Add(bindExp);
}
// return our compiled mapping, this will ensure it is cached to use through our record looping
return Expression.Lambda<Action<IDataReader, T>>(Expression.Block(exps), new[] { paramExp, targetExp }).Compile();
}
internal Converter()
{
// Fire off our mapping functionality
_convertAction = (Action<IDataReader, T>)_convertActionMap.GetOrAdd(typeof(T), (t) => GetMapFunc());
}
internal T CreateItemFromRow(IDataReader dataReader)
{
T result = new T();
_convertAction(dataReader, result);
return result;
}
}
Exception
System.IndexOutOfRangeException {"Mileage"}
Stacktrace
at System.Data.ProviderBase.FieldNameLookup.GetOrdinal(String fieldName)
at System.Data.SqlClient.SqlDataReader.GetOrdinal(String name)
at System.Data.SqlClient.SqlDataReader.get_Item(String name)
at lambda_method(Closure , IDataReader , Typing )
at o7th.Class.Library.Data.Converter`1.CreateItemFromRow(IDataReader dataReader) in d:\Backup Folder\Development\o7th Web Design\o7th.Class.Library.C-Sharp\o7th.Class.Library\Data Access Object\Converter.cs:line 50
at o7th.Class.Library.Data.Wrapper.Map[T](DbDataReader dr) in d:\Backup Folder\Development\o7th Web Design\o7th.Class.Library.C-Sharp\o7th.Class.Library\Data Access Object\Wrapper.cs:line 33
Question
How can I fix it, so that it will not fail when I have an extra property that the reader may not have as column and vice versa? Of course the quick band-aid would be to simply add NULL As Mileage to this query in example, however, this is not a solution to the problem :)
Here's Map<T> using reflection:
// Map our datareader object to a strongly typed list
private static IList<T> Map<T>(DbDataReader dr) where T : new()
{
try
{
// initialize our returnable list
List<T> list = new List<T>();
T item = new T();
PropertyInfo[] properties = (item.GetType()).GetProperties();
while (dr.Read()) {
int fc = dr.FieldCount;
for (int j = 0; j < fc; ++j) {
var pn = properties[j].Name;
var gn = dr.GetName(j);
if (gn == pn) {
properties[j].SetValue(item, dr[j], null);
}
}
list.Add(item);
}
// return it
return list;
}
catch (Exception ex)
{
// Catch an exception if any, an write it out to our logging mechanism, in addition to adding it our returnable message property
_Msg += "Wrapper.Map Exception: " + ex.Message;
ErrorReporting.WriteEm.WriteItem(ex, "o7th.Class.Library.Data.Wrapper.Map", _Msg);
// make sure this method returns a default List
return default(List<T>);
}
}
Note:
This method is 63% slower than using expression trees...

As noted in comments, the problem is that there exists no column in the reader for the specified property. The idea is to loop by the column names of reader first, and check to see if matching property exists. But how do one get the list of column names beforehand?
One idea is to use expression trees itself to build the list of column names from the reader and check it against properties of the class. Something like this
var paramExp = Expression.Parameter(typeof(IDataRecord), "o7thDR");
var loopIncrementVariableExp = Expression.Parameter(typeof(int), "i");
var columnNamesExp = Expression.Parameter(typeof(List<string>), "columnNames");
var columnCountExp = Expression.Property(paramExp, "FieldCount");
var getColumnNameExp = Expression.Call(paramExp, "GetName", Type.EmptyTypes,
Expression.PostIncrementAssign(loopIncrementVariableExp));
var addToListExp = Expression.Call(columnNamesExp, "Add", Type.EmptyTypes,
getColumnNameExp);
var labelExp = Expression.Label(columnNamesExp.Type);
var getColumnNamesExp = Expression.Block(
new[] { loopIncrementVariableExp, columnNamesExp },
Expression.Assign(columnNamesExp, Expression.New(columnNamesExp.Type)),
Expression.Loop(
Expression.IfThenElse(
Expression.LessThan(loopIncrementVariableExp, columnCountExp),
addToListExp,
Expression.Break(labelExp, columnNamesExp)),
labelExp));
would be the equivalent of
List<string> columnNames = new List<string>();
for (int i = 0; i < reader.FieldCount; i++)
{
columnNames.Add(reader.GetName(i));
}
One may continue with the final expression, but there is a catch here making any further effort along this line futile. The above expression tree will be fetching the column names every time the final delegate is called which in your case is for every object creation, which is against the spirit of your requirement.
Another approach is to let the converter class have a pre-defined awareness of the column names for a given type, by means of attributes (see for an example) or by maintaining a static dictionary like (Dictionary<Type, IEnumerable<string>>). Though it gives more flexibility, the flip side is that your query need not always include all the column names of a table, and any reader[notInTheQueryButOnlyInTheTableColumn] would result in exception.
The best approach as I see is to fetch the column names from the reader object, but only once. I would re-write the thing like:
private static List<string> columnNames;
private static Action<IDataReader, T> GetMapFunc()
{
var exps = new List<Expression>();
var paramExp = Expression.Parameter(typeof(IDataRecord), "o7thDR");
var targetExp = Expression.Parameter(typeof(T), "o7thTarget");
var getPropInfo = typeof(IDataRecord).GetProperty("Item", new[] { typeof(string) });
foreach (var columnName in columnNames)
{
var property = typeof(T).GetProperty(columnName);
if (property == null)
continue;
// use 'columnName' instead of 'property.Name' to speed up reader lookups
//in case of certain readers.
var columnNameExp = Expression.Constant(columnName);
var getPropExp = Expression.MakeIndex(
paramExp, getPropInfo, new[] { columnNameExp });
var castExp = Expression.TypeAs(getPropExp, property.PropertyType);
var bindExp = Expression.Assign(
Expression.Property(targetExp, property), castExp);
exps.Add(bindExp);
}
return Expression.Lambda<Action<IDataReader, T>>(
Expression.Block(exps), paramExp, targetExp).Compile();
}
internal T CreateItemFromRow(IDataReader dataReader)
{
if (columnNames == null)
{
columnNames = Enumerable.Range(0, dataReader.FieldCount)
.Select(x => dataReader.GetName(x))
.ToList();
_convertAction = (Action<IDataReader, T>)_convertActionMap.GetOrAdd(
typeof(T), (t) => GetMapFunc());
}
T result = new T();
_convertAction(dataReader, result);
return result;
}
Now that begs the question why not pass the data reader directly to constructor? That would be better.
private IDataReader dataReader;
private Action<IDataReader, T> GetMapFunc()
{
var exps = new List<Expression>();
var paramExp = Expression.Parameter(typeof(IDataRecord), "o7thDR");
var targetExp = Expression.Parameter(typeof(T), "o7thTarget");
var getPropInfo = typeof(IDataRecord).GetProperty("Item", new[] { typeof(string) });
var columnNames = Enumerable.Range(0, dataReader.FieldCount)
.Select(x => dataReader.GetName(x));
foreach (var columnName in columnNames)
{
var property = typeof(T).GetProperty(columnName);
if (property == null)
continue;
// use 'columnName' instead of 'property.Name' to speed up reader lookups
//in case of certain readers.
var columnNameExp = Expression.Constant(columnName);
var getPropExp = Expression.MakeIndex(
paramExp, getPropInfo, new[] { columnNameExp });
var castExp = Expression.TypeAs(getPropExp, property.PropertyType);
var bindExp = Expression.Assign(
Expression.Property(targetExp, property), castExp);
exps.Add(bindExp);
}
return Expression.Lambda<Action<IDataReader, T>>(
Expression.Block(exps), paramExp, targetExp).Compile();
}
internal Converter(IDataReader dataReader)
{
this.dataReader = dataReader;
_convertAction = (Action<IDataReader, T>)_convertActionMap.GetOrAdd(
typeof(T), (t) => GetMapFunc());
}
internal T CreateItemFromRow()
{
T result = new T();
_convertAction(dataReader, result);
return result;
}
Call it like
List<T> list = new List<T>();
var converter = new Converter<T>(dr);
while (dr.Read())
{
var obj = converter.CreateItemFromRow();
list.Add(obj);
}
There are a number of improvements that I can suggest, though.
The generic new T() you're calling in CreateItemFromRow is slower, it uses reflection behind the scenes. You can delegate that part to expression trees as well which should be faster
Right now GetProperty call isn't case insensitive, meaning your column names will have to exactly match the property name. I would make it case insensitive using one of those Bindings.Flag.
I'm not sure at all why you are using a ConcurrentDictionary as a caching mechanism here. A static field in a generic class <T> will be unique for every T. The generic field itself can act as cache. Also why is the Value part of ConcurrentDictionary of type object?
As I said earlier, it's not the best to strongly tie a type and the column names (which you're doing by caching one particular Action delegate per type). Even for the same type your queries can be different selecting different set of columns. It's best to leave it to data reader to decide.
Use Expression.Convert instead of Expression.TypeAs for value type conversion from object.
Also note that reader.GetOrdinal is much faster way to perform data reader lookups.
I would re-write the whole thing like:
readonly Func<IDataReader, T> _converter;
readonly IDataReader dataReader;
private Func<IDataReader, T> GetMapFunc()
{
var exps = new List<Expression>();
var paramExp = Expression.Parameter(typeof(IDataRecord), "o7thDR");
var targetExp = Expression.Variable(typeof(T));
exps.Add(Expression.Assign(targetExp, Expression.New(targetExp.Type)));
//does int based lookup
var indexerInfo = typeof(IDataRecord).GetProperty("Item", new[] { typeof(int) });
var columnNames = Enumerable.Range(0, dataReader.FieldCount)
.Select(i => new { i, name = dataReader.GetName(i) });
foreach (var column in columnNames)
{
var property = targetExp.Type.GetProperty(
column.name,
BindingFlags.Public | BindingFlags.Instance | BindingFlags.IgnoreCase);
if (property == null)
continue;
var columnNameExp = Expression.Constant(column.i);
var propertyExp = Expression.MakeIndex(
paramExp, indexerInfo, new[] { columnNameExp });
var convertExp = Expression.Convert(propertyExp, property.PropertyType);
var bindExp = Expression.Assign(
Expression.Property(targetExp, property), convertExp);
exps.Add(bindExp);
}
exps.Add(targetExp);
return Expression.Lambda<Func<IDataReader, T>>(
Expression.Block(new[] { targetExp }, exps), paramExp).Compile();
}
internal Converter(IDataReader dataReader)
{
this.dataReader = dataReader;
_converter = GetMapFunc();
}
internal T CreateItemFromRow()
{
return _converter(dataReader);
}

Query a Dictionary of Dictionaries?

Please can you advise me on how to query a Dictionary of Dictionaries, and/or a Dictionary of List?
private Dictionary<string, Dictionary<DateTime, double>> masterDict= new Dictionary<string, Dictionary<DateTime, double>>();
Private Dictionary<string, List<DateTime>> masterList= new Dictionary<string, List<DateTime>>();
I know if I do the following, I get a list of the dictionaries contained in masterDict, but I'm not sure how to get at the values of those dictionaries.
foreach (var kvp in masterDictMethod())
{
Console.WriteLine("Key = {0}, Value = {1}",
kvp.Key, kvp.Value);
}
Thanks for looking ;)

In you foreach kvp.Value is the inner dictionary of every masterDict entry i.e. Dictionary<DateTime, double>
So, just foreach also over kvp.Value and you will get the inner values.
e.g.
foreach (var kvp1 in masterDictMethod())
{
Console.WriteLine("Key = {0}, Inner Dict:", kvp1.Key);
foreach (var kvp2 in kvp1.Value)
{
Console.WriteLine("Date = {0}, Double = {1}", kvp2.Key, kvp2.Value);
}
}

Use masterDict.Values

This one is:
var masterDictionary = new Dictionary<string, Dictionary<DateTime, double>>();
var query =
from kvp1 in masterDictionary
from kvp2 in kvp1.Value
select new {TheString = kvp1.Key, TheDate = kvp2.Key, TheDouble = kvp2.Value };
foreach(var x in query)
{
Console.WriteLine("{0} {1} {2}", x.TheString, x.TheDate, x.TheDouble);
}
And then the other one is:
var masterList= new Dictionary<string, List<DateTime>>();
var query =
from kvp in masterList
from val in kvp.Value
select new {TheString = kvp.Key, TheDate = val);
foreach(var x in query)
{
Console.WriteLine("{0} {1}", x.TheString, x.TheDate);
}

foreach (var key in masterDict.Keys)
{
var nestedDict = masterDict[key];
}

You asked about lists, dictionaries and dictionaries containing other dictionaries.
I had a similar topic recently, where I wanted to have a queryable dictionary (i.e. an extension method which allows to pass a query expression as lambda parameter), that you can use like:
var result = myDictionary.QueryDictionary(w => myList.Any(a => a == w.Key));
The purpose of this code line is to check if any key of the dictionary is contained in myList.
So what I did is this, I wrote the following extension method:
// extension method using lambda parameters
public static Dictionary<string, T> QueryDictionary<T>(
this Dictionary<string, T> myDict,
Expression<Func<KeyValuePair<string,T>, bool>> fnLambda)
{
return myDict.AsQueryable().Where(fnLambda).ToDictionary(t => t.Key, t => t.Value);
}
It can be used for every dictionary which has keys of type string and items of every object type T.
Now you can easily write queries by passing a lambda expression, as in the following example:
var list1 = new List<string>() { "a", "b" };
var myDict = new Dictionary<string, object>();
myDict.Add("a", "123"); myDict.Add("b", "456"); myDict.Add("c", "789");
var result = myDict.QueryDictionary(w => list1.Any(a => a == w.Key));
The result will contain items a and b, because they are contained in list1.
You can also query a dictionary of dictionaries, here's a C# example for LinqPad, but it can be used as a console application as well (just comment out the .Dump() statements and replace them by Console.WriteLine(...) statements):
void Main()
{
// *** Set up some data structures to be used later ***
var list1 = new List<string>() { "a", "b", "d" }; // a list
var myDict = new Dictionary<string, object>(); // the dictionary
myDict.Add("a", "123"); myDict.Add("b", "456"); myDict.Add("c", "789");
var myDict2 = new Dictionary<string, object>(); // 2nd dictionary
myDict2.Add("a", "123"); myDict2.Add("b", "456"); myDict2.Add("c", "789");
myDict.Add("d", myDict2); // add 2nd to first dictionary
// *** 1. simple query on dictionary myDict ***
var q1 = myDict.QueryDictionary(w => list1.Any(a => a == w.Key));
q1.Dump();
// *** 2. query dictionary of dictionary (q3 contains result) ***
var q2 =
(Dictionary<string, object>)q1.QueryDictionary(w => w.Key.Equals("d")).First().Value;
var q3 = q2.QueryDictionary(w => w.Key.Equals("b"));
q3.Dump();
}
// *** Extension method 'QueryDictionary' used in code above ***
public static class Extensions
{
public static Dictionary<string, T> QueryDictionary<T>(
this Dictionary<string, T> myDict,
Expression<Func<KeyValuePair<string, T>, bool>> fnLambda)
{
return myDict.AsQueryable().Where(fnLambda).ToDictionary(t => t.Key, t => t.Value);
}
}
Since this solution is using Generics, you can pass any lambda expression as search parameter, so it is very flexible.