Develop Reference

How to handle list dynamically Using reflection and cast returned data to list

When I add a list in instances value and return that list its provides always duplicate value. I wanna read data from the database table values. But its first iterations add a single row value, the second time its provides 2nd row value and replaces the 1st list values.
var instance = (T)Activator.CreateInstance(type);
while (reader.Read())
{
type.GetProperties().ToList().ForEach(property =>
{
property.SetValue(instance,reader[property.Name]);
});
list.Add(instance);
}
return list;

If each row in reader represents a different instance: create a new instance for each row; i.e.
while (reader.Read())
{
var instance = (T)Activator.CreateInstance(type);
// ...
list.Add(instance);
}
return list;

Convert dynamically shaped query results into IEnumerabale or IQueryable

I am attempting to take the results of a Stored Procedure that that contains dynamic columns, different shapes of data, and convert that into an IEnumerable or IQueryable.
In the middleware, I am using EF and since I have never worked with dynamic queries, I am at a loss. I looked at the DataTable.ToSchemaTable() to see if that could convert to anything useful and hit a wall. Recently, I started looking into dynamics, however, I still can't seem to figure out how to convert to IEnumerable.
The clientside needs to have data in the form of {"FieldName":"FieldValue"}, and the result set needs to be pre-processed via an IEnumerable prior to being converted to JSON.

Here is the code I am working with that solved the issue.
public static class DataTableExtensions
{
public static IEnumerable<dynamic> AsDynamicEnumerable(this DataTable table)
{
if (table == null)
{
yield break;
}
foreach (DataRow row in table.Rows)
{
IDictionary<string, object> dRow = new ExpandoObject();
foreach (DataColumn column in table.Columns)
{
var value = row[column.ColumnName];
dRow[column.ColumnName] = Convert.IsDBNull(value) ? null : value;
}
yield return dRow;
}
}
}

Save row # to list when conditions are met

I have a datatable imported from a csv. What I'm trying to do is compare all of the rows to each other to find duplicates. In the case of duplicates I am going to add the row # to a list, then write the list to an array and deal with the duplicates after that.
//find duplicate rows and merge them.
foreach (DataRow dr in dt.Rows)
{
//loop again to compare rows
foreach (DataRow dx in dt.Rows)
{
if (dx[0]==dr[0] && dx[1]==dr[1] && dx[2] == dr[2] && dx[3] == dr[3] && dx[4] == dr[4] && dx[5] == dr[5] && dx[7] == dr[7])
{
dupeRows.Add(dx.ToString());
}
}
}
for testing I have added:
listBox1.Items.AddRange(dupeRows.ToArray());
which simply outputs System.Data.DataRow.
How do I store the duplicate row index ids?

The basic problem is that you saved a string describing the type of the row (what DataRow.ToString() returns by default) at the time you decided the row was a duplicate
Assuming you've read your CSV straight in with some library/driver rather than line by line (which would have been a good time to dedupe) let's use a dictionary to dedupe:
Dictionary<string, DataRow> d = new Dictionary<string, DataRow>();
foreach(var ro in dataTable.Rows){
//form a key for the dictionary
string key = string.Format("{0}\t{1}\t{2}\t{3}\t{4}\t{5}\t{7}", ro.ItemArray);
d[key] = ro;
}
That's it; at the end of this operation the d.Values will be a deduped collection of DataRow. 1000 rows will require 1000 operations so this will likely be orders of magnitude faster than comparing every row to every other row, which would need a million operations for a thousand rows
I've used tabs to separate the values when I formed the key - assuming your data contains no tabs. Best reliability will be achieved if you use a character that does not appear in the data
If you've read your CSV line by line and done a manual string split on comma (i.e. a primitive way of reading a CSV) you could do this operation then instead; after you split you have an array that can be used in place of ro.ItemArray. Process the entire file, creating rows (and adding to the dictionary) only if d.ContainsKey returns false. If the dictionary already contains that row, skip on rather than creating a row

The output (System.Data.DataRow) that you are seeing is expected since there is no custom implementation of DataRow.ToString() found in your project, framework is calling base class's (which is System.Object) ToString() for which the default implementation returns data type of object which invokes that method.
I see three solutions here:
If possible, try to read the DataTable into custom objects (like
MyDataTable, MyDataRow) so, you can create your own ToString() like
below:
public class MyDataRow
{
public override string ToString()
{
return "This is my custom data row formatted string";
}
}
in the for loop, when you found duplicated row, either just add
index/id (sort of primary key) of dx to array and then have another
for loop to retrieve dupes.
Third is same as mentioned by Caius Jard.

Problem in converting a generic list<string> or string[] array to datatable

I have the below function
public static DataTable ToTable<T>(this IEnumerable<T> listItem)
{
//Return null if the list is empty
if (listItem == null || listItem.Count() == 0) return null;
//Gets the type of the object
var listType = listItem.First().GetType();
//Initialize a new datatable
var dataTable = new DataTable(listType.Name);
//Create the datatable column names and types
listType.GetProperties().ToList().ForEach(col => dataTable.Columns.Add(col.Name, col.PropertyType));
//Get the datatable column names
var dataTableColumnNames = dataTable.GetDatatableColumnNames();
listItem.ToList().ForEach(item =>
{
//create a new datarow
var dataRow = dataTable.NewRow();
dataTableColumnNames
.Where(propName => listType.GetProperty(propName) != null)
.ToList()
.ForEach(columnName =>
//Exception happens here in the next line
dataRow[columnName] = listType.GetProperty(columnName).GetValue(item, null));
//Add the row to the data table
dataTable.Rows.Add(dataRow);
});
//Commit the changes to the datatable
dataTable.AcceptChanges();
return dataTable;
}
It works great for dictionary object and generic list as List<MyClass> .. but not for
List<string> or string[].
For those I am getting an exception as Parameter count mismatch.
The error is coming at
dataRow[columnName] = listType.GetProperty(columnName).GetValue(item, null));
What is the mistake that is happening?
Please help

Here's the deal. The index operator is actually considered a property when using reflection, hence parameter count mismatch.
If you break into your code and check the properties that are actually being enumerated by GetProperties(), you'll see the "Chars" property. That's the String's index operator. Since you didn't provide an index, you're getting a Parameter Count Mismatch error.
In essence, I assume string doesn't have any properties you want to put in your data table, but rather the string instance IS what you want to put in the data table.
You could create a model to store the string in, with the string as a property on the model, then the string would be stored with your current code. Otherwise, you will need to rethink your table generation algorithm for primitive types.
I hope this helps :)

Because one of the public properties of string is an indexer and you pass null as the index value. So you effectively end up doing this: string[null] which ends up in an exception.
I haven't verified this as I don't have VS available right now so I might be wrong but I'm pretty sure that's the problem.
Update: This question answers how you detect an indexed property: C# Reflection Indexed Properties

How can I make this extension method more generic?

I'm having a brain fart trying to make the following method more generic such that any List<T> can be passed in for the columnValues parameter. Here's what I have:
public static DataRow NewRow(this DataTable dataTable, List<string> columnValues)
{
DataRow returnValue = dataTable.NewRow();
while (columnValues.Count > returnValue.Table.Columns.Count)
{
returnValue.Table.Columns.Add();
}
returnValue.ItemArray = columnValues.ToArray();
return returnValue;
}
I could change it to a List<object> and convert the original list prior to passing it to the method but I'm sure there is a better option :-)
Edit:
Frank's post made me rethink this. In most cases that source List<T> would be a List<object> since the column values will most likely be different types.
For my initial use a List<string> made sense because I was creating a dataset from a CSV parse which is all text at that point.

Why not just use params object[]:
public static DataRow NewRow(this DataTable dataTable, params object[] objects)
{
DataRow returnValue = dataTable.NewRow();
while (objects.Length > returnValue.Table.Columns.Count)
{
returnValue.Table.Columns.Add();
}
returnValue.ItemArray = objects;
return returnValue;
}
Then you can just call it like this:
myDataTable.NewRow(1,2,"hello");

You're basically out of luck, because the Item Array of the DataRow is an array of objects, that is, ultimately you can only pass in list of objects.
If you put in a generic parameter of the list all items of the list would have to be of that type, which is highly unlikely to be useful.
Having said that, in order to get numerous columns, all with different types, you could change your extension method to accept an object into which you instantiate an anonymous type:
table.NewRow(new { A = "Hello", B = 1, C = DateTime.Now })
With the aid to convert the anonymous type values to a string,object dictionary either by reflection or by a dynamic method it should be a fairly useful thing.

What about
IEnumerable<object>
in connection with
columnValues.Select(x => x.ToString()).ToArray();

What about using a closure to specify how to generate the ItemArray based upon your input type
public static DataRow NewRow<T>(this DataTable dataTable, List<T> columnValues, Func<T, string> itemArrayCriteria)
{
DataRow returnValue = dataTable.NewRow();
while (columnValues.Count > returnValue.Table.Columns.Count)
{
returnValue.Table.Columns.Add();
}
returnValue.ItemArray = columnValues.Select(x => itemArrayCriteria(x)).ToArray();
return returnValue;
}