Develop Reference

Convert dynamically shaped query results into IEnumerabale or IQueryable

I am attempting to take the results of a Stored Procedure that that contains dynamic columns, different shapes of data, and convert that into an IEnumerable or IQueryable.
In the middleware, I am using EF and since I have never worked with dynamic queries, I am at a loss. I looked at the DataTable.ToSchemaTable() to see if that could convert to anything useful and hit a wall. Recently, I started looking into dynamics, however, I still can't seem to figure out how to convert to IEnumerable.
The clientside needs to have data in the form of {"FieldName":"FieldValue"}, and the result set needs to be pre-processed via an IEnumerable prior to being converted to JSON.

Here is the code I am working with that solved the issue.
public static class DataTableExtensions
{
public static IEnumerable<dynamic> AsDynamicEnumerable(this DataTable table)
{
if (table == null)
{
yield break;
}
foreach (DataRow row in table.Rows)
{
IDictionary<string, object> dRow = new ExpandoObject();
foreach (DataColumn column in table.Columns)
{
var value = row[column.ColumnName];
dRow[column.ColumnName] = Convert.IsDBNull(value) ? null : value;
}
yield return dRow;
}
}
}

Remove foreach loop with the help of linq

I have some problem with my code. I want to replace the ForEach loop with the help of LINQ here, is there any way or solution to solve my problem? My code is given bellow.
static public string table2Json(DataSet ds, int table_no)
{
try
{
object[][] tb = new object[ds.Tables[table_no].Rows.Count][];
int r = 0;
foreach (DataRow dr in ds.Tables[table_no].Rows)
{
tb[r] = new object[ds.Tables[table_no].Columns.Count];
int col = 0;
foreach (DataColumn column in ds.Tables[table_no].Columns)
{
tb[r][col] = dr[col];
if ((tb[r][col]).Equals(System.DBNull.Value))
{
tb[r][col] = "";
}
col++;
}
r++;
}
string table = JsonConvert.SerializeObject(tb, Formatting.Indented);
return table;
}
catch (Exception ex)
{
tools.log(ex.Message);
throw ex;
}
}

This question really asks 3 different things:
how to serialize a DataTable
how to change the DataTable serialization format and finally
how to replace nulls with empty strings, even though an empty string isn't a NULL.
JSON.NET already handles DataSet and DataTable instance serialization with a DataTableConverter whose source can be found here. You could just write :
var str = JsonConvert.SerializeObject(data);
Given this DataTable :
var dataTable=new DataTable();
dataTable.Columns.Add("Name",typeof(string));
dataTable.Columns.Add("SurName",typeof(string));
dataTable.Rows.Add("Moo",null);
dataTable.Rows.Add("AAA","BBB");
You get :
[{"Name":"Moo","SurName":null},{"Name":"AAA","SurName":"BBB"}]
DataTables aren't 2D arrays and the column names and types matter. Generating a separate row object with named fields is far better than generating an object[] array. It also allows makes it far easier for clients to handle the JSON string without knowing its schema in advance. With an object[] for each row, the clients will have to know what's stored in each location in advance.
If you want to use a different serialization format, you could customize the DataTableConverter. Another option though, is to use DataRow.ItemArray to get the values as an object[] and LINQ to get the rows, eg :
object[][] values=dataTable.Rows.Cast<DataRow>()
.Select(row=>row.ItemArray)
.ToArray();
Serializing this produces :
[["Moo",null],["AAA","BBB"]]
And there's no way to tell which item is the name and which is the surname any more.
Replacing DBNulls with strings in this last form needs an extra Select() to replace DBNull.Value with "" :
object[][] values=dataTable.Rows.Cast<DataRow>()
.Select(row=>row.ItemArray
.Select(x=>x==DBNull.Value?"":x)
.ToArray())
.ToArray();
Serializing this produces :
[["Moo",""],["AAA","BBB"]]
That's what was asked, but now we have no way to tell whether the Surname is an empty string, or just doesn't exist.
This may sound strange, but Arabic names may be one long name without surname. Makes things interesting for airlines or travel agents that try to issue tickets (ask me how I know).
We can get rid of ToArray() if we use var :
var values=dataTable.Rows.Cast<DataRow>()
.Select(row=>row.ItemArray
.Select(x=>x==DBNull.Value?"":x));
JSON serialization will work the same.

LINQ is not a nice fit for this sort of thing because you are using explicit indexes r and col into multiple "array structures" (and there is no easy/tidy way to achieve multiple, parallel enumeration).
Other issues
tb is repeatedly newed, filled with data and then replaced in the next iteration, so you end up capturing only the last row of input to the JSON string - that's a logical bug and won't work as I think you intend.
The inner foreach loop declares but does not use the iteration variable column - that's not going to break anything but it is redundant.
You will get more mileage out of using JSON.Net properly (or coding the foreach loops as for loops instead if you want to navigate the structures yourself).

How to use dynamic Linq with List<dynamic> object

I have a List of dynamic objects that I am trying to use dynamic Linq on. I am using dynamic objects because I do not know the properties that will be coming into the object. Linq works on my dynamic object, but, to avoid giant hard coding if statements, I would like to use dynamic Linq to search my list. The top half of the code snippet works but I need it to work dynamically so I can create a query string from my properties and filter that way.
public List<dynamic> GetFilteredLocationData(List<dynamic> locationData, string searchTerm){
//Does work
List<dynamic> totalResults = locationData.Where(x => x.Street.ToLower().Contains(searchTerm.ToLower()) ||
x.Street.ToLower().Contains(searchTerm.ToLower()) ||
x.Zip.ToLower().Contains(searchTerm.ToLower()));
//Does not work
var testQueryString = "(Street == \"king\")";
var testResult = locationData.Where(testQueryString);
return totalResults;
}
The runtime error I receive: No property or field 'Street' exists in type 'Object'
That error makes sense as object by default doesn't contain 'Street' but I'd expect the dynamic Linq to behave like the code above it. Is there something I am doing wrong here, or should I take a different approach? I can provide more detail if needed.
Thanks in advance!

Finally I got a working solution! It may not be the most efficient but it works for my needs and allows me to keep the dynamic nature I was hoping to retain. The solution was to drop Linq entirely and use a good old for-each loop. The Important part was the IDictionary which allowed me to search each row for the key value pair. This is the same functionality I was going for, just ditched linq.
public List<dynamic> GetFilteredLocationData(List<dynamic> locationData, string searchTerm){
List<dynamic> totalResults = new List<dynamic>();
List<string> locationProperties = new List<string> {"dynamic properties here, this was filled by call to DB for info pertaining to certain location combined with unique data"}
foreach (var locData in locationData)
{
var currentLoc = locData;
var currentLocDict = (IDictionary<string, object>)currentLoc;
bool containsSearchTerm = CheckIfLocationContainsSearch(currentLocDict, allLocationProperties, searchTerm);
if (containsSearchTerm)
{
totalResults.Add(locData);
}
}
}
public bool CheckIfLocationContainsSearch(IDictionary<string,object> location, List<string> locationProperties, string searchTerm){
foreach (var locProp in locationProperties)
{
if (location[locProp].ToString().ToLower().Contains(searchTerm))
{
return true;
}
}
return false;
}

how to add an associative index to an array. c#

i have an array of custom objects. i'd like to be able to reference this array by a particular data member, for instance myArrary["Item1"]
"Item1" is actually the value stored in the Name property of this custom type and I can write a predicate to mark the appropriate array item. However I am unclear as to how to let the array know i'd like to use this predicate to find the array item.
I'd like to just use a dictionary or hashtable or NameValuePair for this array, and get around this whole problem but it's generated and it must remain as CustomObj[]. i'm also trying to avoid loading a dictionary from this array as it's going to happen many times and there could be many objects in it.
For clarification
myArray[5] = new CustomObj() // easy!
myArray["ItemName"] = new CustomObj(); // how to do this?
Can the above be done? I'm really just looking for something similar to how DataRow.Columns["MyColumnName"] works
Thanks for the advice.

What you really want is an OrderedDictionary. The version that .NET provides in System.Collections.Specialized is not generic - however there is a generic version on CodeProject that you could use. Internally, this is really just a hashtable married to a list ... but it is exposed in a uniform manner.
If you really want to avoid using a dictionary - you're going to have to live with O(n) lookup performance for an item by key. In that case, stick with an array or list and just use the LINQ Where() method to lookup a value. You can use either First() or Single() depending on whether duplicate entries are expected.
var myArrayOfCustom = ...
var item = myArrayOfCustom.Where( x => x.Name = "yourSearchValue" ).First();
It's easy enough to wrap this functionality into a class so that external consumers are not burdened by this knowledge, and can use simple indexers to access the data. You could then add features like memoization if you expect the same values are going to be accessed frequently. In this way you could amortize the cost of building the underlying lookup dictionary over multiple accesses.

If you do not want to use "Dictionary", then you should create class "myArrary" with data mass storage functionality and add indexers of type "int" for index access and of type "string" for associative access.
public CustomObj this [string index]
{
get
{
return data[searchIdxByName(index)];
}
set
{
data[searchIdxByName(index)] = value;
}
}
First link in google for indexers is: http://www.csharphelp.com/2006/04/c-indexers/

you could use a dictionary for this, although it might not be the best solution in the world this is the first i came up with.
Dictionary<string, int> d = new Dictionary<string, int>();
d.Add("cat", 2);
d.Add("dog", 1);
d.Add("llama", 0);
d.Add("iguana", -1);
the ints could be objects, what you like :)
http://dotnetperls.com/dictionary-keys

Perhaps OrderedDictionary is what you're looking for.

you can use HashTable ;
System.Collections.Hashtable o_Hash_Table = new Hashtable();
o_Hash_Table.Add("Key", "Value");

There is a class in the System.Collections namespace called Dictionary<K,V> that you should use.
var d = new Dictionary<string, MyObj>();
MyObj o = d["a string variable"];
Another way would be to code two methods/a property:
public MyObj this[string index]
{
get
{
foreach (var o in My_Enumerable)
{
if (o.Name == index)
{
return o;
}
}
}
set
{
foreach (var o in My_Enumerable)
{
if (o.Name == index)
{
var i = My_Enumerable.IndexOf(0);
My_Enumerable.Remove(0);
My_Enumerable.Add(value);
}
}
}
}
I hope it helps!

It depends on the collection, some collections allow accessing by name and some don't. Accessing with strings is only meaningful when the collection has data stored, the column collection identifies columns by their name, thus allowing you to select a column by its name. In a normal array this would not work because items are only identified by their index number.

My best recommendation, if you can't change it to use a dictionary, is to either use a Linq expression:
var item1 = myArray.Where(x => x.Name == "Item1").FirstOrDefault();
or, make an extension method that uses a linq expression:
public static class CustomObjExtensions
{
public static CustomObj Get(this CustomObj[] Array, string Name)
{
Array.Where(x => x.Name == Name).FirstOrDefault();
}
}
then in your app:
var item2 = myArray.Get("Item2");
Note however that performance wouldn't be as good as using a dictionary, since behind the scenes .NET will just loop through the list until it finds a match, so if your list isn't going to change frequently, then you could just make a Dictionary instead.

I have two ideas:
1) I'm not sure you're aware but you can copy dictionary objects to an array like so:
Dictionary dict = new Dictionary();
dict.Add("tesT",40);
int[] myints = new int[dict.Count];
dict.Values.CopyTo(myints, 0);
This might allow you to use a Dictionary for everything while still keeping the output as an array.
2) You could also actually create a DataTable programmatically if that's the exact functionality you want:
DataTable dt = new DataTable();
DataColumn dc1 = new DataColumn("ID", typeof(int));
DataColumn dc2 = new DataColumn("Name", typeof(string));
dt.Columns.Add(dc1);
dt.Columns.Add(dc2);
DataRow row = dt.NewRow();
row["ID"] = 100;
row["Name"] = "Test";
dt.Rows.Add(row);
You could also create this outside of the method so you don't have to make the table over again every time.

DataGridView filtering

I'm creating a control that should be able to take any kind of list. Essentially the following code:
void BindData(IList list)
{
BindingSource bs = new BindindSource();
bs.DataSource = list;
this.DataGridView.DataSource = bs;
}
Now I have a textbox that I want to use to filter the data in my grid. I figured it'd be as simple as setting the bs.Filter property but apparently not. The bs.SupportsFiltering returns false as well.
Is this an issue with me using the IList? If so, is there another collection class / interface that I can use to achieve the same effect? (Again, I'm not sure what the type is of the objects in the list.

Not knowing the type I'm getting passed, I resulted in filtering the data by hand.
Here's my code snippet. It works well. Hopefully it doesn't prove to be too slow with larger amounts of data. :: Fingers Crossed ::
List<object> filteredData = new List<object>();
foreach (object data in this.DataSource)
{
foreach (var column in this.Columns)
{
var value = data.GetType().GetProperty(column.Field).GetValue(data,null)
.ToString();
if (value.Contains(this.ddFind.Text))
{
filteredData.Add(data);
break;
}
}
}
this.ddGrid.DataSource = filteredData;

The IBindingListView interface supplements the data-binding capabilities of the IBindingList interface by adding support for filtering of the list.
A couple of solutions for generic IBindingListView implementations can be found here.