Develop Reference

ML.net Load From Enumerable Trouble

I have been struggling to create the proper data structure for ML.net and get it to load into my application. Essentially, I have an application where the training data will be dynamic and the type and/or size will not be known prior to runtime. In addition, I have to convert the training data from a non-standard primitive types (ie. App_Bool, or App_Number... rather than simply using bool or double, etc.) So, this has been proving to be a problem as I try to convert my training data into a generic data type which can then be loaded from memory using the LoadFromEnumerable function.
I have four basic data type classes:
public class MLIntData
{
public MLIntData(string label, List<object> l)
{
Label = label;
foreach (App_Integer element in l)
Features.Add((int)element.Value);
}
public List<int> Features { get; set; } = new List<int>();
public string Label { get; set; } = "";
}
public class MLNumberData
{
public MLNumberData(string label, List<object> l)
{
Label = label;
foreach (App_Number element in l)
Features.Add((double)element.Value);
}
public List<double> Features { get; set; } = new List<double>();
public string Label { get; set; } = "";
}
public class MLBoolData
{
public MLBoolData(string label, List<object> l)
{
Label = label;
foreach (App_Boolean element in l)
Features.Add((bool)element.Value);
}
public List<bool> Features { get; set; } = new List<bool>();
public string Label { get; set; } = "";
}
public class MLTextData
{
public MLTextData(string label, List<object> l)
{
Label = label;
foreach (App_String element in l)
Features.Add(element.Value.ToString());
}
public List<string> Features { get; set; } = new List<string>();
public string Label { get; set; } = "";
}
So, each base class will contain a label for the data and then a list of features which will either be of type bool, double, int, or string.
Now, in my ML.net code I'm trying to load in the training data and then create an IDataView object of the data. First I loop through the input data (which is originally of the generic type object) then create the new classes of data.
List<object> data = new List<object>();
for(int i = 0; i < input.Count; i++)
{
MLCodifiedData codifiedData = input[i].Value as MLCodifiedData;
Type dataType = codifiedData.Features[0].GetType();
if (dataType == typeof(App_Boolean))
{
data.Add(new MLBoolData(codifiedData.Label, codifiedData.Features));
}
else if (dataType == typeof(App_Number))
{
data.Add(new MLNumberData(codifiedData.Label, codifiedData.Features));
}
else if (dataType == typeof(App_Integer))
{
data.Add(new MLIntData(codifiedData.Label, codifiedData.Features));
}
if (dataType == typeof(App_String))
{
data.Add(new MLTextData(codifiedData.Label, codifiedData.Features));
}
}
IDataView TrainingData = mlContext.Data.LoadFromEnumerable<object>(data);
I have tried creating a schema definition (which can be passed in as the second parameter in the LoadFromEnumerable method, but I can't seem to get that to work. I've also tried creating a schema using the schema builder to create a schema, but that doesn't seem to work either. Right now, I'm using one of the datasets that is included in one of the sample files. And to preempt questions, yes, I know I could simply load the data as file and read it in that way... However, in my app I need to first read in the CSV into memory, then create the data structure so I can't really use many of the examples which are geared toward reading in a CSV file using the LoadFromTextFile method. Can anyone provide support as to how I could setup a dynamic in-memory collection and get it converted into a IDataView object?

How do I loop through all fields in an object in C#?

I have an object like this:
public class Filters {
public int var1 = 1,
var2 = 2,
var3 = 3;
}
I declare this object here:
Filters filter1 = new Filters();
And I want to access var1, var2, and var3 in a loop and do something with it. i.e.:
foreach (var prop in filter1.props) {
Console.WriteLine(filter1[prop] + 3);
}
and the output would be:
4
5
6
I imagine I need to do a foreach loop for each property using
foreach(PropertyInfo p in filter1.GetType().GetProperties()), but I don't know how to 1) loop through props var1, var2, var3, and 2) how to subset the prop from filter1 using the name stored in the variable

If you describe your variable as properties like bellow,
public class Filters
{
public int var1 { get; set; } = 1;
public int var2 { get; set; } = 2;
public int var3 { get; set; } = 3;
}
You can access these properties with
GetType().GetProperties()
then the main method will give you what you ask for
static void Main(string[] args)
{
Filters filter1 = new Filters();
foreach (var prop in filter1.GetType().GetProperties())
{
Console.WriteLine("{0}={1}", prop.Name, (int)prop.GetValue(filter1, null) + filter1.GetType().GetProperties().Length);
}
Console.ReadKey();
}
Result will be
4
5
6

Thanks to everyone who answered-- a couple hints helped me get there. I just started in C# so I didn't know what fields/props were, so thanks #SeM #John. But with that, and with answers by #Icepickle & #arslanaybars with GetProperties() but for fields instead:
FieldInfo[] fields = typeof(GeneralFilters).GetFields();
for (int i = 0; i < fields.Length; i++)
{
//MANIPULATE HERE
BlankTemplate tempFilter = (BlankTemplate)fields[i].GetValue(filters);
// Ignore this for now. tempFilter.selectedItems;
}
where BlankTemplate is defined here:
public class BlankTemplate
{
public string[] selectedItems;
public bool selectAll = false;
}
And now in tempFilter I have the object that I need to use at every iteration
Thanks!!!
Edit: I realize that this doesn't answer the question of how to subset using the stringified name of the object fields. What I envisioned before is generating array of field names, then looping through and subsetting the data in the fields using the field names, like in javascript:
var fieldNames = Object.keys(filterObject);
for (var i = 0; i < fieldNames.length; i++) {
doSomething( filterObject[fieldNames[i]] );
}
But it seems to be a bit different in C#

An alternative answer to your question could be the following, say you have a class of filters like the following
public class Filter
{
public IDictionary<string, object> Properties { get; } = new Dictionary<string, object>();
}
This would allow you to have a dynamic set of filters, you can assign new properties as a consumer and iterate the existing ones. That seems to fit your current requirements.
As for a real answer, as many in the comments have pointed out, if you want to iterate properties, then you should actually use them. In your sample code, you have provided fields instead.
So your class Filter would probably end up looking like this (note that I think var1...var3 are the most horrible names you can use as I cannot imagine what they might define in the end):
public class Filter
{
public int Var1 { get; set; } = 1;
public int Var2 { get; set; } = 2;
public int Var3 { get; set; } = 3;
}
and then you could have something similar like:
var filter = new Filter();
var filterType = filter.GetType();
var readableProperties = filterType.GetProperties().Where( p => p.GetGetMethod() != null );
foreach (var property in readableProperties)
{
var value = (int)property.GetValue( filter );
Console.WriteLine( $"{property.Name} = {value + 3}" );
}
To ensure that you only select those you actually want, you can ofcourse check the name if it equals to Var1, Var2, Var3 or matches a regex expression, or whatever you like to think of ;)
A sample of the code here, you can find in this dotnetfiddle (though without autoproperties and $)

For loop to add image to imagelist

I'm cleaning up my code trying to short in some things
Now I've stumbled across:
ImageList.Add(test.Properties.Resources.test1);
ImageList.Add(test.Properties.Resources.test2);
ImageList.Add(test.Properties.Resources.test3);
ImageList.Add(test.Properties.Resources.test4);
ImageList.Add(test.Properties.Resources.test5);
(There are 15 of these)
Was wondering if this could be shortened with a for loop
Something like:
for(int i=1; i<=15; i++)
ImageList.Add(test.Properties.Resources.test +i);
Now ofcourse this won't work but I have no clue how to do this (if even possible)

You can iterate over resources via this code
using System.Collections;
using System.Globalization;
using System.Resources;
...
ResourceSet resourceSet = MyResourceClass.ResourceManager.GetResourceSet(CultureInfo.CurrentUICulture, true, true);
foreach (DictionaryEntry entry in resourceSet)
{
string resourceKey = entry.Key;
object resource = entry.Value;
}

You can use reflection, to get the values:
public class Something
{
public int Test1 { get; set; }
public int Test2 { get; set; }
public int Test3 { get; set; }
public int Test4 { get; set; }
}
var thing = new Something();
var imageProperties = typeof(Something)
.GetProperties()
.Where(p => p.Name.StartsWith("Test"));
var imagesToAdd = imageProperties
.Select(property => property.GetValue(thing))
.ToList();

You could define a property of type IEnumerable<Image> in the class of Resources object
public IEnumerable<Image> Images
{
get
{
yield return test1;
yield return test2;
yield return test3;
yield return test4;
yield return test5;
...
}
}
and then use it to fill ImageList
foreach(var image in test.Properties.Resources.Images)
{
ImageList.Add(image);
}

I just found out that there is a library for evaluating C# expression called Flee. Apparently you can use it to evaluate C# code so that you can loop over variable names, just like JavaScript, but the need for it most likely means a design flaw.
http://flee.codeplex.com/

C# List - Sorting Data

I am trying to add elements into a list, order them and then output them, there a number of "columns" if you like, per list
List<Info> infoList = new List<Info>();
while (dr.Read())
{
meeting_id = dr.GetValue(0).ToString();
try
{
Appointment appointment = Appointment.Bind(service, new ItemId(meeting_id));
Info data = new Info();
data.Start = appointment.Start;
data.Fruit = Convert.ToInt32(dr.GetValue(1));
data.Nuts = Convert.ToInt32(dr.GetValue(2));
infoList.Add(data);
}
Then to output it I want to order it by Start and then display all associated columns
for (int i = 0; i < infoList.Count; i++)
{
meet = meet + infoList[i];
}
First question: is the way I am inputting the data right?
Second question: How to I output all the columns to display all the associated columns? Is this possible? Is there a better practice?
Thanks
EDIT:
The class if you are interested:
public class Info
{
public DateTime Start { get; set; }
public int Fruit { get; set; }
public int Nuts { get; set; }
}

You can use Enumerable.OrderBy extension for enumerating your collection in some particular order (e.g. ordered by Start property value):
foreach(var info in infoList.OrderBy(i => i.Start))
{
// use info object here
// info.Fruits
// info.Nuts
}
BTW consider to add sorting on database side - that will be more efficient

How to compare two distinctly different objects with similar properties

This is all in C#, using .NET 2.0.
I have two lists of objects. They are not related objects, but they do have certain things in common that can be compared, such as a GUID-based unique identifier. These two lists need to be filtered by another list which just contains GUIDs which may or may not match up with the IDs contained in the first two lists.
I have thought about the idea of casting each object list to just object and sorting by that, but I'm not sure that I'll be able to access the ID property once it's cast, and I'm thinking that the method to sort the two lists should be somewhat dumb in knowing what the list to be sorted is.
What would be the best way to bring in each object list so that it can be sorted against the list with only the IDs?

You should make each of your different objects implement a common interface. Then create an IComparer<T> for that interface and use it in your sort.

Okay, if you have access to modify your original classes only to add the interface there, Matthew had it spot on. I went a little crazy here and defined out a full solution using 2.0 anonymous delegates. (I think I'm way addicted to 3.0 Lambda; otherwise, I probably would've written this out in foreach loops if I was using 2005 still).
Basically, create an interface with the common properties. Make yoru two classes implement the interface. Create a common list casted as the interface, cast and rip the values into the new list; remove any unmatched items.
//Program Output:
List1:
206aa77c-8259-428b-a4a0-0e005d8b016c
64f71cc9-596d-4cb8-9eb3-35da3b96f583
List2:
10382452-a7fe-4307-ae4c-41580dc69146
97f3f3f6-6e64-4109-9737-cb72280bc112
64f71cc9-596d-4cb8-9eb3-35da3b96f583
Matches:
64f71cc9-596d-4cb8-9eb3-35da3b96f583
Press any key to continue . . .
using System;
using System.Collections.Generic;
using System.Text;
namespace ConsoleApplication8
{
class Program
{
static void Main(string[] args)
{
//test initialization
List<ClassTypeA> list1 = new List<ClassTypeA>();
List<ClassTypeB> list2 = new List<ClassTypeB>();
ClassTypeA citem = new ClassTypeA();
ClassTypeB citem2 = new ClassTypeB();
citem2.ID = citem.ID;
list1.Add(new ClassTypeA());
list1.Add(citem);
list2.Add(new ClassTypeB());
list2.Add(new ClassTypeB());
list2.Add(citem2);
//new common list.
List<ICommonTypeMakeUpYourOwnName> common_list =
new List<ICommonTypeMakeUpYourOwnName>();
//in english, give me everything in list 1
//and cast it to the interface
common_list.AddRange(
list1.ConvertAll<ICommonTypeMakeUpYourOwnName>(delegate(
ClassTypeA x) { return (ICommonTypeMakeUpYourOwnName)x; }));
//in english, give me all the items in the
//common list that don't exist in list2 and remove them.
common_list.RemoveAll(delegate(ICommonTypeMakeUpYourOwnName x)
{ return list2.Find(delegate(ClassTypeB y)
{return y.ID == x.ID;}) == null; });
//show list1
Console.WriteLine("List1:");
foreach (ClassTypeA item in list1)
{
Console.WriteLine(item.ID);
}
//show list2
Console.WriteLine("\nList2:");
foreach (ClassTypeB item in list2)
{
Console.WriteLine(item.ID);
}
//show the common items
Console.WriteLine("\nMatches:");
foreach (ICommonTypeMakeUpYourOwnName item in common_list)
{
Console.WriteLine(item.ID);
}
}
}
interface ICommonTypeMakeUpYourOwnName
{
Guid ID { get; set; }
}
class ClassTypeA : ICommonTypeMakeUpYourOwnName
{
Guid _ID;
public Guid ID {get { return _ID; } set { _ID = value;}}
int _Stuff1;
public int Stuff1 {get { return _Stuff1; } set { _Stuff1 = value;}}
string _Stuff2;
public string Stuff2 {get { return _Stuff2; } set { _Stuff2 = value;}}
public ClassTypeA()
{
this.ID = Guid.NewGuid();
}
}
class ClassTypeB : ICommonTypeMakeUpYourOwnName
{
Guid _ID;
public Guid ID {get { return _ID; } set { _ID = value;}}
int _Stuff3;
public int Stuff3 {get { return _Stuff3; } set { _Stuff3 = value;}}
string _Stuff4;
public string Stuff4 {get { return _Stuff4; } set { _Stuff4 = value;}}
public ClassTypeB()
{
this.ID = Guid.NewGuid();
}
}
}

Using only .NET 2.0 methods:
class Foo
{
public Guid Guid { get; }
}
List<Foo> GetFooSubset(List<Foo> foos, List<Guid> guids)
{
return foos.FindAll(foo => guids.Contains(foo.Guid));
}
If your classes don't implement a common interface, you'll have to implement GetFooSubset for each type individually.

I'm not sure that I fully understand what you want, but you can use linq to select out the matching items from the lists as well as sorting them. Here is a simple example where the values from one list are filtered on another and sorted.
List<int> itemList = new List<int>() { 9,6,3,4,5,2,7,8,1 };
List<int> filterList = new List<int>() { 2, 6, 9 };
IEnumerable<int> filtered = itemList.SelectMany(item => filterList.Where(filter => filter == item)).OrderBy(p => p);

I haven't had a chance to use AutoMapper yet, but from what you describe you wish to check it out. From Jimmy Bogard's post:
AutoMapper conventions
Since AutoMapper flattens, it will
look for:
Matching property names
Nested property names (Product.Name
maps to ProductName, by assuming a
PascalCase naming convention)
Methods starting with the word “Get”,
so GetTotal() maps to Total
Any existing type map already
configured
Basically, if you removed all the
“dots” and “Gets”, AutoMapper will
match property names. Right now,
AutoMapper does not fail on mismatched
types, but for some other reasons.

I am not totally sure what you want as your end results, however....
If you are comparing the properties on two different types you could project the property names and corresponding values into two dictionaries. And with that information do some sort of sorting/difference of the property values.
Guid newGuid = Guid.NewGuid();
var classA = new ClassA{Id = newGuid};
var classB = new ClassB{Id = newGuid};
PropertyInfo[] classAProperties = classA.GetType().GetProperties();
Dictionary<string, object> classAPropertyValue = classAProperties.ToDictionary(pName => pName.Name,
pValue =>
pValue.GetValue(classA, null));
PropertyInfo[] classBProperties = classB.GetType().GetProperties();
Dictionary<string, object> classBPropetyValue = classBProperties.ToDictionary(pName => pName.Name,
pValue =>
pValue.GetValue(classB, null));
internal class ClassB
{
public Guid Id { get; set; }
}
internal class ClassA
{
public Guid Id { get; set; }
}
classAPropertyValue
Count = 1
[0]: {[Id, d0093d33-a59b-4537-bde9-67db324cf7f6]}
classBPropetyValue
Count = 1
[0]: {[Id, d0093d33-a59b-4537-bde9-67db324cf7f6]}

Thist should essentially get you what you want - but you may be better of using linq
class T1
{
public T1(Guid g, string n) { Guid = g; MyName = n; }
public Guid Guid { get; set; }
public string MyName { get; set; }
}
class T2
{
public T2(Guid g, string n) { ID = g; Name = n; }
public Guid ID { get; set; }
public string Name { get; set; }
}
class Test
{
public void Run()
{
Guid G1 = Guid.NewGuid();
Guid G2 = Guid.NewGuid();
Guid G3 = Guid.NewGuid();
List<T1> t1s = new List<T1>() {
new T1(G1, "one"),
new T1(G2, "two"),
new T1(G3, "three")
};
List<Guid> filter = new List<Guid>() { G2, G3};
List<T1> filteredValues1 = t1s.FindAll(delegate(T1 item)
{
return filter.Contains(item.Guid);
});
List<T1> filteredValues2 = t1s.FindAll(o1 => filter.Contains(o1.Guid));
}
}