Develop Reference

Dealing with VarVector<Single> Feature column in ML.NET

I have a tricky model I'm trying to normalize for binary classification. Here's an example of the model structure. I renamed a few things just to simplify.
public class Review
{
public bool Label { get; set; }
public ReviewItem ReviewItem { get; set; }
public List<OtherItem> OtherItems { get; set; }
}
public class ReviewItem
{
public string SomeText { get; set; }
public float SomeNumber { get; set; }
public bool SomeBool { get; set; }
}
public class OtherItem
{
public string SomeDifferentText { get; set; }
public float SomeDifferentNumber { get; set; }
public bool SomeDifferentBool { get; set; }
}
There can be any number of OtherItem in the List. This is what I tried to flatten the model a bit.
public class ReviewMlModel
{
public bool Label { get; set; }
public string ReviewItem_SomeText { get; set; }
public float ReviewItem_SomeNumber { get; set; }
public bool ReviewItem_SomeBool { get; set; }
public string[] OtherItem_SomeDifferentText { get; set; }
public float[] OtherItem_SomeDifferentNumber { get; set; }
public bool[] OtherItem_SomeDifferentBool { get; set; }
}
From there I tried this to normalize it:
var data = mlContext.Data.LoadFromEnumerable(allReviews);
var dataPrepEstimator = mlContext.Transforms.Text.FeaturizeText("ReviewItem_SomeText")
.Append(mlContext.Transforms.Text.FeaturizeText("OtherItem_SomeDifferentText"))
.Append(mlContext.Transforms.Conversion.ConvertType("ReviewItem_SomeBool"))
.Append(mlContext.Transforms.Conversion.ConvertType("OtherItem_SomeDifferentBool"))
.Append(mlContext.Transforms.Concatenate("Features",
"ReviewItem_SomeText", "OtherItem_SomeDifferentText", "ReviewItem_SomeBool",
"OtherItem_SomeDifferentBool", "ReviewItem_SomeNumber", "OtherItem_SomeDifferentNumber"));
var transformedData = dataPrepEstimator.Fit(data).Transform(data);
var model = mlContext.BinaryClassification.Trainers.AveragedPerceptron()
.Fit(transformedData);
This gives me the exception on the line where I try to create the model:
Schema mismatch for feature column 'Features': expected Vector<Single>, got VarVector<Single> (Parameter 'inputSchema')
I'm guessing this is due to the fact these arrays all have variable lengths, but I don't see a way to transform the VarVector. Do I need to go make the original allReviews enumerable have the same length array for every array? Or am I way off track with how I flattened the original model?

Looks like it was a combination of changing the arrays to all have the same size, as well as adding [VectorType(size)] attribute to all the array properties.

AutoMapper returns NULL when returning a list

Code without AutoMapper:
List<CountryDM> countryDMList = _countryRepo.GetCountry();
List<CountryVM> countryVMList = new List<CountryVM>();
foreach (CountryDM countryDM in countryDMList)
{
countryVMList.Add(CountryVM.ToViewModel(countryDM));
}
return countryVMList;
I used AutoMapper for the above task. But it returns a NULL list. Please refer the below code:
List<CountryDM> countryDMList = _countryRepo.GetCountry();
Mapper.CreateMap<List<CountryDM>, List<CountryVM>>();
List<CountryVM> countryVMList = new List<CountryVM>();
return Mapper.Map<List<CountryVM>>(countryDMList);
public class CountryDM
{
public int ID { get; set; }
public string CountryCode { get; set; }
public string Description { get; set; }
}
public class CountryVM
{
public int ID { get; set; }
public string CountryCode { get; set; }
public string Description { get; set; }
}

You don't need to define a mapping between lists, just between objects, AutoMapper will know how to extrapolate that:
Mapper.CreateMap<CountryDM, CountryVM>();
the rest stays the same

Remove null values from a List<classname>

I have a List which is getting filled correct, however after its being filled I want to remove all keys with a null value on a simple way, if exists, something like:
posts.RemoveAll(item => item == null);
I have multiple posts in this List<>
For me the problem is to find the Key/Value to access as in a Dictionary.
Does somebody know the way to do this?
UPDATE:
my class looks like this:
public class iPost
{
public int id { get; set; }
public int post_origin_post_id { get; set; }
public int pid { get; set; }
public int container_type_id { get; set; }
public int post_member_id { get; set; }
public int post_toGroup_id { get; set; }
public string title { get; set; }
public string comment { get; set; }
public string latitude { get; set; }
public string longitude { get; set; }
public string post_ip { get; set; }
public bool canShared { get; set; }
public bool isShared { get; set; }
public int share_type_id { get; set; }
public int views { get; set; }

Where is your key? A c#-List has no keys.
Do you mean somethong like this:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
namespace DemoApp
{
class Program
{
static void Main(string[] args)
{
List<DemoClass> list = new List<DemoClass>();
list.Add(new DemoClass() { Key = "A1" });
list.Add(new DemoClass() { Key = "A2" });
list.Add(new DemoClass() { Key = "A3" });
// Print
foreach (var it in list) { Console.WriteLine(it.Key); }
Console.WriteLine();
// set one key to null
list[1].Key = null;
list.RemoveAll(Item => Item.IsToDelete());
// Print
foreach (var it in list) { Console.WriteLine(it.Key); }
Console.WriteLine("As you can see, A2 is missing");
Console.ReadLine();
}
}
class DemoClass
{
public string Key
{
get;
set;
}
public object SomeValue
{
get;
set;
}
public bool IsToDelete()
{
if(Key == null || SomeValue == null)
{
return true;
}
return false;
}
}
}
Edit
You can write an own attribute, which is placed over all members who are not allwed to be null. Than you can write an class which read this information and than decise if your instance ha to be removed from the list. Absract example:
[ProofForNull]
public string Key
{
get;
set;
}
[ProofForNull]
public string Value1
{
get;
set;
}
[ProofForNull]
public string Value2
{
get;
set;
}
Than you change IsToDelete(). It has to get all member via reflection and proof if they have the [ProofForNull] attribute. If one member with the attribute is null, IsToDelete() returns true.
With this method you have one initial work to do, but the implementation for your classes is much easier.

C# Accessing a methods value dynamically using a string

I am currently setting some strings via this method:
string marketlabel = allmarketdata.#return.markets.COLXPM.label.ToString();
I would like to set the market label dynamically by having a string for the actual market choice.
string currentMarketSelected= this.marketTextBox.Text; // Specific market: COLXPM
string marketlabel=allmarketdata.#return.markets.currentMarketSelected.label.ToString();
I have been searching for a few hours and probably am not explaining correctly. I tried some stuff with reflections with no success. Basically what I want to do is have a textbox or list which contains all the market names and based on which one is selected start setting the data.
Above is the best type of example of what I want to do even though it is not syntactically possible to use a variable in place.
public class Markets
{
public COLXPM COLXPM { get; set; }
//Lots of markets below here
}
public class COLXPM
{
public string marketid { get; set; }
public string label { get; set; }
public string lasttradeprice { get; set; }
public string volume { get; set; }
public string lasttradetime { get; set; }
public string primaryname { get; set; }
public string primarycode { get; set; }
public string secondaryname { get; set; }
public string secondarycode { get; set; }
public List<Recenttrade> recenttrades { get; set; }
public List<Sellorder> sellorders { get; set; }
public List<Buyorder> buyorders { get; set; }
}
public class Return
{
public Markets markets { get; set; }
}
public class RootObject
{
public int success { get; set; }
public Return #return { get; set; }
}
The proposed solution below that worked
string currentMarketSelected = "DOGEBTC"; // Just selecting one of the markets to test it works
var property = allmarketdata.#return.markets.GetType().GetProperty(currentMarketSelected);
dynamic market = property.GetMethod.Invoke(allmarketdata.#return.markets, null);
string marketlabel = market.label.ToString(); //Gets all my selected market data

Here is a solution using reflection.
string currentMarketSelected= this.marketTextBox.Text; // Specific market: COLXPM
var property = allmarketdata.#return.markets.GetType().GetProperty(currentMarketSelected);
dynamic market = property.GetGetMethod().Invoke(allmarketdata.#return.markets, null);
string marketlabel=market.label.ToString();

You need something like this:
public class Markets
{
public COLXPM this[string key]
{
get
{
COLXPM colxpm;
switch (key)
{
// TODO : use "key" to select instance of COLXPM;
case "example1":
colxpm = ...;
break;
default:
throw new NotSupportedException();
}
return colxpm;
}
}
}
Then you can do something like:
string marketlabel=allmarketdata.#return.markets[currentMarketSelected]label.ToString();
This is an indexed property.

Updating List<T> in DbContext

I have a Model like this
public class Challenge
{
public int ID { get; set; }
public string Name { get; set; }
public string Blurb { get; set; }
public int Points { get; set; }
public string Category { get; set; }
public string Flag { get; set; }
public List<string> SolvedBy { get; set; }
}
public class ChallengeDBContext : DbContext
{
public DbSet<Challenge> Challenges { get; set; }
}
and then Controller like this. But I cannot update the List "SolvedBy", the next time I step through with the debugger, the list is still empty.
[HttpPost]
public string Index(string flag = "", int id=0)
{
Challenge challenge = db.Challenges.Find(id);
if (flag == challenge.Flag)
{
var chall = db.Challenges.Find(id);
if (chall.SolvedBy == null)
{
chall.SolvedBy = new List<string>();
}
chall.SolvedBy.Add(User.Identity.Name);
db.Entry(chall).State = EntityState.Modified;
db.SaveChanges();
//congrats, you solved the puzzle
return "got it";
}
else
{
return "fail";
}
}
is there any way around it to make a list of strings kept in the database?

EF don't know how to store an array in database table so it just ignore it. You can create another table/entity or use XML/JSON to store the list. You can serialize the list before saving and deserialize it after loading from database

A List<T> in a model would normally map to a second table, but in your DbContext you only have a single table. Try adding a second table.
public class ChallengeDBContext : DbContext
{
public DbSet<Challenge> Challenges { get; set; }
public DbSet<Solution> Solutions {get; set;}
}
public class Challenge
{
public int ID { get; set; }
public string Name { get; set; }
public string Blurb { get; set; }
public int Points { get; set; }
public string Category { get; set; }
public string Flag { get; set; }
public List<Solution> SolvedBy { get; set; }
}
public class Solution
{
public int ID { get; set; }
public string Name { get; set; }
}
Then your controller can use code along the lines of...
var chall = db.Challenges.Find(id);
if (chall.SolvedBy == null)
{
chall.SolvedBy = new List<Solution>();
}
chall.SolvedBy.Add(new Solution {Name=User.Identity.Name});
None of the above has been tested and I may have made some mistakes there, but the general principle I want to illustrate is the fact that you need another table. The List<T> represents a JOIN in SQL.