Develop Reference

Dynamic Datamodel JSON Binding C#

I have a json dataset that varies widely. it can have a field that is either an object or a list. how do I map these into a data model property if when I'm ingesting the data it can be either or. I can have a list of if else statements to check if a property is a list or object but for 10 fields that would grow rather big and make the code ugly.
field:{property1: 3, property2:4}
or
field:{[property1: 3, property2:4], [property1: 5, property2:6]}

if I understand you correctly, you receive data whose format is partially unknown to you.
var input = "{\"field\":{\"property1\": 3, \"property2\":4, \"property3\": 5, \"property4\":6}}";
var obj = JsonConvert.DeserializeObject<JObject>(input);
var res = new MyClass(){Fields = new List<MyField>()};
var field = obj.SelectToken("field") as JObject;
if(field != null)
{
foreach (var item in field.Properties())
{
res.Fields.Add(new MyField()
{
Name = item.Name,
Value = item.Value.Value<int>()
});
}
}
public sealed class MyField
{
public string Name { get; set; }
public object Value { get; set; }
}
public sealed class MyClass
{
public List<MyField> Fields { get; set; }
}

Filter a list of objects based on a parameter

class Officer
{
Person person;
}
class Person
{
string name;
}
Suppose I've a list of Officer and a list of Person. I want to filter these two list based on some criteria.
So I wrote this method:
public List<Person> filterName(List<Person> list)
{
// some filter logic
}
And I'm using this method for the two lists in the following way:
main()
{
...
List<Officer> officers = Initialize();
List<Person> validNames= filterNames(officers.Select(o=>o.person.name).ToList())
foreach (var officer in officers)
{
if (!validNames.Contains(officer.Person.name))
officers.remove(officer);
}
// finally I have a valid set of officers here
List<Person> persons = Initialize();
var filteredPersons = filterNames(persons.Select(o=>o.name).ToList())
}
Is there a good way to use generics so I can avoid the following code in the main method()?
List<string> validNames = filterNames(officers.Select(o=>o.fullName).ToList())
foreach (var officer in officers)
{
if (!validNames.Contains(officer.name))
officers.remove(officer);
}
And use generics somehow to update the officers list using generics.

New answer based on recent edits:
var officers = new List<Officer>
{
new Officer { Name = "Officer Foo" },
new Officer { Name = "Officer Bar" }
};
officers.RemoveAll(o => o.Name.Contains("Bar"));
// Officers now only contains "Officer Foo"
------------ Old answer here ----------------
Can you use OOP here and derive Person and Officer from something in common?
If so, then you can easily take their common property and filter on that instead of writing two separate pieces of logic to deal with each of them.
static void Main(string[] args)
{
var officers = new List<Officer>
{
new Officer { Name = "Officer Foo" },
new Officer { Name = "Officer Bar" }
};
var workers = new List<Worker>
{
new Worker { Name = "Worker Foo" },
new Worker { Name = "Worker Bar" }
};
var people = workers.Cast<IPerson>().Concat(officers);
var filteredPeople = Program.Filter(people, "Foo");
Console.ReadKey(true);
}
static IEnumerable<IPerson> Filter(IEnumerable<IPerson> people, string keyword)
{
return people.Where(p => p.Name.Contains(keyword));
}
interface IPerson
{
string Name { get; set; }
}
class Officer : IPerson
{
public string Name { get; set; }
}
class Worker : IPerson
{
public string Name { get; set; }
}

Ok, let's assume you have some complicated FilterNames function that operates on a list, and your goal is to filter out based on some Person criteria. I would rewrite the filter like this:
public bool FilterPerson(Person p)
{
//Some complicated logic
//Returns true if person should be kept
//Returns false if the person should be rejected
}
Now you can use that in a Linq statement:
var officers = Initialize().Where(o => FilterPerson(o.Person)).ToList();
No need to remove items from the list. You could still use the interim object, it just requires an additional step:
var officers = Initialize(); //Returns List<Officer>
var filteredOfficers = officers.Where(o => FilterPerson(o.Person)).ToList();

Populate SelectListItem from generic data type

I have the following code which returns results from a database table comprising of an Id field and a Name field, and transfers it to a list of SelectListItems (this populates a dropdown box in my view.)
var locationTypes = await APIHelper.GetAsync<List<LocationType>>(url);
var items = new List<SelectListItem>();
items.AddRange(locationTypes.Select(locationType =>
{
var item = new SelectListItem();
item.Value = locationType.LocationTypeId.ToString();
item.Text = locationType.Name;
return item;
}));
I am repeating this a lot throughout my application, substituting LocationType for various other things. The item.Value always gets the Id property of the data returned (the Id field is always in the format of {TableName}+"Id"), and the item.Text always gets ".Name" property.
How can I make this generic? I am trying to achieve something like this, although it is syntactically incorrect and may be the incorrect approach:
var myGenericObjects = await APIHelper.GetAsync<List<T>>(url)
var items = new List<SelectListItem>();
items.AddRange(myGenericObjects .Select(myGenericObjects =>
{
var item = new SelectListItem();
item.Value = myGenericObject.Field[0].ToString();
item.Text = myGenericObject.Name;
return item;
}));

You can create a custom extension for a generic list object, then, using reflection retrieve the values that you are wanting to map to the SelectListItem.Text and Name fields. Note I am using "nameof" in order to prevent any confusion or magic string representations of the properties to which I am trying to map.
I did define a default value of "Name" to the namePropertyName parameter. Per your description it sounded like, by convention, most of your DTOs have the property "Name" in them. If that's not the case simply remove the default value that is defined.
There are additional checks that could be made to this extension to prevent NullReference and ArgumentExceptions as well, but for simplicity of the example were left out. Example: Ensuring a value is provided in the idPropertyName and namePropertyName parameters and ensuring those property names exist on the provided generic object prior to conversion.
public static class ListExtensions
{
public static List<SelectListItem> ToSelectList<T>(this List<T> list, string idPropertyName, string namePropertyName = "Name")
where T : class, new()
{
List<SelectListItem> selectListItems = new List<SelectListItem>();
list.ForEach(item =>
{
selectListItems.Add(new SelectListItem
{
Text = item.GetType().GetProperty(namePropertyName).GetValue(item).ToString(),
Value = item.GetType().GetProperty(idPropertyName).GetValue(item).ToString()
});
});
return selectListItems;
}
}
Example Use:
var testList = new List<TestDto>
{
new TestDto { Name = "Test0", TestId = 0 },
new TestDto { Name = "Test1", TestId = 1 },
new TestDto { Name = "Test2", TestId = 2 },
new TestDto { Name = "Test3", TestId = 3 },
new TestDto { Name = "Test4", TestId = 4 },
};
var selectList = testList.ToSelectList(nameof(TestDto.TestId), nameof(TestDto.Name));
Here is the TestDto class for reference:
public class TestDto
{
public int TestId { get; set; }
public string Name { get; set; }
}

Some Prep Work
If you can change the table column names, then use a convention. For example, always name the "Value" column "X", and the "Text" column "Y" (give them better names). Then make all the classes for those tables implement an interface similar to this:
public interface ICustomLookup
{
string X { get; set; }
string Y { get; set; }
}
public class SomeClass : ICustomLookup
{
public string X { get; set; }
public string Y { get; set; }
}
Then an extension method like so:
public static class EnumerableExtension
{
public static SelectList ToSelectList(this IEnumerable<ICustomLookup> items)
{
return new SelectList(items.Select(thisItem => new SelectListItem
{
Text = thisItem.X,
Value = thisItem.Y
}));
}
}
Usage
var items = new List<SomeClass>
{
new SomeClass { X = "XOne", Y = "YOne" },
new SomeClass { X = "XTwo", Y = "YTwo" }
};
SelectList selectList = items.ToSelectList();

Updating Custom Class in List<T>

I am trying to update a List which is a List of Interfaces to concrete classes.
I add to the List each Market type i am interested in, for this Example these Markets are A and B
I loop over all the markets, (sample provided with 3 markets A B & C, we are only interested in A and B) And determine which is of interest to us.
Once found we pass this to an extraction method too do its work and create an instance of the Correct Market_ class type.
This all works fine, but when i try to update the list with the Updates it does not get reflected in the List.
Code below, any Suggestions?
Thanks
public class Test
{
public Test()
{
TheMarkets MarketsToUpdate = new TheMarkets();
List<SpecificCompanyMarket> lstMarks = new List<SpecificCompanyMarket>();
lstMarks.Add(new SpecificCompanyMarket(1234, "A", "Some HTML DATA HERE"));
lstMarks.Add(new SpecificCompanyMarket(5874, "B", "Some HTML DATA HERE"));
lstMarks.Add(new SpecificCompanyMarket(2224, "C", "Some HTML DATA HERE"));
foreach (var item in lstMarks)
{
if (MarketsToUpdate.IsMarketWeAreInterestedIn(item.MarketName))
{
ITheMarkets MarkToUpdate = ExtractMarketData(item);
var obj = MarketsToUpdate.MarketsWeAreInterestedIn.FirstOrDefault(x => x.MarketName() == "A");
if (obj != null)
{
obj = MarkToUpdate;
}
}
}
//Look At MarketsToUpdate Now and the item has not changed, still original values
//I was expecting to see the new values for the fields in A, not the default 0's
}
public ITheMarkets ExtractMarketData(SpecificCompanyMarket item)
{
ITheMarkets market = null;
if (item.MarketName.ToUpper() == "A")
{
Market_A marketType = new Market_A();
marketType.SomeValue1 = 123;
marketType.SomeValue2 = 158253;
market = marketType;
}
//Other Market extractions here
return market;
}
}
public class SpecificCompanyMarket
{
public int MarketId { get; set; }
public string MarketName { get; set; }
public string MarketDataHTML { get; set; }
public SpecificCompanyMarket(int MID, string MName, string MData)
{
MarketId = MID;
MarketName = MName;
MarketDataHTML = MData;
}
}
public class TheMarkets
{
public List<ITheMarkets> MarketsWeAreInterestedIn = new List<ITheMarkets>();
public TheMarkets()
{
Market_A A = new Market_A();
Market_B B = new Market_B();
MarketsWeAreInterestedIn.Add(A);
MarketsWeAreInterestedIn.Add(B);
}
public bool IsMarketWeAreInterestedIn(string strMarketName)
{
bool blnRetVal = false;
foreach (var item in MarketsWeAreInterestedIn)
{
if (item.MarketName().ToUpper().Trim().Equals(strMarketName.ToUpper().Trim()))
{
blnRetVal = true;
break;
}
}
return blnRetVal;
}
}
public interface ITheMarkets
{
string MarketName();
}
public class Market_A : ITheMarkets
{
public string LabelType { get; private set; }
public double SomeValue1 { get; set; }
public double SomeValue2 { get; set; }
public double SomeValue3 { get; set; }
public Market_A()
{
LabelType = "A";
}
public string MarketName()
{
return LabelType;
}
}
public class Market_B : ITheMarkets
{
public string LabelType { get; private set; }
public List<string> SomeList { get; set; }
public double SomeValue { get; set; }
public Market_B()
{
LabelType = "B";
}
public string MarketName()
{
return LabelType;
}
}

This is a short example to get you going. Loop through your list, find the object you want to update, create a new object of that type and then find the original objects index in the list and overwrite it in place. You are essentially just replacing the object in the list with a new one not mutating the existing one.
foreach (var item in lstMarks)
{
//your code to get an object with data to update
var yourObjectToUpdate = item.GetTheOneYouWant();
//make updates
yourObjectToUpdate.SomeProperty = "New Value";
int index = lstMarks.IndexOf(item);
lstMarks[index] = yourObjectToUpdate;
}

You are extracting an obj from marketWeAreInterestedIn list using LINQ's firstOrDefault extension. This is a new object and not a reference to the obj in that list. Therefore, no updates will be reflected in the object inside that list. Try using 'indexof'

You are not storing "list of interfaces" in your list. List<T> stores an array of pointers to objects that support T interface. Once you enumerate (with Linq in your case) your list, you copy a pointer from list, which is not associated with list itself in any way. It is just a pointer to your instance.
To do what you want, you will have to build new list while enumerating the original one, adding objects to it, according to your needs, so the second list will be based on the first one but with changes applied that you need.
You can also replace specific instance at specific index instead of building new list in your code, but to do this you will need to enumerate your list with for loop and know an index for each item:
list[index] = newvalue;
But there is a third solution to update list item directly by Proxying them. This is an example
class ItemProxy : T { public T Value { get; set; } }
var list = new List<ItemProxy<MyClass>>();
list.Insert(new ItemProxy { Value = new MyClass() });
list.Insert(new ItemProxy { Value = new MyClass() });
list.Insert(new ItemProxy { Value = new MyClass() });
foreach(var item in list)
if(item // ...)
item.Value = new MyClass(); // done, pointer in the list is updated.
Third is the best case for perfomance, but it will be better to use this proxying class for something more than just proxying.

Splitting a complex object to two separated key and value collections

I'm trying to implement the C# aspect of a LightWeight JSON Spec JsonR, but cannot get my head around any kind of recursion :-/ If anyone could help out here it would be more than greatly appreciated.
// Mockup class
public class User {
public string Name { get; set; }
public int Age { get; set; }
public List<string> Photos { get; set; }
public List<Friend> Friends { get; set; }
}
// Mockup class
public class Friend {
public string FirstName { get; set; }
public string LastName { get; set; }
}
// Initialize objects
var userList = new List<User>();
userList.Add(new User() {
Name = "Robert",
Age = 32,
Photos = new List<string> { "1.jpg", "2.jpg" },
Friends = new List<Friend>() {
new Friend() { FirstName = "Bob", LastName = "Hope"},
new Friend() { FirstName = "Mr" , LastName = "T"}
}
});
userList.Add(new User() {
Name = "Jane",
Age = 21,
Photos = new List<string> { "4.jpg", "5.jpg" },
Friends = new List<Friend>() {
new Friend() { FirstName = "Foo" , LastName = "Bar"},
new Friend() { FirstName = "Lady" , LastName = "Gaga"}
}
});
The idea behind it all is to now take the above object and split it into 2 separate collections, one containing the keys, and the other containing the values. Like this we can eventually only send the values over the wire thus saving lots of bandwidth, and then recombine it on the client (a js implementation for recombining exists already)
If all went well we should be able to get this out of the above object
var keys = new object[] {
"Name", "Age", "Photos",
new { Friends = new [] {"FirstName", "LastName"}}};
var values = new [] {
new object[] {"Robert", 32, new [] {"1.jpg", "2.jpg"},
new [] { new [] {"Bob", "Hope"},
new [] {"Mr", "T"}}},
new object[] {"Jane", 21, new [] {"4.jpg", "5.jpg"},
new [] { new [] {"Foo", "Bar"},
new [] {"Lady", "Gaga"}}}};
As a verification we can test the conformity of the result with
Newtonsoft.Json.JsonConvert.SerializeObject(keys).Dump("keys");
// Generates:
// ["Name","Age","Photos",{"Friends":["FirstName","LastName"]}]
Newtonsoft.Json.JsonConvert.SerializeObject(values).Dump("values");
// Generates:
// [["Robert",32,["1.jpg","2.jpg"],[["Bob","Hope"],["Mr","T"]]],["Jane",21,["4.jpg","5.jpg"],[["Foo","Bar"],["Lady","Gaga"]]]]
A shortcut i explored was to take advantage of Newton's JArray/JObject facilities like this
var JResult = Newtonsoft.Json.JsonConvert.DeserializeObject(
Newtonsoft.Json.JsonConvert.SerializeObject(userList));
Like this we end up with a sort of array object that we can already start iterating on
Anyone think they can crack this in a memory/speed efficient way ?

I have a solution that works with your example data. It is not a universal solution and may fail with other examples, but it shows how to use recursions. I did not include any error handling. A real-world solution would have to.
I use this helper method which gets the item type of the generic lists:
private static Type GetListItemType(Type listType)
{
Type itemType = null;
foreach (Type interfaceType in listType.GetInterfaces()) {
if (interfaceType.IsGenericType &&
interfaceType.GetGenericTypeDefinition() == typeof(IList<>)) {
itemType = interfaceType.GetGenericArguments()[0];
break;
}
}
return itemType;
}
Now, the recursion:
public void SplitKeyValues(IList source, List<object> keys, List<object> values)
{
Type itemType = GetListItemType(source.GetType());
PropertyInfo[] properties = itemType.GetProperties();
for (int i = 0; i < source.Count; i++) {
object item = source[i];
var itemValues = new List<object>();
values.Add(itemValues);
foreach (PropertyInfo prop in properties) {
if (typeof(IList).IsAssignableFrom(prop.PropertyType) &&
prop.PropertyType.IsGenericType) {
// We have a List<T> or array
Type genericArgType = GetListItemType(prop.PropertyType);
if (genericArgType.IsValueType || genericArgType == typeof(string)) {
// We have a list or array of a simple type
if (i == 0)
keys.Add(prop.Name);
List<object> subValues = new List<object>();
itemValues.Add(subValues);
subValues.AddRange(
Enumerable.Cast<object>(
(IEnumerable)prop.GetValue(item, null)));
} else {
// We have a list or array of a complex type
List<object> subKeys = new List<object>();
if (i == 0)
keys.Add(subKeys);
List<object> subValues = new List<object>();
itemValues.Add(subValues);
SplitKeyValues(
(IList)prop.GetValue(item, null), subKeys, subValues);
}
} else if (prop.PropertyType.IsValueType ||
prop.PropertyType == typeof(string)) {
// We have a simple type
if (i == 0)
keys.Add(prop.Name);
itemValues.Add(prop.GetValue(item, null));
} else {
// We have a complex type.
// Does not occur in your example
}
}
}
}
I call it like this:
List<User> userList = InitializeObjects();
List<object> keys = new List<object>();
List<object> values = new List<object>();
SplitKeyValues(userList, keys, values);
InitializeObjects initializes the user list as you did above.
UPDATE
The problem is that you are using an anonymous type new { Friends = ... }. You would have to create an anonymous type dynamically by using reflection. And that's pretty nasty. The article "Extend Anonymous Types using Reflection.Emit" seems to do it. (I didn't test it).
Maybe an easier approach would do the job. I suggest creating a helper class for the description of class types.
public class Class
{
public string Name { get; set; }
public List<object> Structure { get; set; }
}
Now let's replace an else case in the code above:
...
} else {
// We have a list or array of a complex type
List<object> subKeys = new List<object>();
var classDescr = new Class { Name = genericArgType.Name, Structure = subKeys };
if (i == 0)
keys.Add(classDescr);
List<object> subValues = new List<object>();
itemValues.Add(subValues);
SplitKeyValues(
(IList)prop.GetValue(item, null), subKeys, subValues);
}
...
The result is:

You may want to try an external tool, like AutoMapper, which is a convention based mapping library.
I suggest to try the flattening feature with your own conventions.
I'm sorry I cannot write out an example because of lack of time, but I think the ideas behind this open source library might help a lot.