Develop Reference

Dotnet core -> Newtonsoft DeserializeObject<T> with Dynamic Child [duplicate]

Given following json result:
The default json result has a known set of fields:
{
"id": "7908",
"name": "product name"
}
But can be extended with additional fields (in this example _unknown_field_name_1 and _unknown_field_name_2) of which the names are not known when requesting the result.
{
"id": "7908",
"name": "product name",
"_unknown_field_name_1": "some value",
"_unknown_field_name_2": "some value"
}
I would like the json result to be serialized and deserialized to and from a class with properties for the known fields and map the unknown fields (for which there are no properties) to a property (or multiple properties) like a dictionary so they can be accessed and modified.
public class Product
{
public string id { get; set; }
public string name { get; set; }
public Dictionary<string, string> fields { get; set; }
}
I think I need a way to plug into a json serializer and do the mapping for the missing members myself (both for serialize and deserialize).
I have been looking at various possibilities:
json.net and custom contract resolvers (can't figure out how to do it)
datacontract serializer (can only override onserialized, onserializing)
serialize to dynamic and do custom mapping (this might work, but seems a lot of work)
let product inheriting from DynamicObject (serializers work with reflection and do not invoke the trygetmember and trysetmember methods)
I'm using restsharp, but any serializer can be plugged in.
Oh, and I cannot change the json result, and this or this didn't help me either.
Update:
This looks more like it: http://geekswithblogs.net/DavidHoerster/archive/2011/07/26/json.net-custom-convertersndasha-quick-tour.aspx

An even easier option to tackling this problem would be to use the JsonExtensionDataAttribute from JSON .NET
public class MyClass
{
// known field
public decimal TaxRate { get; set; }
// extra fields
[JsonExtensionData]
private IDictionary<string, JToken> _extraStuff;
}
There's a sample of this on the project blog here
UPDATE Please note this requires JSON .NET v5 release 5 and above

See https://gist.github.com/LodewijkSioen/5101814
What you were looking for was a custom JsonConverter

This is a way you could solve it, although I don't like it that much. I solved it using Newton/JSON.Net. I suppose you could use the JsonConverter for deserialization aswell.
private const string Json = "{\"id\":7908,\"name\":\"product name\",\"_unknown_field_name_1\":\"some value\",\"_unknown_field_name_2\":\"some value\"}";
[TestMethod]
public void TestDeserializeUnknownMembers()
{
var #object = JObject.Parse(Json);
var serializer = new Newtonsoft.Json.JsonSerializer();
serializer.MissingMemberHandling = Newtonsoft.Json.MissingMemberHandling.Error;
serializer.Error += (sender, eventArgs) =>
{
var contract = eventArgs.CurrentObject as Contract ?? new Contract();
contract.UnknownValues.Add(eventArgs.ErrorContext.Member.ToString(), #object[eventArgs.ErrorContext.Member.ToString()].Value<string>());
eventArgs.ErrorContext.Handled = true;
};
using (MemoryStream memoryStream = new MemoryStream(Encoding.UTF8.GetBytes(Json)))
using (StreamReader streamReader = new StreamReader(memoryStream))
using (JsonReader jsonReader = new JsonTextReader(streamReader))
{
var result = serializer.Deserialize<Contract>(jsonReader);
Assert.IsTrue(result.UnknownValues.ContainsKey("_unknown_field_name_1"));
Assert.IsTrue(result.UnknownValues.ContainsKey("_unknown_field_name_2"));
}
}
[TestMethod]
public void TestSerializeUnknownMembers()
{
var deserializedObject = new Contract
{
id = 7908,
name = "product name",
UnknownValues = new Dictionary<string, string>
{
{"_unknown_field_name_1", "some value"},
{"_unknown_field_name_2", "some value"}
}
};
var json = JsonConvert.SerializeObject(deserializedObject, new DictionaryConverter());
Console.WriteLine(Json);
Console.WriteLine(json);
Assert.AreEqual(Json, json);
}
}
class DictionaryConverter : JsonConverter
{
public DictionaryConverter()
{
}
public override bool CanConvert(Type objectType)
{
return objectType == typeof(Contract);
}
public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
{
throw new NotImplementedException();
}
public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer)
{
var contract = value as Contract;
var json = JsonConvert.SerializeObject(value);
var dictArray = String.Join(",", contract.UnknownValues.Select(pair => "\"" + pair.Key + "\":\"" + pair.Value + "\""));
json = json.Substring(0, json.Length - 1) + "," + dictArray + "}";
writer.WriteRaw(json);
}
}
class Contract
{
public Contract()
{
UnknownValues = new Dictionary<string, string>();
}
public int id { get; set; }
public string name { get; set; }
[JsonIgnore]
public Dictionary<string, string> UnknownValues { get; set; }
}
}

I thought I'd throw my hat in the ring since I had a similar problem recently. Here's an example of the JSON I wanted to deserialize:
{
"agencyId": "agency1",
"overrides": {
"assumption.discount.rates": "value: 0.07",
".plan": {
"plan1": {
"assumption.payroll.growth": "value: 0.03",
"provision.eeContrib.rate": "value: 0.35"
},
"plan2": {
".classAndTier": {
"misc:tier1": {
"provision.eeContrib.rate": "value: 0.4"
},
"misc:tier2": {
"provision.eeContrib.rate": "value: 0.375"
}
}
}
}
}
}
This is for a system where overrides apply at different levels and are inherited down the tree. In any case, the data model I wanted was something that would allow me to have a property bag with these special inheritance rules also supplied.
What I ended up with was the following:
public class TestDataModel
{
public string AgencyId;
public int Years;
public PropertyBagModel Overrides;
}
public class ParticipantFilterModel
{
public string[] ClassAndTier;
public string[] BargainingUnit;
public string[] Department;
}
public class PropertyBagModel
{
[JsonExtensionData]
private readonly Dictionary<string, JToken> _extensionData = new Dictionary<string, JToken>();
[JsonIgnore]
public readonly Dictionary<string, string> Values = new Dictionary<string, string>();
[JsonProperty(".plan", NullValueHandling = NullValueHandling.Ignore)]
public Dictionary<string, PropertyBagModel> ByPlan;
[JsonProperty(".classAndTier", NullValueHandling = NullValueHandling.Ignore)]
public Dictionary<string, PropertyBagModel> ByClassAndTier;
[JsonProperty(".bargainingUnit", NullValueHandling = NullValueHandling.Ignore)]
public Dictionary<string, PropertyBagModel> ByBarginingUnit;
[OnSerializing]
private void OnSerializing(StreamingContext context)
{
foreach (var kvp in Values)
_extensionData.Add(kvp.Key, kvp.Value);
}
[OnSerialized]
private void OnSerialized(StreamingContext context)
{
_extensionData.Clear();
}
[OnDeserialized]
private void OnDeserialized(StreamingContext context)
{
Values.Clear();
foreach (var kvp in _extensionData.Where(x => x.Value.Type == JTokenType.String))
Values.Add(kvp.Key, kvp.Value.Value<string>());
_extensionData.Clear();
}
}
The basic idea is this:
The PropertyBagModel on deserialization by JSON.NET has the ByPlan, ByClassAndTier, etc. fields populated and also has the private _extensionData field populated.
Then JSON.NET calls the private OnDeserialized() method and that will move the data from _extensionData to Values as appropriate (or drop it on the floor otherwise - presumably you could log this if it was something you wanted to know). We then remove the extra gunk from _extensionData so it doesn't consume memory.
On serialization, the OnSerializing method gets calls where we move stuff into _extensionData so it gets saved.
When serialization has finished, OnSerialized gets called and we remove the extra stuff from _extensionData.
We could further delete and recreate the _extensionData Dictionary when needed but I didn't see a real value in this as I'm not using tons of these objects. To do this we'd just create on OnSerializing and delete on OnSerialized. On OnDeserializing, instead of clearing, we could free it.

I was looking into a similar issue and found this post.
Here is a way to do it using reflection.
To make it more generic, one should check the type of the property instead of simply using ToString() in propertyInfo.SetValue, unless OFC all the actual properties are strings.
Also, lowercase property names is not standard in C# but given that GetProperty is case sensitive there are few other options.
public class Product
{
private Type _type;
public Product()
{
fields = new Dictionary<string, object>();
_type = GetType();
}
public string id { get; set; }
public string name { get; set; }
public Dictionary<string, object> fields { get; set; }
public void SetProperty(string key, object value)
{
var propertyInfo = _type.GetProperty(key);
if (null == propertyInfo)
{
fields.Add(key,value);
return;
}
propertyInfo.SetValue(this, value.ToString());
}
}
...
private const string JsonTest = "{\"id\":7908,\"name\":\"product name\",\"_unknown_field_name_1\":\"some value\",\"_unknown_field_name_2\":\"some value\"}";
var product = new Product();
var data = JObject.Parse(JsonTest);
foreach (var item in data)
{
product.SetProperty(item.Key, item.Value);
}

Dynamically convert property to Type

Is it possible to simplify this logic, Is there generic way to do it.
The code finds marked attributes and parses it according to the attribute type.
Please suggest some way to optimize this code, all the data type of Product class will be string, I'm getting product input as xml directly converting serialized data to a class with decimal,int,float will not give proper error message, If there is list of item it throws error in xml we wont know which row has caused the error.
using System;
using System.Collections.Generic;
using System.Linq;
using System.Reflection;
using System.Text;
using System.Threading.Tasks;
namespace TestSolution
{
public interface ICustomParser
{
bool Parse(string input);
}
public class DecimalParserAttribute : Attribute, ICustomParser
{
public bool Parse(string input)
{
if (input == null) return false;
decimal decimalValue;
return decimal.TryParse(input, out decimalValue);
}
}
public class intParserAttribute : Attribute, ICustomParser
{
public bool Parse(string input)
{
if (input == null) return false;
int intValue;
return int.TryParse(input, out intValue);
}
}
public class Product
{
[DecimalParser]
public string Weight { get; set; }
[intParser]
public string NoOfItems { get; set; }
[intParser]
public string GRodes { get; set; }
[intParser]
public string WRodes { get; set; }
}
class Program
{
static void Main(string[] args)
{
var sb = Validate(new Product() { NoOfItems = "1", GRodes = "4", Weight = "5", WRodes = "23" });
Console.WriteLine(sb);
sb = Validate(new Product() { NoOfItems = "1", GRodes = "4w", Weight = "5", WRodes = "23" });
Console.WriteLine(sb);
Console.ReadKey();
}
private static string Validate(Product product)
{
var sb = new StringBuilder();
foreach (var property in product.GetType().GetProperties())
{
var value = Convert.ToString(property.GetValue(product, null));
var sel = property.GetAttribute<ICustomParser>();
if (sel == null) continue;
var parserinstance = (ICustomParser)Activator.CreateInstance(sel.GetType());
if (parserinstance.Parse(value)) continue;
sb.AppendLine(string.Format("{0} Has invalid value", property.Name));
}
return sb.ToString();
}
}
public static class Extensions
{
public static T GetAttribute<T>(this PropertyInfo property)
{
return (T)property.GetCustomAttributes(false).Where(s => s is T).FirstOrDefault();
}
}
}

If you only have one type (Product), it probably isn't worth it - just write the validation code explicitly without all the fancy stuff. If, however, you have multiple types to inspect (perhaps known only at runtime):
It really depends how fast it needs to be. How often does this run? If it is periodic, then there isn't a vast amount to do - the main change would be to just cast the parser directly:
var parserInstance = (ICustomParser)sel;
(it already is the attribute type)
If it is critical path, then there is a lot you can do to beef it up, but you get into the realm of metaprogramming - which is essentially what most tools like serializers and ORMs do to reduce runtime reflection. If you're not familiar with hacking IL at runtime, I would recommend looking at a tool like "Sigil" (available on nuget) that makes it hard to get wrong (or at least: tells you what you've done wrong). Essentially, you can inspect the data structure and then emit the IL that matches what it would look like if you were doing it all in explicit code; for example, emitting IL that looks kinda like:
static readonly DecimalParserAttribute _decimal = new DecimalParserAttribute();
public static void Validate(Product product) {
var sb = new StringBuilder();
if(!_decimal.Parse(product.Weight)) {
sb.Append(...);
}
// ... etc
...,
}

Converting an anonymous object's properties and values to a Dictionary

I am creating a library for an existing API. I currently have QueryParameter classes for each request class. The QueryParameter classes are simple but they do vary (not all requests take the same query parameters).
Here is an example of a QueryParameter class:
public class ApiRequestAQueryParameters
{
public string Name { get; set; }
public int Start { get; set; }
public int Stop { get; set; }
}
I am interested in a way to convert a class like this into a Dictionary that I can feed to our Web client. I am hoping to have a reusable method like:
private Dictionary<string, string> GenerateQueryParameters(object queryParametersObject)
{
// perform conversion
}
This way I won't have to pull out the QueryParameter properties for each request (there will be dozens of requests)
The reason that I am using QueryParameter classes instead of making QueryParameter a Dictionary property of each API request class is to be developer friendly. I want to make it so that others can build these API requests by looking at the classes.

There are 2 ways: 1) use reflection and 2) serialize to json and back.
Here is the 1st method:
private Dictionary<string, string> GenerateQueryParameters(object queryParametersObject)
{
var res = new Dictionary<string, string>();
var props = queryParametersObject.GetType().GetProperties();
foreach (var prop in props)
{
res[prop.Name] = prop.GetValue(queryParametersObject).ToString();
}
return res;
}

You can do something like this:
private Dictionary<string, string> GenerateQueryParameters(object queryParameters)
{
var startStop = new StartStop() { Start = queryParameters.Start, Stop = queryParameters.Stop};
var result = new Dictionary<string, string>();
result.Add(queryParameters.Name, startStop);
return result;
}
public class StartStop
{
public int Start { get; set; }
public int Stop { get; set; }
}

This may be the perfect case to utilize ExpandoObjects. An ExpandoObject is a dynamic type, whose properties can be created at run time. ExpandoObject implements IDictionary < string, object > so it's easy to convert to a Dictionary < string, object > .
In the example below, an ExpandoObject is created and converted to a Dictionary < string, object > and then converted to a Dictionary < string, string >.
dynamic apiVar = new ExpandoObject();
apiVar.Name = "Test";
apiVar.Start = 1;
apiVar.Stop = 2;
var iDict = (IDictionary<string, object>) apiVar;
/* if you can utilize a Dictionary<string, object> */
var objectDict = iDict.ToDictionary(i => i.Key, i => i.Value);
/* if you need a Dictionary<string, string> */
var stringDict = iDict.ToDictionary( i=>i.Key, i=> i.Value.ToString());
There are also different ways of setting properties on an ExpandoObject. Below is an example of setting a property by a variable name.
dynamic apiVar = new ExpandoObject();
var propertyName = "Name";
apiVar[propertyName] = "Test";
propertyName = "Start";
apiVar[propertyName] = 1;
propertyName = "Stop";
apiVar[propertyName] = 2;

I always reuse the RouteValueDictionary class for this. It has a constructor that accepts any object and the class itself implements IDictionary.
It's available in the System.Web dll
private Dictionary<string, string> GenerateQueryParameters(object queryParametersObject)
{
return new RouteValueDictionary(queryParametersObject).ToDictionary(d => d.Key, d => Convert.ToString(d.Value));
}

Refactoring two lists of strings

I have two lists of strings. One is object friendly name and other one is object class name.
"Car","Animal","Plane"
"MachineClass","AnimalClass","FlyClass".
I use friendly names to show user input and class names to dynamically create class instances by using reflection, so i need both lists and like you can see "Car" as friendly name can be bound to "Machine.cs".
I need some more creative way of working and translating both of these lists rather than using switch statement which converts from one string to other, those lists have numerous items and i can make small spelling which will cause error. Moreover sometimes, i send string of test name for evaluation whereas same problem with spellings can apply there.
I tried to think about using enums but still digits doesn't ring a bell for conversions between both lists.

public enum Things
{
Car,
Animal,
Plane
}
var dict = new Dictionary<Things, string> {
{ Things.Car, "MachineClass" },
{ Things.Animal, "AnimalClass" },
{ Things.Plane, "FlyClass" } };
string classname = dict[Things.Plane]; // FlyClass
Now if you wanted real types, that you can instantiate:
var realtypes = dict.ToDictionary(
kvp => kvp.Key,
kvp => System.Type.GetType("Namespace." + kvp.Value));
A fully working example is on http://ideone.com/TTuBP:
using System;
using System.Collections.Generic;
using System.Linq;
//public interface IThing {}
public class MachineClass /* : IThing */ { }
public class AnimalClass /* : IThing */ { }
public class Plane /* : IThing */ { }
public class Program
{
public enum Things
{
Car,
Animal,
Plane
}
private static readonly IDictionary<Things, string> _classNameMap =
new Dictionary<Things, string> {
{ Things.Car, "MachineClass" },
{ Things.Animal, "AnimalClass" },
{ Things.Plane, "FlyClass" } };
public static void Main(string[] args)
{
var realtypes = _classNameMap.ToDictionary(
kvp => kvp.Key,
kvp => System.Type.GetType(/*"Namespace." +*/ kvp.Value));
Type dynamicType = realtypes[Things.Plane]; // typeof(Namespace.FlyClass)
foreach (var realtype in realtypes)
Console.WriteLine("{0}, class {1}",
realtype.Key, realtype.Value);
}
}

Why not use a dictionary?
Dictionary<string, string> classMap_ = new Dictionary<string, string>(StringComparer.OrdinalIgnoreCase);
classMap_["Car"] = "MachineClass";
classMap_["Animal"] = "AnimalClass";
classMap_["Plane"] = "FlyClass":
Creating the appropriate class is as simple as:
string className = classMap_["Car"];

Use maps HashMap to maintain friendly name and class name mapping

I don't recomend to use reflection for creation objects of classes from performance point of view, insteed you can use lambda expressions to create objects
look at the following code
var animals = new Dictionary<string, Func<object>>
{
{"Animal 1", () => new MyAnimal1()},
{"Animal 2", () => new MyAnimal2()}
};
var createAnimal = animals["Animal 1"];
var animal = createAnimal();
where
private class MyAnimal1
{
}
private class MyAnimal2
{
}
insteed this classes you should use your classes.

Why not using databinding?
class Wrapper
{
public string UserFriendly{get;set;}
public string Technical{get;set;}
}
var data = new[]
{
new Wrapper("Car", "MachineClass"),
new Wrapper("Plane", "FlyClass"),
};
combo.Datasource = data;
combo.DisplayMember = "UserFriendly";
combo.ValueMember = "Technical";
Combo will display the friendlyname but when asking for SelectedValue you will get the technical name.

NHibernate QueryByExample including only certain properties

I have created a custom Property Selector to accept an array in the constructor to say which properties should be included in the search. The approach works well as long as there are no component types, but how do I deal with those? Here is an example:
public class Customer
{
public virtual int Id { get; private set; }
public virtual Name Name { get; set; }
public virtual bool isPreferred { get; set; }
//...etc
}
public class Name
{
public string Title { get; set; }
public string Firstname { get; set; }
public string Lastname { get; set; }
public string Fullname { get; }
}
public class CustomerPropertySelector : Example.IPropertySelector
{
private string[] _propertiesToInclude = { };
public CustomerPropertySelector(string[] propertiesToInclude)
{
this._propertiesToInclude = propertiesToInclude;
}
public bool Include(object propertyValue, String propertyName, NHibernate.Type.IType type)
{
//...Checking for null and zeros etc, excluded for brevity
if (!_propertiesToInclude.Contains(propertyName))
return false;
return true;
}
}
I would like to be able to search by first name, but not necessarily last. The property name is Name however, so both first and last names seem to be part of the same property, and something like Name.Firstname, which would normally work as a criterion, doesn't seem to work here. What would be the best way around that?
EXAMPLE:
Customer exampleCust = new Customer(FirstName: "Owen");
IList<Customer> matchedCustomers = _custRepo.GetByExample(exampleCust, new string[] { "Name.FirstName" });
Given that there are 2 customers in db, only one named "Owen", but both have isPreferred = false, I would like my query to only return the first one. Standard QBE will return both based on isPreferred property.
SOLUTION:
Thank you for the answers, the solution is mostly based on answer by therealmitchconnors, however I couldn't have done it without Mark Perry's answer either.
The trick was to realise that rather than including Name.FirstName property, I actually want to exclude Name.LastName, since QBE only allows us to exclude properties. I used a method adapted from therealmitchconnors's answer to help me determine fully qualified names of properties. Here is the working code:
public IList<T> GetByExample(T exampleInstance, params string[] propertiesToInclude)
{
ICriteria criteria = _session.CreateCriteria(typeof(T));
Example example = Example.Create(exampleInstance);
var props = typeof(T).GetProperties();
foreach (var prop in props)
{
var childProperties = GetChildProperties(prop);
foreach (var c in childProperties)
{
if (!propertiesToInclude.Contains(c))
example.ExcludeProperty(c);
}
}
criteria.Add(example);
return criteria.List<T>();
}
private IEnumerable<string> GetChildProperties(System.Reflection.PropertyInfo property)
{
var builtInTypes = new List<Type> { typeof(bool), typeof(byte), typeof(sbyte), typeof(char),
typeof(decimal), typeof(double), typeof(float), typeof(int), typeof(uint), typeof(long),
typeof(ulong), typeof(object), typeof(short), typeof(ushort), typeof(string), typeof(DateTime) };
List<string> propertyNames = new List<string>();
if (!builtInTypes.Contains(property.PropertyType) && !property.PropertyType.IsGenericType)
{
foreach (var subprop in property.PropertyType.GetProperties())
{
var childNames = GetChildProperties(subprop);
propertyNames = propertyNames.Union(childNames.Select(r => property.Name + "." + r)).ToList();
}
}
else
propertyNames.Add(property.Name);
return propertyNames;
}
I wasn't sure of the best way to determine whether a property was a component class or not, any suggestions on how to improve the code are very welcome.

The following code would replace the logic you are using to populate propertiesToInclude. I changed it from an array to a list so I could use the Add method because I am lazy, but I think you get the picture. This does only work for one sub-level of properties. For n levels you would need to recurse.
List<string> _propertiesToInclude = new List<string>();
Type t;
var props = t.GetProperties();
foreach (var prop in props)
{
if (prop.PropertyType.IsClass)
foreach (var subprop in prop.PropertyType.GetProperties())
_propertiesToInclude.Add(string.Format("{0}.{1}", prop.Name, subprop.Name));
else
_propertiesToInclude.Add(prop.Name);
}

I thought I had something but reading your question again you want to know why the QBE NHibernate code doesn't work with component properties.
I think you need to create a sub-criteria query for the Name part.
Perhaps something like this:
public IList<Customer> GetByExample(Customer customer, string[] propertiesToExclude){
Example customerQuery = Example.Create(customer);
Criteria nameCriteria = customerQuery.CreateCriteria<Name>();
nameCriteria.Add(Example.create(customer.Name));
propertiesToExclude.ForEach(x=> customerQuery.ExcludeProperty(x));
propertiesToExclude.ForEach(x=> nameCriteria.ExcludeProperty(x));
return customerQuery.list();
}
This is an example from the NHibernate Test Project, it shows how to exclude Component properties.
[Test]
public void TestExcludingQBE()
{
using (ISession s = OpenSession())
using (ITransaction t = s.BeginTransaction())
{
Componentizable master = GetMaster("hibernate", null, "ope%");
ICriteria crit = s.CreateCriteria(typeof(Componentizable));
Example ex = Example.Create(master).EnableLike()
.ExcludeProperty("Component.SubComponent");
crit.Add(ex);
IList result = crit.List();
Assert.IsNotNull(result);
Assert.AreEqual(3, result.Count);
master = GetMaster("hibernate", "ORM tool", "fake stuff");
crit = s.CreateCriteria(typeof(Componentizable));
ex = Example.Create(master).EnableLike()
.ExcludeProperty("Component.SubComponent.SubName1");
crit.Add(ex);
result = crit.List();
Assert.IsNotNull(result);
Assert.AreEqual(1, result.Count);
t.Commit();
}
}
Source code link