I was trying out Json.net's ability to serialize and deserialize dictionaries, and thanks to this post I was able to find a good solution to serializing dictionaries in a simple way.
It was working great, but in a certain circumstance it broke in a (to me) nonsensical way such that I couldn't help but spend the next three hours debugging.
Here is the problem. When serializing this class
public class ReferenceTesting
{
public List<Scenario> scenarios = new List<Scenario>();
private Dictionary<Scenario, float> _Dict = new Dictionary<Scenario, float>();
[JsonProperty]
public List<KeyValuePair<Scenario, float>> SerializedDict
{
get { return _Dict.ToList(); }
set { _Dict = value.ToDictionary(x => x.Key, x => x.Value); }
}
public ReferenceTesting(int number = 0)
{
for (int i = 0; i < number; i++)
{
Scenario s1 = new Scenario();
scenarios.Add(s1);
_Dict.Add(s1, i);
}
}
public override string ToString()
{
string s = "";
for (int i = 0; i < scenarios.Count(); i++)
{
Scenario scenario = scenarios[i];
s += $"scenario{i} \n";
}
foreach (KeyValuePair<Scenario, float> scenario in SerializedDict)
{
s += $"Key: {scenario.Key}, Value: {scenario.Value} \n";
}
return s;
}
}
Everything works as expected, meaning when I instantiate
new Reference(3);
and then serialize and deserialize, I end up with an object with as expected 3 items in the list, and 3 items in the dictionary.
Output:
scenario0
scenario1
scenario2
Key: Scenario, Value: 0
Key: Scenario, Value: 1
Key: Scenario, Value: 2
However, by adding the default constructor
public ReferenceTesting() { }
the serialization works, writing out 3 items in list and dictionary, but deserialization does not work with the property. Meaning I end up with
scenario0
scenario1
scenario2
as output.
The big surprise with this is that the two constructors do the exact same thing - which is nothing when number = 0 (which it is when Json.net creates it, I doublechecked). So this means that the serializer has to be doing something under the hood to treat the property differently if there is or is not a default constructor.
How does Json.NET deserialize an object differently when it has a parameterized constructor vs. when it has a default constructor?
Json.NET is a streaming deserializer. Whenever possible it deserializes as it streams through the JSON rather than preloading the complete JSON into an intermediate representation before final deserialization.
Thus, when deserializing a JSON object with a default constructor, it first constructs the corresponding .Net object. It then recursively populates the object's .Net members by streaming through the key/value pairs in the JSON until the end of the JSON object. For each pair encountered, it finds the corresponding .Net member. If the value is a primitive type, it deserializes the primitive and sets the value. But if the value is a complex type (JSON object or array) it constructs the child object if necessary, sets the value back in the parent, and then populates it recursively as it continues to stream.
However, when deserializing an object with a parameterized constructor, Json.NET cannot use this streaming algorithm and instead must first fully deserialize the JSON object to an intermediate table of deserialized .Net name/value pairs, matching each JSON value to its corresponding .Net constructor argument or property by name and then deserializing to the type declared in .Net. Only then can the object be constructed by passing the deserialized constructor parameters into the constructor, and setting the remainder as property values.
For details on this process, see
JSON.net: how to deserialize without using the default constructor?
How does JSON deserialization in C# work.
(There is a third algorithm for ISerializable objects which does not apply in your case.)
Why is my surrogate public List<KeyValuePair<Scenario, float>> SerializedDict property not deserialized correctly when deserializing via a default constructor?
The reason is explained in this answer to Why are all the collections in my POCO are null when deserializing some valid json with the .NET Newtonsoft.Json component, and arises about of the specifics of Json.NET's Populate() algorithm:
It calls the getter in the parent class to get the current value of the property being deserialized.
If null, and unless a custom constructor is being used, it allocates an instance of the property's returned type (using the JsonContract.DefaultCreator method for the type).
It calls the setter in the parent to set the allocated instance back into the parent.
It proceeds to populate the instance of the type.
It does not set the instance back a second time, after it has been populated.
Thus the setter for SerializedDict is not called after the list is populated.
But when the parent class has a parameterized constructor, the property value SerializedDict is fully deserialized before its parent is constructed, so the setter is called with a fully populated surrogate list.
How can I create a surrogate collection property that works in both scenarios?
You can use an array instead of a list. Since an array cannot be resized, it must be fully deserialized and populated before it can be set back in the parent object:
public class ReferenceTesting
{
public KeyValuePair<Scenario, float> [] SerializedDict
{
get { return _Dict.ToArray(); }
set { _Dict = value.ToDictionary(x => x.Key, x => x.Value); }
}
// Remainder unchanged
You could make the array property be private if you want, by marking it with [JsonProperty].
By the way, your current deserialization creates duplicate Scenario objects in the scenarios and _Dict collections as shown by demo fiddle #1 here.
One way to fix this would be to serialize just _Dict (assuming that all scenarios are in the dictionary). Another would be to use PreserveReferencesHandling, e.g. by adding [JsonObject(IsReference = true)] to Scenario:
[JsonObject(IsReference = true)]
public class Scenario
{
// Remainder unchanged
}
Notes:
There is no standard for serialization of references in JSON. Json.NET's implementation may not match that of other serializers.
PreserveReferencesHandling doesn't work for objects with parameterized constructors (see here for details), so make sure Scenario doesn't have one.
Demo fiddle #2 here showing everything working correctly with a default constructor, and #3 here with a parameterized constructor.
Related
I'm developing an .NET Core 3.1 API, and I had a situation where I needed to iterate an object using foreach. To be able to do this, I used Reflection:
var properties = myClass.GetType().GetProperties();
After that, the code goes through the foreach as normal, and then I return the modified properties object to an external API, but it returns a timeout error message, I think it is because the PropertyInfo[] class isn't very appropriate for returning like this, or it's something else, I don't know.
Because of that, I want to convert properties "back" to myClass, or maybe convert it into an dictionary, it would be better to return the original class instead of PropertyInfo[].
How can I convert PropertyInfo[] into a class?
Thanks!
It sounds like you're trying to serialize this class to send to an API. What format does your API accept? If it accepts e.g. json, you should just use a json serializer (like JSON.net - which uses things like reflection under the hood anyways). If you are trying to serialize to a more exotic or unsupported format, then you can do it this way. What's missing is that the Properties[] array doesn't contain the values of your properties, only the definition. You can use the properties array to fetch the values:
public class MyClass
{
public int a { get; set;}
public string b {get; set; }
}
void Main()
{
var instance = new MyClass{a=1,b="2"};
var properties = instance.GetType().GetProperties();
var value_of_a = properties.First(p => p.Name == "a").GetValue(instance);
}
Each property has a "GetValue" method on it that can extract the value corresponding to that property from the class the property came from - note that up above you call 'GetType()' before you call 'GetProperties' - this means you're getting properties for the Type, not the instance. You then need to take those property definitions back to the instance to get the property values.
When deserializing from JSON to C# objects, why is there a setter property when you technically you are only "getting" the property from a deserialized JSON object? Wouldn't it be appropriate to just have:
public class Json
{
public string MyKey { get; }
}
Rather than:
public class Json
{
public string MyKey { get; set; }
}
I'm not 100% clear as to why I need the "set" property there for my use-case?
Thanks for the help.
When you are deserializing, you are taking the values from the json string and setting them in the c# object so you need the setters to populate the object.
If you were serializing the c# object to a json string, then you would only need the getter because you are getting the values from the c# object.
The JSon Deserializer has to create an instance of your objects and restore their state (the values of their properties) from the serialized Json Stream.
In order to do so, It needs to be able to inject those value in the instances, either by setting their properties (for which it needs a public setter) or by passing them as a parameter to an appropriate constructor (one which has a parameter of the same name and type as the value that needs to be set).
My server responses consists of a set of known and unknown properties. For the known ones, I created a DTO class with members for each property. The unknown properties shall be put inside a dictionary annotated with the [ExtensionData] attribute:
[JsonObject(MemberSerialization = MemberSerialization.OptIn)]
public class Dto
{
[JsonExtensionData]
private readonly Dictionary<string, object> unknownProperties = new Dictionary<string, object>();
public IDictionary<string, object> UnknownProperties
{
get
{
return new ReadOnlyDictionary<string, object>(this.unknownProperties);
}
}
[JsonProperty(Required = Required.Default, PropertyName = "KNOWN_PROPERTY")]
public string KnownProperty { get; private set; }
}
Null is allowed as value for KnownProperty. If I try to deserialize a JSON object that contains KNOWN_PROPERTY : null, this property is also contained in the dictionary UnknownProperties, if I configure the serializer with NullValueHandling.Ignore. This is done even though a class member exists for KNOWN_PROPERTY:
static void Main(string[] args)
{
string jsonString = #"{
KNOWN_PROPERTY : null,
UNKNOWN_PROPERTY : null
}";
JsonSerializer serializer = JsonSerializer.CreateDefault(new JsonSerializerSettings()
{
NullValueHandling = NullValueHandling.Ignore
});
using (var textReader = new StringReader(jsonString))
{
Dto dto = serializer.Deserialize<Dto>(new JsonTextReader(textReader));
foreach (var pair in dto.UnknownProperties)
{
Console.WriteLine("{0}: {1}", pair.Key, pair.Value == null ? "null" : pair.Value.ToString());
}
}
}
Output:
KNOWN_PROPERTY : null
UNKNOWN_PROPERTY : null
If I configure the serializer with NullValueHandling.Include or set a value for KNOWN_PROPERTY in the JSON string, the dictionary contains only UNKNOWN_PROPERTY, as expected.
For my understanding [ExtensionData] is not working correctly if NullValueHandling is set to ignore, since the documentation states the extension is used only if no matching class member is found.
Is the behavior I'm seeing intended? Can I do something to avoid this? Because I don't like to send null values to the server, I'd like to stick to the currently set NullValueHandling.
I'm using Json.NET 8.0.2
Update
Reported in JsonExtensionData should not include the null values that are real object properties. #1719 and fixed in commit e079301. The fix should be included in the next release of Json.NET after 11.0.2.
Original Answer
Confirmed - in JsonSerializerInternalReader.PopulateObject(object, JsonReader, JsonObjectContract, JsonProperty, string) there is the following logic:
// set extension data if property is ignored or readonly
if (!SetPropertyValue(property, propertyConverter, contract, member, reader, newObject))
{
SetExtensionData(contract, member, reader, memberName, newObject);
}
The intent seems to be to put the value into the extension data if the property is to be ignored, but Json.NET puts it into the extension data if the value is to be ignored -- a slightly different concept. I agree this could be a bug. You might want to report it.
There is a workaround. Json.NET has two attributes that affect how null/default values are serialized:
NullValueHandling. Specifies to include or ignore null values when serializing and deserializing objects. Values are Include and Ignore.
DefaultValueHandling. This has more elaborate semantics:
Include: Include members where the member value is the same as the member's default value when serializing objects. Included members are written to JSON. Has no effect when deserializing.
Ignore: Ignore members where the member value is the same as the member's default value when serializing objects so that is is not written to JSON. This option will ignore all default values (e.g. null for objects and nullable types; 0 for integers, decimals and floating point numbers; and false for booleans).
Populate: Members with a default value but no JSON will be set to their default value when deserializing.
IgnoreAndPopulate: Ignore members where the member value is the same as the member's default value when serializing objects and sets members to their default value when deserializing.
So, how do these overlapping settings interact? It turns out that Json.NET checks them both: serialize if both settings are agreeable, deserialize if both settings are agreeable. And DefaultValueHandling.IgnoreAndPopulate appears to do what you want -- it omits nulls when serializing, but reads and sets them when deserializing, if present.
Thus I was able to get your desired behavior with the following JsonProperty:
public class Dto
{
[JsonProperty(Required = Required.Default, PropertyName = "KNOWN_PROPERTY", DefaultValueHandling = DefaultValueHandling.IgnoreAndPopulate, NullValueHandling = NullValueHandling.Include)]
public string KnownProperty { get; private set; }
// Remainder as before.
}
Prototype fiddle.
I have a type like this:
class Foo<T>
{
public string Text { get; set; }
public T Nested { get; set; }
public static string ToJson(Foo<T> foo) { [...] }
}
ToJson serializes a Foo<Bar> instance to JSON in a way that is impossible to achieve by tweaking JsConfig. Also, ToJson relies on ServiceStack.Text to serialize Nested, which can be an instance of Foo<Baz>.
Unfortunately, the way JsConfig is implemented implies that there will be a JsConfig<T> set of static variables for Foo<Bar> and other for Foo<Baz>. Also, AFAIK, ServiceStack.Text offers no way to configure JSON serialization for open generic types (i.e.: something like JsConfig.Add(typeof(Foo<>), config)). I tried to solve this issue by creating this static constructor for Foo<T>:
static Foo() {
JsConfig<Foo<T>>.RawSerializeFn = ToJson;
}
This doesn't work all the time. It depends on the order the static constructors are invoked by the runtime. Apparently, ServiceStack.Text caches serializers functions and sometimes is doing it before the static constructor is called depending on the order operations are invoked in the API, so:
var outer = new Foo<Baz> { Text = "text" };
outer.ToJson(); // OK, because Nested is null
var inner = new Foo<Bar>();
inner.ToJson(); // OK, because JsConfig<Foo<Bar>>.RawSerializeFn is Foo<T>.ToJson
outer.Nested = inner;
outer.ToJson(); // NOT OK, because SS.Text uses the default serializer for Foo<T>, not Foo<T>.ToJson
I can't set all the serializers in JsConfig<Foo<T>> beforehand because T can be virtually any type, even other generic types.
Is it possible to define custom serialization routines for open generic types (that can be nested) in ServiceStack.Text?
I solved for this in my own way with a wrapper and a custom deserializer. I created a base type for all of my abstract types. That base type tells the system which type it is:
public class SetSettingItemBase
{
public string Key { get; set; }
public string ValueType { get; set; }
}
So the base is essentially the metadata -- the setting key + the value type. The object DTO, then, simply extends it by adding the actual value:
public class SetSettingItem : SetSettingItemBase
{
public object Value { get; set; }
}
Note that it's just an object. This is the DTO, not my actual object. I can cast it later, or convert it to a real/generic type after serialization.
My custom serialization then is:
JsConfig<SetSettingItem>.RawDeserializeFn = str =>
{
var baseSettings = str.FromJson<SetSettingItemBase>();
var ret = baseSettings.MapTo<SetSettingItem>();
if(true) // actual condition removed here... unimportant to the example
{
var dataType = Constants.KnownSettingsTypes[baseSettings.ValueType];
var method = typeof(JsonExtensions).GetMethod("JsonTo").MakeGenericMethod(dataType);
var key = "Value";
var parsed = JsonObject.Parse(str);
if(parsed.Object(key) == null)
key = "value";
ret.Value = method.Invoke(null, new object[] { parsed, key });
}
return ret;
};
This method first deserializes to the simple base. So the Value passed in from the DTO is ignored when deserializing baseSettings. I then call MapTo to prepare the actual SetSettingItem DTO. MapTo is just a wrapper around AutoMapper. You could just as easily use SS's built in mapper here.
For security, I have a set list of types that I allow as settings. Example:
KnownSettingsTypes.Add("string", typeof(string));
KnownSettingsTypes.Add("int", typeof(int));
KnownSettingsTypes.Add("nullableint", typeof(int?));
KnownSettingsTypes.Add("nullablepercentage", typeof(double?));
KnownSettingsTypes.Add("feegrid", typeof(FeeGrid));
After that, I use reflection to get the JsonTo method, passing in the generic type parameter dynamically from the KnownSettingsTypes dictionary.
And then finishing it all up, I parse the object using the generic JsonObject.Parse method and then looking for the Value or value (depending on case sensitivity) and explicitly convert that JsonObject using the dynamic method I created earlier.
The end result is I can pass in settings of all different types here as a part of my DTOs.
This served my purposes for the time being, but looking at the example I could see it improving in two ways:
After parsing, I could convert my SetSettingItem to a SettingItem<T> so I could use it as a strongly-typed object in my code. Remember, this example is just for the DTOs to get it across the wire.
Instead of requiring the person to pass in the type for me to check against, I could check against the setting Key to know which type it is supposed to be and parse accordingly. In my example, even if I check against the master list of settings and their types, I'd still probably require them to pass in the type just as a precaution and throw an exception if they didn't match.
I'm trying to serialize a class hierarchy to a Json string using DataContractJsonSerializer, in a WCF service.
the default behaviour for serializing a derived class is to add the following key value pair to the object:
"__type":"ClassName:#Namespace"
My problem is that namespaces are long and they bloat the Json string.
I would like to somehow intervene with the serialization and output this instead:
"__type":"ClassName"
and on deserialization intervene again to point to the correct namespace (which i know in runtime).
Is there any way to do such a thing?
This page describes the circumstances under which the __type property is emitted. In short, in WCF, if you use a derived type, and a KnownTypeAttribute, then you're going to get a __type property.
Example:
Assume
[DataContract]
[KnownType(typeof(Subscriber))]
public class Person { ... }
[DataContract]
public class Subscriber : Person { ... }
This code generates a __type property:
var o = new Subscriber("Fleming");
var serializer = new DataContractJsonSerializer(typeof(Person));
serializer.WriteObject(Console.OpenStandardOutput(), o);
But this code does not:
var o = new Subscriber("Fleming");
var serializer = new DataContractJsonSerializer(typeof(Subscriber));
serializer.WriteObject(Console.OpenStandardOutput(), o);
Notice that the second snip uses a DCJS with the same type as the object being serialized.
To avoid the __type, don't use derived types, or to be precise, use a serializer typed to the type you are actually serializing. If the serialization is being performed implicitly by a WCF method, then the method must be typed appropriately. In my example, it means you must use a return type of "Subscriber", and not the parent type, "Person".
The __type is emitted into the JSON stream by the (private) WriteServerTypeAttribute method on the
(internal) System.Runtime.Serialization.Json.XmlJsonWriter class. There is no public, documented, supported way to modify that, as far as I can tell.
To avoid this, you'd maybe need to return a string from the WCF method, perform the serialization yourself, and post-process the emitted JSON.
If you don't mind the __type thing, but just want to remove the qualifying namespace from the value, then put your types in the global namespace. In other words, put them outside of any namespace declaration in code.
Example: When the data types reside in a namespace, and when I used a derived type, the serialized JSON looks like this:
{
"__type":"Subscriber:#My.Custom.Namespace",
"Index":604455,
"Name":"Fleming",
"Id":580540
}
When the data types reside in the global namespace, it looks like this:
{
"__type":"Subscriber:#",
"Index":708759,
"Name":"Fleming",
"Id":675323
}
Adding the namespace parameter to the data contract does the trick.
[DataContract(Namespace = "")]
Cheeso's answer was excellent. I did discover a refinement to cleaning up the __type field though:
Rather than removing your subclass from its namespace you can add a property like the following:
[DataMember(Name = "__type")]
public string SubclassType
{
get
{
return "Subscriber";
}
set { }
}
You still get stuck with the ugly name "__type" but I found that because I was returning a list of subtypes I wanted to specify the type name anyway. You could even return a value of "" to further reduce response size. You could also just declare the property as:
public string __type
but I found that to accentuate the hack so I stuck with an appropriate property name and then renamed it.
-Joey
Note: I typed up this answer below and later realized that DataContractResolver is currently not supported with DataContractJsonSerializer. It may soon be with the next release of the framework, however. This is also useful if you are looking at more than just JSON.
**
You can do this with a DataContractResolver, which lets you map types to xsi:type (__type) information and vice-versa in a custom manner.
To do this, check out this blog post on DataContractResolver, plus this conceptual topic, plus this sample
#Cheeso wrote:
To avoid this, you'd maybe need to return a string from the WCF
method, perform the serialization yourself, and post-process the
emitted JSON.
Here's how I implemented that post-processing. I thought I'd post it here JIC it might help someone else.
First some boilerplate to show how I generate my JSON string:
// Instantiate & populate the object to be serialized to JSON
SomeClass xyz = new SomeClass();
... populate it ...
// Now serialize it
DataContractJsonSerializer ser = new DataContractJsonSerializer(xyz.GetType()); // Note xyz.GetType()
... serialize the object to json, many steps omitted here for brevity ...
string json = sr.ReadToEnd();
(Serialization is based on examples from https://msdn.microsoft.com/en-us/library/bb412179%28v=vs.110%29.aspx )
Note that the [DataContract] on SomeClass does not include the (name="") syntax that I've seen suggested elsewhere. That only removes the namespace from the __type at the cost of needing to adorn ALL your DataContract attrs, which clutters your code. Instead, my post-processor handles the assembly name in the __type field.
And now the string json contains the JSON text, but unfortunately includes all that "__type" junk that you don't want but can't suppress.
So here's my post-processing code that removes it:
// This strips out that unsuppressable __type clutter generated by the KnownType attributes
Attribute[] attrs = Attribute.GetCustomAttributes(xyz.GetType());
foreach (Attribute attr in attrs)
{
if (attr is KnownTypeAttribute)
{
KnownTypeAttribute a = (KnownTypeAttribute)attr;
string find = "\"__type\":\"" + a.Type.ReflectedType.Name + "." + a.Type.Name + ":#" + a.Type.Namespace + "\",";
json = json.Replace(find, "");
}
}
This makes a few assumptions, most notably that the __type field ends with a comma, which assumes that another field follows it, though (a) my objects always have at least 1 field and (b) I've found that the __type field is always 1st in the serialized object's output.
As always, you may have to adjust something to your situation, but I find it works well for mine.
Some times ago i decided this problem.
I use DataContractJsonSerializer
You will have __type in json, if your method for serialization have Base class parameter, but you give it subClass as parameter.
More details:
[DataContract]
[KnownType(typeof(B))]
public abstract class A
{
[DataMember]
public String S { get; set; }
}
[DataContract]
public class B : A
{
[DataMember]
public Int32 Age { get; set; }
}
public static String ToJson<T>(this T value)
{
var serializer = new DataContractJsonSerializer(typeof(T));
using (var stream = new MemoryStream())
{
serializer.WriteObject(stream, value);
return Encoding.UTF8.GetString(stream.ToArray());
}
}
You have two methods for test:
public static void ReadTypeDerived(A type)
{
Console.WriteLine(ToJson(type));
}
and
public static void ReadType<T>(T type)
{
Console.WriteLine(ToJson(type));
}
In first test you wiil have
"{\"__type\":\"B:#ConsoleApplication1\",\"S\":\"Vv\",\"Age\":10}"
In second:
"{\"S\":\"Vv\",\"Age\":10}"