Develop Reference

Json.Net not serializing decimals the same way twice

I was testing Json.NET serialization of a shopping cart I'm working on and noticed that when I serialize -> deserialize -> serialize again, I'm getting a difference in the trailing zero formatting of some of the decimal fields. Here is the serialization code:
private static void TestRoundTripCartSerialization(Cart cart)
{
string cartJson = JsonConvert.SerializeObject(cart, Formatting.Indented);
Console.WriteLine(cartJson);
Cart cartClone = JsonConvert.DeserializeObject<Cart>(cartJson);
string cloneJson = JsonConvert.SerializeObject(cartClone, Formatting.Indented);
Console.WriteLine(cloneJson);
Console.WriteLine("\r\n Serialized carts are " + (cartJson == cloneJson ? "" : "not") + " identical");
}
The Cart implements IEnumerable<T> and has a JsonObjectAttribute to allow it to serialize as an object, including its properties as well as its inner list. The decimal properties of Cart do not change, but some of the decimal properties of objects and their inner objects in the inner list/array do as in this excerpt from output of the code above:
First time serializing:
...
"Total": 27.0000,
"PaymentPlan": {
"TaxRate": 8.00000,
"ManualDiscountApplied": 0.0,
"AdditionalCashDiscountApplied": 0.0,
"PreTaxDeposit": 25.0000,
"PreTaxBalance": 0.0,
"DepositTax": 2.00,
"BalanceTax": 0.0,
"SNPFee": 25.0000,
"cartItemPaymentPlanTypeID": "SNP",
"unitPreTaxTotal": 25.0000,
"unitTax": 2.00
}
}
],
}
Second time serializing:
...
"Total": 27.0,
"PaymentPlan": {
"TaxRate": 8.0,
"ManualDiscountApplied": 0.0,
"AdditionalCashDiscountApplied": 0.0,
"PreTaxDeposit": 25.0,
"PreTaxBalance": 0.0,
"DepositTax": 2.0,
"BalanceTax": 0.0,
"SNPFee": 25.0,
"cartItemPaymentPlanTypeID": "SNP",
"unitPreTaxTotal": 25.0,
"unitTax": 2.0
}
}
],
}
Notice the Total, TaxRate, and some of the others have changed from four trailing zeroes to a single trailing zero. I did find some stuff regarding changes to handling of trailing zeroes in the source code at one point, but nothing that I understood well enough to put together with this. I can't share the full Cart implementation here, but I built a bare bones model of it and couldn't reproduce the results. The most obvious differences were my bare bones version lost some additional inheritance/implementation of abstract base classes and interfaces and some generic type usage on those (where the generic type param defines the type of some of the nested child objects).
So I'm hoping without that someone can still answer: Any idea why the trailing zeroes change? The objects appear to be identical to the original after deserializing either JSON string, but I want to be sure there isn't something in Json.NET that causes a loss of precision or rounding that may gradually change one of these decimals after many serialization round trips.
Updated
Here's a reproducible example. I thought I had ruled out the JsonConverter but was mistaken. Because my inner _items list is typed on an interface, I have to tell Json.NET which concrete type to deserialize back to. I didn't want the actual Type names in the JSON so rather than using TypeNameHandling.Auto, I've given the items a unique string identifier property. The JsonConverter uses that to choose a concrete type to create, but I guess the JObject has already parsed my decimals to doubles? This is maybe my 2nd time implementing a JsonConverter and I don't have a complete understanding of how they work because finding documentation has been difficult. So I may have ReadJson all wrong.
[JsonObject]
public class Test : IEnumerable<IItem>
{
[JsonProperty(ItemConverterType = typeof(TestItemJsonConverter))]
protected List<IItem> _items;
public Test() { }
[JsonConstructor]
public Test(IEnumerable<IItem> o)
{
_items = o == null ? new List<IItem>() : new List<IItem>(o);
}
public decimal Total { get; set; }
IEnumerator IEnumerable.GetEnumerator()
{
return _items.GetEnumerator();
}
IEnumerator<IItem> IEnumerable<IItem>.GetEnumerator()
{
return _items.GetEnumerator();
}
}
public interface IItem
{
string ItemName { get; }
}
public class Item1 : IItem
{
public Item1() { }
public Item1(decimal fee) { Fee = fee; }
public string ItemName { get { return "Item1"; } }
public virtual decimal Fee { get; set; }
}
public class TestItemJsonConverter : JsonConverter
{
public override bool CanConvert(Type objectType) { return (objectType == typeof(IItem)); }
public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
{
object result = null;
JObject jObj = JObject.Load(reader);
string itemTypeID = jObj["ItemName"].Value<string>();
//NOTE: My real implementation doesn't have hard coded strings or types here.
//See the code block below for actual implementation.
if (itemTypeID == "Item1")
result = jObj.ToObject(typeof(Item1), serializer);
return result;
}
public override bool CanWrite { get { return false; } }
public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer) { throw new NotImplementedException(); }
}
class Program
{
static void Main(string[] args)
{
Test test1 = new Test(new List<Item1> { new Item1(9.00m), new Item1(24.0000m) })
{
Total = 33.0000m
};
string json = JsonConvert.SerializeObject(test1, Formatting.Indented);
Console.WriteLine(json);
Console.WriteLine();
Test test1Clone = JsonConvert.DeserializeObject<Test>(json);
string json2 = JsonConvert.SerializeObject(test1Clone, Formatting.Indented);
Console.WriteLine(json2);
Console.ReadLine();
}
}
Snippet from my actual converter:
if (CartItemTypes.TypeMaps.ContainsKey(itemTypeID))
result = jObj.ToObject(CartItemTypes.TypeMaps[itemTypeID], serializer);

If your polymorphic models contain decimal properties, in order not to lose precision, you must temporarily set JsonReader.FloatParseHandling to be FloatParseHandling.Decimal when pre-loading your JSON into a JToken hierarchy, like so:
public class TestItemJsonConverter : JsonConverter
{
public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
{
object result = null;
var old = reader.FloatParseHandling;
try
{
reader.FloatParseHandling = FloatParseHandling.Decimal;
JObject jObj = JObject.Load(reader);
string itemTypeID = jObj["ItemName"].Value<string>();
//NOTE: My real implementation doesn't have hard coded strings or types here.
//See the code block below for actual implementation.
if (itemTypeID == "Item1")
result = jObj.ToObject(typeof(Item1), serializer);
}
finally
{
reader.FloatParseHandling = old;
}
return result;
}
Demo fiddle here.
Why is this necessary? As it turns out, you have encountered an unfortunate design decision in Json.NET. When JsonTextReader encounters a floating-point value, it parses it to either decimal or double as defined by the above-mentioned FloatParseHandling setting. Once the choice is made, the JSON value is parsed into the target type and stored in JsonReader.Value, and the underlying character sequence is discarded. Thus, if a poor choice of floating-point type is made, it's difficult to correct the mistake later on.
So, ideally we would like to choose as a default floating-point type the "most general" floating point type, one that can be converted to all others without loss of information. Unfortunately, in .Net no such type exists. The possibilities are summarized in Characteristics of the floating-point types:
As you can see, double supports a larger range while decimal supports a larger precision. As such, to minimize data loss, sometimes decimal would need to be chosen, and sometimes double. And, again unfortunately, no such logic is built into JsonReader; there is no FloatParseHandling.Auto option to choose the most appropriate representation.
In the absence of such an option or the ability to load the original floating-point value as a string and re-parse it later, you will need to hardcode your converter with an appropriate FloatParseHandling setting based upon your data model(s) when you pre-load your JToken hierarchy.
In cases where your data models contain both double and decimal members, pre-loading using FloatParseHandling.Decimal will likely meet your needs, because Json.NET will throw a JsonReaderException when attempting to deserialize a too-large value into a decimal (demo fiddle here) but will silently round the value off when attempting to deserialize a too-precise value into a double. Practically speaking, it's unlikely you will have floating-point values larger than 10^28 with more than 15 digits of precision + trailing zeros in the same polymorphic data model. In the unlikely chance you do, by using FloatParseHandling.Decimal you'll get an explicit exception explaining the problem.
Notes:
I don't know why double was chosen instead of decimal as the "default default" floating point format. Json.NET was originally released in 2006; my recollection is that decimal wasn't widely used back then, so maybe this is a legacy choice that was never revisited?
When deserializing directly to a decimal or double member, the serializer will override the default floating-point type by calling ReadAsDouble() or ReadAsDecimal(), so precision is not lost when deserializing directly from a JSON string. The problem only arises when pre-loading into a JToken hierarchy then subsequently deserializing.
Utf8JsonReader and JsonElement from system.text.json, Microsoft's replacement for Json.NET in .NET Core 3.0, avoid this problem by always maintaining the underlying byte sequence of a floating-point JSON value, which is one example of the new API being an improvement on the old.
If you actually have values larger than 10^28 with more than 15 digits of precision + trailing zeros in the same polymorphic data model, switching to this new serializer might be a valid option.

JsonConverter to handle dynamic key names [duplicate]

This question already has answers here:
How can I parse a JSON string that would cause illegal C# identifiers?
(3 answers)
Closed 3 years ago.
I am working on code that processes responses from OpenLibrary. This is a rest service that returns books based on a passed in ISBN. An example URL is this:
https://openlibrary.org/api/books?bibkeys=ISBN:9780596005405&jscmd=data&format=json
{
ISBN:9780596005405: {
publishers: [
{
name: "O'Reilly"
}
],
pagination: "xxxii, 854 p. :",
identifiers: {
lccn: [
"2006281089"
],
openlibrary: [
"OL17924716M"
],
isbn_10: [
"0596005407"
],
goodreads: [
"58129"
],
librarything: [
"187028"
],
},
.... other properties omitted for brevity
}
I have these objects:
public class OLResult
{
Publishers Publishers { get; set; }
// other properties
}
public class Publishers
{
// properties go here
}
I created C# objects that are identical to what OpenLibrary returns. However, if you look at the response, you'll notice it's a JSON with a root element with a weird key: ISBN: followed by the passed in ISBN number. Newtonsoft.Json does not know how to map this dynamic key to my OLResult object.
I have created a simple converter to manually convert this JSON to the correct object:
public class OpenLibraryResultConverter : JsonConverter
{
public override bool CanConvert(Type objectType)
{
return true;
}
public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
{
OLResult result = new OLResult();
// Perform magic to copy JSON results to result object.
return result;
}
public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer)
{
throw new NotImplementedException();
}
}
I have also added an attribute [JsonConverter(typeof(OpenLibraryResultConverter))] to my OLResult class. When I call JsonConvert.DeserializeObject<OLResult>(jsonStringResult);, and use the debugger to check the ReadJson method, I can not seem to find the JSON data of the jsonStringResult parameter I passed in to the converter. Most properties of the parameters passed into the ReadJson method appear null or empty.
My question: how do I successfully read out the JSON string in my new JsonConverter?

In case you don't need that dynamic property number: ISBN:XXXXXX you can go with something simple like using partial JSON serialization that you can check out here

It's not clear from the post., but the first part is actually a label, so the overall structure is like this:
{
"ISBN:9780596005405": {...}
}
Note that the label contains a colon, which makes it more difficult to translate directly into a C# object.
But either way, you'll need to either define an object class for that and parse it, or as #Ernestas suggests, skip it.

Serialize json value types with type data

I've almost solved my problem but missing the last piece...
I Have a list of object and I want to be able to add value types (that normaly will be serialized to strings) and get them back as the original type.
For example: Guids or custom value types.
This is a sample custom value type:
public struct ExtString
{
private String Value
{
get;
set;
}
public static implicit operator ExtString(String str)
{
return !String.IsNullOrWhiteSpace(str) ? new ExtString(str) : null;
}
public static implicit operator String(ExtString exStr)
{
return exStr.ToString();
}
public ExtString(String str)
{
this.Value = str;
}
public override String ToString()
{
return this.Value;
}
}
This is the Custom converter:
public class CustomConverter : JsonConverter
{
public override Boolean CanConvert(Type objectType)
{
return objectType.IsValueType;
}
public override bool CanRead => false;
public override void WriteJson(JsonWriter writer, Object value, JsonSerializer serializer)
{
writer.WriteStartObject();
writer.WritePropertyName("$type");
writer.WriteValue(value.GetType().AssemblyQualifiedName);
writer.WritePropertyName("$value");
writer.WriteValue(value.ToString());
writer.WriteEndObject();
}
And here is the sample code for serializing / deserilizing:
var jsonSerializerSettings = new JsonSerializerSettings()
{
TypeNameHandling = TypeNameHandling.All,
Converters = new JsonConverter[]
{
new CustomConverter()
}
};
var list = new List<Object>();
list.Add(Guid.NewGuid());
list.Add((ExtString)"Hello World");
var ser = JsonConvert.SerializeObject(list, Formatting.Indented, jsonSerializerSettings);
var deser = JsonConvert.DeserializeObject(ser, jsonSerializerSettings);
This works almost all the way... Both the Guid and ExtString is serialized correct and the Guid is even deserialized with correct value (without special handling!), and the ExtString is created correct (on deserialize) but with the value null (constructor is called but str is null).
What am I missing? Why does it work for Guid?
Thanks.

All you have to do is to specify the JSON property name for your Value property.
public struct ExtString
{
[JsonProperty("$value")]
private String Value
{
...
More in depth
The Guid mapping works because it is considered a primitive type and Json.Net knows how to create a new instance and the parameter to pass to (using a JsonPrimitiveContract).
On the contrary, ExtString is resolved using a JsonObjectContract which calls the parameterless constructor of ExtString (it is there even if you don't declare it, because it's a value type) and then assigns the properties values to the corresponding json properties values. But the ExtString struct property name is Value while the JSON property name is $value. So a new istance of ExtString is created but the Value property remains null because there aren't properties with name Value. This is why, in your code, you have a new istance of ExtString with the Value property setted to null.
In the above solution I match the property name of the ExtString struct with the name of the property in the input JSON. After creating an instance of the ExtString struct, a mapping of the $value property will be done with success.
Another solution could be:
public struct ExtString
{
private String Value
{
get;
set;
}
[JsonConstructor]
public ExtString([JsonProperty("$value")] String str)
{
this.Value = str;
}
In this case, the constructor with the JsonConstructor attribute will be used instead of the parameterless one. Note that the str parameter must be provided with the appropriate JsonProperty attribute which defines the property name (in the input JSON) whose value will be passed to the constructor. If you omit the JsonProperty attribute, str parameter will be null and so your Value property too.
The difference between the two solutions is on the where the Value property is assigned. In the first solution the property is assigned after creating the object, in the second solution the property is assigned by the constructor.
Think it in this way:
//First solution
var myObject = new ExtString();
myObject.Value = "Hello World";
//Second solution
var myObject = new ExtString("Hello World");
I think the first solution give you much more control setting the value because (modifing the setter method) your logic will be always called, no matter how you create the object or when you assign the value.
Source: a pleasant and immersive analysis of the source code.
I know, my English is very bad :)

How do I deserialize a high-precision decimal value with Json.NET?

I want to deserialize JSON containing long decimal values into custom types to maintain their precision (i.e., a custom BigDecimal class). I'm using Json.NET 9.0.1 and .NET 4.6.1. I've tried using a JsonConverter, but it seems that the value available when ReadJson is called has already been identified and read by Json.NET as a .NET decimal type and is limited to its precision.
Ideally I would have access to the raw string so I could put it in a custom type. I can use string properties on the target object and it deserializes the full string, but then I'd have to further process the object (i.e., copy it into another representation) and that's especially messy across a large schema.
Any thoughts on a better approach?
Here's the target class:
public class DecimalTest
{
public string stringValue { get; set; }
public decimal decimalValue { get; set; }
public BigDecimal bigDecimalValue { get; set; }
}
Here's a test with JSON:
[TestMethod]
public void ReadBigDecimal_Test()
{
var json = #"{
""stringValue"" : 0.0050000012852251529693603515625,
""decimalValue"" : 0.0050000012852251529693603515625,
""bigDecimalValue"" : 0.0050000012852251529693603515625
}";
var settings = new JsonSerializerSettings();
settings.FloatParseHandling = FloatParseHandling.Decimal;
settings.Converters.Add(new JsonBigDecimalConverter());
var result = JsonConvert.DeserializeObject<DecimalTest>(json, settings);
Assert.IsNotNull(result);
Assert.AreEqual("0.0050000012852251529693603515625", result.stringValue);
Assert.AreEqual(0.0050000012852251529693603516m, result.decimalValue);
// *** This case fails ***
Assert.AreEqual("0.0050000012852251529693603515625", result.bigDecimalValue.ToString());
}
Here's the custom converter:
public class JsonBigDecimalConverter : JsonConverter
{
public override bool CanConvert(Type objectType)
{
return (objectType == typeof(BigDecimal));
}
public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
{
// *** reader.Value here already appears to be a .NET decimal.
// *** If I had access to the raw string I could get this to work.
return BigDecimal.Parse(reader.Value.ToString());
}
public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer)
{
throw new NotImplementedException();
}
}

Could you try if the following implementation of ReadJson works as you expect:
public override object ReadJson(
JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
{
var token = JToken.Load(reader);
return BigDecimal.Parse(token.ToString());
}
Update
Unfortunately the above won't work. There seems to be no way to read the raw string from the JSON data.
Also note that in my tests the assert for stringValue fails first. See this working example: https://dotnetfiddle.net/s0pqg3
I assume this is because Json.NET internally immediately parses any number token it encounters according to the specified FloatParseHandling. The raw data is never preserved.
I think the only solution is to wrap the big decimal string in quotes like so:
"bigDecimalValue" : "0.0050000012852251529693603515625"
Here is a working example that does exactly that in order to preserve the desired precision: https://dotnetfiddle.net/U1UG3z

Deserializing JSON using Newtonsoft in C#

I have the following JSON:
[
{
"name": "codeURL",
"value": "abcd"
},
{
"name": "authURL",
"value": "fghi"
}
]
I created the following objects:
public class ConfigUrlModel {
[JsonProperty("name")]
public abstract string name { get; set; }
[JsonProperty("value")]
public abstract string value { get; set; }
}
public class ConfigUrlsModel {
[JsonProperty]
public List<ConfigUrlModel> ConfigUrls { get; set; }
}
I am deserializing with the following line:
resultObject = Newtonsoft.Json.JsonConvert.DeserializeObject<ConfigUrlsModel>(resultString);
ConfigUrlsModel result = resultObject as ConfigUrlsModel;
I am getting the following error:
Exception JsonSerializationException with no inner exception: Cannot deserialize JSON array into type 'Microsoft.Xbox.Samples.Video.Services.Jtv.Models.ConfigUrl.ConfigUrlsModel'.
Exception JsonSerializationException with no inner exception: Cannot deserialize JSON array into type 'Microsoft.Xbox.Samples.Video.Services.Jtv.Models.ConfigUrl.ConfigUrlsModel'.
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.EnsureArrayContract(Type objectType, JsonContract contract)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateList(JsonReader reader, Type objectType, JsonContract contr at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.EnsureArrayContract(Type objectType, JsonContract contract)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateList(JsonReader reader, Type objectType, JsonContract contrNavigationService:OnNavigateToMessage PageSourceUri=/Microsoft.Xbox.Sample.UI;component/ErrorPrompt/ErrorPromptView.xaml
What am I doing wrong? How do I fix this?

The root JSON container is an array, not an object, so deserialize it thusly:
var configUrls = Newtonsoft.Json.JsonConvert.DeserializeObject<List<ConfigUrlModel>>(resultString);
var result = new ConfigUrlsModel { ConfigUrls = configUrls }; // If you still need the root object.
A JSON array is an ordered list of values [value1, value2, ..., value], which is what is shown in your question. Json.NET will convert .NET arrays and collections to JSON arrays, so you need to deserialize to a collection type.

The JSON you are sending is an array, but you are attempting to deserialize it to an object. Ether change your JSON so it matches the object defintion at the top level, and has a matching attribute, like this:
{
"ConfigUrls":[
{
"name":"codeURL",
"value":"abcd"
},
{
"name":"authURL",
"value":"fghi"
}
]
}
Or change your deserialize call to:
var urls = DeserializeObject<List<ConfigUrlModel>>(json);
This will return a List<ConfigUrlModel> which you can either use directly, or wrap in a ConfigUrlModels instance if you need to.
Additionally it is possible to deserialize this JSON directly to the desired class, by creating a custom newtonsoft JsonConverter sublcass. But that's going to make the code a little less clear, so avoid it if possible.