Populate object where objects are reused and arrays are replaced? - c#

Using Json.NET, I'd like to map a JObject onto a .NET object with the following behavior:
For (non-null) array properties of the JObject, replace the entire collection on the target object.
For (non-null) object properties of the JObject, reuse the target object's property if it's not null, and map just the provided properties onto it.
JsonSerializer.Populate seems to be what I want, as described in this answer. As for the behaviors I'm looking for, it seems I can achieve one or the other, but not both, via JsonSerializerSettings.ObjectCreationHandling. ObjectCreationHandling.Replace does what I want with respect to requirement #1, while ObjectCreationHandling.Auto does what I want with respect to requirement #2, but it appends array items onto the existing collection.
What is the recommended way to achieve both requirements here?

Json.NET will automatically replace any arrays or read-only collections. To clear out read-write collections when deserializing, you could create a custom contract resolver that adds an OnDeserializingCallback to every modifiable collection that clears the collection when deserialization begins. Clearing the collection rather that replacing it outright handles cases where the collection is get-only, for instance:
public class RootObject
{
readonly HashSet<int> hashSet = new HashSet<int>();
public HashSet<int> HashSetValues { get { return this.hashSet; } }
}
The contract resolver is as follows:
public class CollectionClearingContractResolver : DefaultContractResolver
{
static void ClearGenericCollectionCallback<T>(object o, StreamingContext c)
{
var collection = o as ICollection<T>;
if (collection == null || collection is Array || collection.IsReadOnly)
return;
collection.Clear();
}
static SerializationCallback ClearListCallback = (o, c) =>
{
var collection = o as IList;
if (collection == null || collection is Array || collection.IsReadOnly)
return;
collection.Clear();
};
protected override JsonArrayContract CreateArrayContract(Type objectType)
{
var contract = base.CreateArrayContract(objectType);
if (!objectType.IsArray)
{
if (typeof(IList).IsAssignableFrom(objectType))
{
contract.OnDeserializingCallbacks.Add(ClearListCallback);
}
else if (objectType.GetCollectItemTypes().Count() == 1)
{
MethodInfo method = typeof(CollectionClearingContractResolver).GetMethod("ClearGenericCollectionCallback", BindingFlags.Static | BindingFlags.NonPublic | BindingFlags.Public);
MethodInfo generic = method.MakeGenericMethod(contract.CollectionItemType);
contract.OnDeserializingCallbacks.Add((SerializationCallback)Delegate.CreateDelegate(typeof(SerializationCallback), generic));
}
}
return contract;
}
}
public static class TypeExtensions
{
public static IEnumerable<Type> GetInterfacesAndSelf(this Type type)
{
if (type == null)
throw new ArgumentNullException();
if (type.IsInterface)
return new[] { type }.Concat(type.GetInterfaces());
else
return type.GetInterfaces();
}
public static IEnumerable<Type> GetCollectItemTypes(this Type type)
{
foreach (Type intType in type.GetInterfacesAndSelf())
{
if (intType.IsGenericType
&& intType.GetGenericTypeDefinition() == typeof(ICollection<>))
{
yield return intType.GetGenericArguments()[0];
}
}
}
}
public static class JsonExtensions
{
public static void Populate<T>(this JToken value, T target) where T : class
{
value.Populate(target, null);
}
public static void Populate<T>(this JToken value, T target, JsonSerializerSettings settings) where T : class
{
using (var sr = value.CreateReader())
{
JsonSerializer.CreateDefault(settings).Populate(sr, target);
}
}
}
Then to use it, do:
var settings = new JsonSerializerSettings
{
ContractResolver = new CollectionClearingContractResolver(),
};
jObject.Populate(rootObject, settings);
Sample fiddle.
Such a contract resolver would also be useful when deserializing objects that populate collections in their default constructor, as in Deserialization causes copies of List-Entries.

One fix is to use a custom JsonConverter that effectively replaces collections by ignoring the existing value when a collection type is detected.
public class ReplaceArrayConverter : JsonConverter
{
public override bool CanConvert(Type objectType) {
// check for Array, IList, etc.
return objectType.IsCollection();
}
public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer) {
// ignore existingValue and just create a new collection
return JsonSerializer.CreateDefault().Deserialize(reader, objectType);
}
public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer) {
JsonSerializer.CreateDefault().Serialize(writer, value);
}
}
Used like so:
var ser = JsonSerializer.CreateDefault(new JsonSerializerSettings {
Converters = new[] { new ReplaceArrayConverter() }
});
using (var reader = jObj.CreateReader()) {
ser.Populate(reader, model);
}

Related

How to force JsonConverter.WriteJson() to be called for a null value

I want to wrap some properties in a JSON object with some metadata, regardless if it's null or not. However, my custom JsonConverter.WriteJson override is not called in case the property is null.
What I get when property is not null:
{"Prop":{"Version":1, "Object":{"Content":"abc"}}}
What I get when it's null:
{"Prop":null}
What I want when it's null:
{"Prop":{"Version":1, "Object":null}}
Due to WriteJson never being called for null values, I do not get the opportunity to control this behavior. Is there any way to force this?
Note that I want to know if this is possible to do with e.g converters or contractresolvers, I can't/don't want to change the MyContent or Wrap classes (see below).
class VersioningJsonConverter : JsonConverter
{
//Does not get called if value is null !!
public override void WriteJson(JsonWriter writer, Object value, JsonSerializer serializer)
{
writer.WriteStartObject();
writer.WritePropertyName("v");
writer.WriteValue(1);
writer.WritePropertyName("o");
if(value == null)
{
//never happens
writer.WriteNull();
}
else
{
writer.WriteStartObject();
writer.WritePropertyName("Content");
writer.WriteValue((value as MyContent).Content);
writer.WriteEndObject();
}
writer.WriteEndObject();
}
public override Object ReadJson(JsonReader reader, Type objectType, Object existingValue, JsonSerializer serializer)
=> throw new NotImplementedException();
public override Boolean CanConvert(Type objectType) => objectType == typeof(MyContent);
public override Boolean CanRead => false;
}
public class MyContent
{
public String Content {get;set;}
}
public class Wrap
{
public MyContent Prop {get;set;}
}
There is no way currently to make Json.NET call JsonConverter.WriteJson() for a null value. This can be seen in JsonSerializerInternalWriter.SerializeValue(...) which immediately writes a null and returns for a null incoming value:
private void SerializeValue(JsonWriter writer, object value, JsonContract valueContract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerProperty)
{
if (value == null)
{
writer.WriteNull();
return;
}
// Remainder omitted
So if you need to translate null member(s) to non-null JSON value(s) but cannot modify the types themselves, you have two options:
Create a custom JsonConverter for the parent declaring type(s) of the member(s) that serializes every parent manually, OR
Create a custom contract resolver that translates the member(s) to ones returning some non-null surrogate or wrapper object.
Option #2 is more maintainable. The following contract resolver should do the job, wrapping the returned value of every member returning a value of the type(s) specified in the incoming list of types with the required version information:
public class CustomContractResolver : DefaultContractResolver
{
// Because contracts are cached, WrappedTypes must not be modified after construction.
readonly HashSet<Type> WrappedTypes = new HashSet<Type>();
public CustomContractResolver(IEnumerable<Type> wrappedTypes)
{
if (wrappedTypes == null)
throw new ArgumentNullException();
foreach (var type in wrappedTypes)
WrappedTypes.Add(type);
}
class VersionWrapperProvider<T> : IValueProvider
{
readonly IValueProvider baseProvider;
public VersionWrapperProvider(IValueProvider baseProvider)
{
if (baseProvider == null)
throw new ArgumentNullException();
this.baseProvider = baseProvider;
}
public object GetValue(object target)
{
return new VersionWrapper<T>(target, baseProvider);
}
public void SetValue(object target, object value) { }
}
class ReadOnlyVersionWrapperProvider<T> : IValueProvider
{
readonly IValueProvider baseProvider;
public ReadOnlyVersionWrapperProvider(IValueProvider baseProvider)
{
if (baseProvider == null)
throw new ArgumentNullException();
this.baseProvider = baseProvider;
}
public object GetValue(object target)
{
return new ReadOnlyVersionWrapper<T>(target, baseProvider);
}
public void SetValue(object target, object value) { }
}
protected override JsonProperty CreateProperty(MemberInfo member, MemberSerialization memberSerialization)
{
JsonProperty property = base.CreateProperty(member, memberSerialization);
if (WrappedTypes.Contains(property.PropertyType)
&& !(member.DeclaringType.IsGenericType
&& (member.DeclaringType.GetGenericTypeDefinition() == typeof(VersionWrapper<>) || member.DeclaringType.GetGenericTypeDefinition() == typeof(ReadOnlyVersionWrapper<>))))
{
var wrapperGenericType = (property.Writable ? typeof(VersionWrapper<>) : typeof(ReadOnlyVersionWrapper<>));
var providerGenericType = (property.Writable ? typeof(VersionWrapperProvider<>) : typeof(ReadOnlyVersionWrapperProvider<>));
var wrapperType = wrapperGenericType.MakeGenericType(new[] { property.PropertyType });
var providerType = providerGenericType.MakeGenericType(new[] { property.PropertyType });
property.PropertyType = wrapperType;
property.ValueProvider = (IValueProvider)Activator.CreateInstance(providerType, property.ValueProvider);
property.ObjectCreationHandling = ObjectCreationHandling.Reuse;
}
return property;
}
}
internal class VersionWrapper<T>
{
readonly object target;
readonly IValueProvider baseProvider;
public VersionWrapper(object target, IValueProvider baseProvider)
{
this.target = target;
this.baseProvider = baseProvider;
}
public int Version { get { return 1; } }
[JsonProperty(NullValueHandling = NullValueHandling.Include)]
public T Object
{
get
{
return (T)baseProvider.GetValue(target);
}
set
{
baseProvider.SetValue(target, value);
}
}
}
internal class ReadOnlyVersionWrapper<T>
{
readonly object target;
readonly IValueProvider baseProvider;
public ReadOnlyVersionWrapper(object target, IValueProvider baseProvider)
{
this.target = target;
this.baseProvider = baseProvider;
}
public int Version { get { return 1; } }
[JsonProperty(NullValueHandling = NullValueHandling.Include)]
public T Object
{
get
{
return (T)baseProvider.GetValue(target);
}
}
}
Then use it as follows to wrap all properties of type MyContent:
static IContractResolver resolver = new CustomContractResolver(new[] { typeof(MyContent) });
// And later
var settings = new JsonSerializerSettings
{
ContractResolver = resolver,
};
var json = JsonConvert.SerializeObject(wrap, Formatting.Indented, settings);
Notes:
You should statically cache the contract resolver for performance reasons explained here.
VersionWrapperProvider<T> creates a wrapper object with the necessary version information as well as a surrogate Object property that gets and sets the underlying value using Json.NET's own IValueProvider.
Because Json.NET does not set back the value of a pre-allocated reference property, but instead simply populates it with the deserialized property values, it is necessary for the setter of VersionWrapper<T>.Object to itself set the value in the parent.
If your wrapped types are polymorphic, in CreateProperty() you may need to check whether any of the base types of property.PropertyType are in WrappedTypes.
Populating a pre-existing Wrap using JsonConvert.PopulateObject should be tested.
This solution may not work when deserializing properties passed to parameterized constructors. DefaultContractResolver.CreatePropertyFromConstructorParameter would need modification in such a situation.
Working sample .Net fiddle here.

Deserialize two values into the same property

I have a client which can call two different versions of a service.
One service only sends a single value:
{
"value" : { ... }
}
The second service always returns multiple values:
{
"values" : [
{ ... },
{ ... }
]
}
Ideally, I'd like to represent this with a single object in my client classes so the user never sees whether it's a single value or multiple values.
public class MyValues
{
public List<Stuff> Values { get; set; }
public Thing Other { get; set; }
}
I think that the only way I'll be able to accomplish this is with a custom JsonConverter class which I apply to MyValues, but I really only want to do something custom when I'm deserializing the property value. I can't seem to figure out if an IContractResolver would be a better way to go (e.g. somehow attach a phantom property to MyValues that deserializes value and puts it into Values.
If I create a custom converter, how to I tell it to deserialize everything else normally (e.g. if Other has an extra properties make sure they are handled appropriately, etc.)
Rather than writing a JsonConverter, you could make a set-only property Value on your MyValues, like so:
public class MyValues
{
[JsonProperty]
Stuff Value
{
set
{
(Values = Values ?? new List<Stuff>(1)).Clear();
Values.Add(value);
}
}
public List<Stuff> Values { get; set; }
public Thing Other { get; set; }
}
It could be public or private if marked with [JsonProperty]. In this case Json.NET will call the Value setter if the singleton "value" property is encountered in the JSON, and call the Values setter if the array "values" property is encountered. Since the property is set-only only the array property will be re-serialized.
To make a custom JsonConverter that has special processing for a few properties of a type but uses default processing for the remainder, you can load the JSON into a JObject, detach and process the custom properties, then populate the remainder from the JObject with JsonSerializer.Populate(), like so:
class MyValuesConverter : CustomPropertyConverterBase<MyValues>
{
protected override void ProcessCustomProperties(JObject obj, MyValues value, JsonSerializer serializer)
{
// Remove the value property for manual deserialization, and deserialize
var jValue = obj.GetValue("value", StringComparison.OrdinalIgnoreCase).RemoveFromLowestPossibleParent();
if (jValue != null)
{
(value.Values = value.Values ?? new List<Stuff>()).Clear();
value.Values.Add(jValue.ToObject<Stuff>(serializer));
}
}
}
public abstract class CustomPropertyConverterBase<T> : JsonConverter where T : class
{
public override bool CanConvert(Type objectType)
{
return typeof(T).IsAssignableFrom(objectType);
}
public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
{
if (reader.TokenType == JsonToken.Null)
return null;
var jObj = JObject.Load(reader);
var contract = (JsonObjectContract)serializer.ContractResolver.ResolveContract(objectType);
var value = existingValue as T ?? (T)contract.DefaultCreator();
ProcessCustomProperties(jObj, value, serializer);
// Populate the remaining properties.
using (var subReader = jObj.CreateReader())
{
serializer.Populate(subReader, value);
}
return value;
}
protected abstract void ProcessCustomProperties(JObject obj, T value, JsonSerializer serializer);
public override bool CanWrite { get { return false; } }
public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer)
{
throw new NotImplementedException();
}
}
public static class JsonExtensions
{
public static JToken RemoveFromLowestPossibleParent(this JToken node)
{
if (node == null)
return null;
var contained = node.AncestorsAndSelf().Where(t => t.Parent is JContainer && t.Parent.Type != JTokenType.Property).FirstOrDefault();
if (contained != null)
contained.Remove();
// Also detach the node from its immediate containing property -- Remove() does not do this even though it seems like it should
if (node.Parent is JProperty)
((JProperty)node.Parent).Value = null;
return node;
}
}

In JSON.NET how to get a reference to every deserialized object?

I'm attempting to implement IDeserializationCallback using JSON.NET. I'm deserializing an object, and I would like to generate a list of all the objects which were deserialized which implement IDeserializationCallback, what would be the best way to do this? Does JSON.NET have any appropriate extension point to facilitate this? I have a (seemingly) working solution below, however it is quite ugly, so I'm convinced there must be a better way to do this. Any help is appreciated, thanks!
private static JsonSerializer serializer = new JsonSerializer();
static cctor()
{
serializer.Converters.Add(new DeserializationCallbackConverter());
}
public static T Deserialize<T>(byte[] data)
{
using (var reader = new JsonTextReader(new StreamReader(new MemoryStream(data))))
using (DeserializationCallbackConverter.NewDeserializationCallbackBlock(reader))
return serializer.Deserialize<T>(reader);
}
private class DeserializationCallbackConverter : JsonConverter
{
[ThreadStatic]
private static ScopedConverter currentConverter;
public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer)
{
throw new NotImplementedException();
}
public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
{
return currentConverter.ReadJson(reader, objectType, serializer);
}
public override bool CanConvert(Type objectType)
{
return currentConverter == null ? false : currentConverter.CanConvert();
}
public override bool CanWrite
{
get { return false; }
}
public static IDisposable NewDeserializationCallbackBlock(JsonReader reader)
{
return new ScopedConverter(reader);
}
private class ScopedConverter : IDisposable
{
private JsonReader jsonReader;
private string currentPath;
private List<IDeserializationCallback> callbackObjects;
public ScopedConverter(JsonReader reader)
{
jsonReader = reader;
callbackObjects = new List<IDeserializationCallback>();
currentConverter = this;
}
public object ReadJson(JsonReader reader, Type objectType, JsonSerializer serializer)
{
var lastPath = currentPath;
currentPath = reader.Path;
var obj = serializer.Deserialize(reader, objectType);
currentPath = lastPath;
var dc = obj as IDeserializationCallback;
if (dc != null && callbackObjects != null)
callbackObjects.Add(dc);
return obj;
}
public bool CanConvert()
{
return jsonReader.Path != currentPath;
}
public void Dispose()
{
currentConverter = null;
foreach (var obj in callbackObjects)
obj.OnDeserialization(null);
}
}
}
You can create a custom contract resolver that adds an extra, artificial OnDeserialized callback that tracks creation of reference type objects. Here's one example:
public interface IObjectCreationTracker
{
void Add(object obj);
ICollection<object> CreatedObjects { get; }
}
public class ReferenceObjectCreationTracker : IObjectCreationTracker
{
public ReferenceObjectCreationTracker()
{
this.CreatedObjects = new HashSet<object>();
}
public void Add(object obj)
{
if (obj == null)
return;
var type = obj.GetType();
if (type.IsValueType || type == typeof(string))
return;
CreatedObjects.Add(obj);
}
public ICollection<object> CreatedObjects { get; private set; }
}
public class ObjectCreationTrackerContractResolver : DefaultContractResolver
{
readonly SerializationCallback callback = (o, context) =>
{
var tracker = context.Context as IObjectCreationTracker;
if (tracker != null)
tracker.Add(o);
};
protected override JsonContract CreateContract(Type objectType)
{
var contract = base.CreateContract(objectType);
contract.OnDeserializedCallbacks.Add(callback);
return contract;
}
}
And then use it as follows:
public static class JsonExtensions
{
public static T DeserializeWithTracking<T>(string json, out ICollection<object> objects)
{
var tracker = new ReferenceObjectCreationTracker();
var settings = new JsonSerializerSettings
{
ContractResolver = new ObjectCreationTrackerContractResolver(),
Context = new StreamingContext(StreamingContextStates.All, tracker),
// Add other settings as required.
TypeNameHandling = TypeNameHandling.Auto,
};
var obj = (T)JsonConvert.DeserializeObject<T>(json, settings);
objects = tracker.CreatedObjects;
return obj;
}
}
Note that this only returns instances of non-string reference types. Returning instances of value types is more problematic as there is no obvious way to distinguish between a value type that eventually gets embedded into a larger object via a property setter and one that is retained in the object graph as a boxed reference, e.g. as shown in this question. If the boxed value type eventually gets embedded in some larger object there is no way to retain a direct reference to it.
Also note the use of StreamingContext.Context to pass the tracker down into the callback.
You may want to cache the contract resolver for best performance.
Update
In answer to the updated question of how to implement IDeserializationCallback with Json.NET, the above should work for reference types. For value types that implement this interface, you could:
Call the method immediately in the OnDeserialized callback rather than deferring it until serialization is complete, or
Throw an exception indicating that IDeserializationCallback is not supported for structs.

id field resolution when using JObject.FromObject

I have a family of custom Json Converters. They work like this:
public override void WriteJson(Newtonsoft.Json.JsonWriter writer, object value, Newtonsoft.Json.JsonSerializer serializer) {
JObject jo = JObject.FromObject(value);
// do my own stuff to the JObject here -- basically adding a property. The value of the property depends on the specific converter being used.
jo.WriteTo(writer, this);
}
The problem with this is that the id field of the JObject is always 1. Not good. So I tried using an inner serializer to get the id field:
private JsonSerializer _InnerSerializer {get;set;}
private JsonSerializer InnerSerializer {
get {
if (_InnerSerializer == null) {
_InnerSerializer = new JsonSerializer();
}
return _InnerSerializer;
}
}
public override void WriteJson(Newtonsoft.Json.JsonWriter writer, object value, Newtonsoft.Json.JsonSerializer serializer) {
JsonSerializer inner = this.InnerSerializer;
jo = JObject.FromObject(value, inner);
//my stuff here
jo.WriteTo(writer, this);
}
That gives a different id each time, even if it hits the same object twice. What I really want is to use Json's usual id resolution with my custom serialization. How can I do that?
Your idea of using an inner serializer will not work as-is. The id-to-object mapping table is held in a private field JsonSerializerInternalBase._mappings with no way to copy it from the outer serializer to the inner serializer.
As an alternative, you could make a recursive call to serialize using the same serializer, and have the converter disable itself using a thread-static pushdown stack along the lines of Generic method of modifying JSON before being returned to client and JSON.Net throws StackOverflowException when using [JsonConvert()]. You would need to enhance the converters in these examples to manually check for and add the necessary "$id" and "$ref" properties by making use of the JsonSerializer.ReferenceResolver property.
However, since your converters just add properties, a more straightforward solution to your problem might be to create a custom contract resolver that allows types to customize their contract as it is generated via a callback method declared in an attribute applied to the type, for instance:
public class ModifierContractResolver : DefaultContractResolver
{
// As of 7.0.1, Json.NET suggests using a static instance for "stateless" contract resolvers, for performance reasons.
// http://www.newtonsoft.com/json/help/html/ContractResolver.htm
// http://www.newtonsoft.com/json/help/html/M_Newtonsoft_Json_Serialization_DefaultContractResolver__ctor_1.htm
// "Use the parameterless constructor and cache instances of the contract resolver within your application for optimal performance."
// See also https://stackoverflow.com/questions/33557737/does-json-net-cache-types-serialization-information
static ModifierContractResolver instance;
// Explicit static constructor to tell C# compiler not to mark type as beforefieldinit
static ModifierContractResolver() { instance = new ModifierContractResolver(); }
public static ModifierContractResolver Instance { get { return instance; } }
protected override JsonObjectContract CreateObjectContract(Type objectType)
{
var contract = base.CreateObjectContract(objectType);
// Apply in reverse order so inherited types are applied after base types.
foreach (var attr in objectType.GetCustomAttributes<JsonObjectContractModifierAttribute>(true).Reverse())
{
var modifier = (JsonObjectContractModifier)Activator.CreateInstance(attr.ContractModifierType, true);
modifier.ModifyContract(objectType, contract);
}
return contract;
}
}
public abstract class JsonObjectContractModifier
{
public abstract void ModifyContract(Type objectType, JsonObjectContract contract);
}
[System.AttributeUsage(AttributeTargets.Class, AllowMultiple = true, Inherited = true)]
public class JsonObjectContractModifierAttribute : System.Attribute
{
private readonly Type _contractModifierType;
public Type ContractModifierType { get { return _contractModifierType; } }
public JsonObjectContractModifierAttribute(Type contractModifierType)
{
if (contractModifierType == null)
{
throw new ArgumentNullException("contractModifierType");
}
if (!typeof(JsonObjectContractModifier).IsAssignableFrom(contractModifierType))
{
throw new ArgumentNullException(string.Format("{0} is not a subtype of {1}", contractModifierType, typeof(JsonObjectContractModifier)));
}
this._contractModifierType = contractModifierType;
}
}
Then, apply it to your types as in the following example:
[JsonObjectContractModifier(typeof(TestContractModifier))]
public class Test
{
public string A { get; set; }
public string B { get; set; }
public string C { get; set; }
}
class TestContractModifier : JsonObjectContractModifier
{
class EmptyValueProvider : IValueProvider
{
// Explicit static constructor to tell C# compiler not to mark type as beforefieldinit
static EmptyValueProvider() { }
internal static readonly EmptyValueProvider Instance = new EmptyValueProvider();
#region IValueProvider Members
public object GetValue(object target)
{
var test = target as Test;
if (test == null)
return null;
return test.A == null && test.B == null && test.C == null;
}
public void SetValue(object target, object value)
{
var property = target as Test;
if (property == null)
return;
if (value != null && value.GetType() == typeof(bool) && (bool)value == true)
{
property.A = property.B = property.C = null;
}
}
#endregion
}
public override void ModifyContract(Type objectType, JsonObjectContract contract)
{
var jsonProperty = new JsonProperty
{
PropertyName = "isEmpty",
UnderlyingName = "isEmpty",
PropertyType = typeof(bool?),
NullValueHandling = NullValueHandling.Ignore,
Readable = true,
Writable = true,
DeclaringType = typeof(Test),
ValueProvider = EmptyValueProvider.Instance,
};
contract.Properties.Add(jsonProperty);
}
}
And serialize as follows:
var settings = new JsonSerializerSettings
{
PreserveReferencesHandling = PreserveReferencesHandling.Objects, // Or PreserveReferencesHandling.All
ContractResolver = ModifierContractResolver.Instance,
};
var json = JsonConvert.SerializeObject(root, Formatting.Indented, settings);
This produces the following JSON:
[
{
"$id": "1",
"A": "hello",
"B": "goodbye",
"C": "sea",
"isEmpty": false
},
{
"$ref": "1"
},
{
"$id": "2",
"A": null,
"B": null,
"C": null,
"isEmpty": true
},
}
As you can see, both the synthetic "isEmpty" property and reference handling properties are present. Prototype fiddle.

Strategies for migrating serialized Json.NET document between versions/formats

I'm using Json.Net to serialize some application data. Of course, the application specs have slightly changed and we need to refactor some of the business object data. What are some viable strategies to migrate previously serialized data to our new data format?
For example, say we have orignally had a business object like:
public class Owner
{
public string Name {get;set;}
}
public class LeaseInstrument
{
public ObservableCollection<Owner> OriginalLessees {get;set;}
}
We serialize an instance of a LeaseInstrument to a file with Json.Net. Now, we change our business objects to look like:
public class Owner
{
public string Name {get;set;}
}
public class LeaseOwner
{
public Owner Owner { get;set;}
public string DocumentName {get;set;}
}
public class LeaseInstrument
{
public ObservableCollection<LeaseOwner> OriginalLessees {get;set;}
}
I have looked into writing a custom JsonConverter for LeaseInstrument, but the ReadJson method is not ever hit...instead an exception is thrown before the deserializer reaches that point:
Additional information: Type specified in JSON
'System.Collections.ObjectModel.ObservableCollection`1[[BreakoutLib.BO.Owner,
BreakoutLib, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null]],
System, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089'
is not compatible with 'System.Collections.ObjectModel.ObservableCollection`1[[BreakoutLib.BO.LeaseOwner, BreakoutLib, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null]], System, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089'. Path 'Is.$values[8].OriginalLessors.$type', line 3142, position 120.
I mean, no joke, Json.Net, that's why I'm trying to run a JsonConverter when deserializing these objects, so I can manually handle the fact that the serialized type doesn't match the compiled type!!
For what it's worth, here are the JsonSerializerSettings we are using:
var settings = new JsonSerializerSettings
{
PreserveReferencesHandling = PreserveReferencesHandling.Objects,
ContractResolver = new WritablePropertiesOnlyResolver(),
TypeNameHandling = TypeNameHandling.All,
ObjectCreationHandling = ObjectCreationHandling.Reuse
};
You have the following issues:
You serialized using TypeNameHandling.All. This setting serializes type information for collections as well as objects. I don't recommend doing this. Instead I suggest using TypeNameHandling.Objects and then letting the deserializing system choose the collection type.
That being said, to deal with your existing JSON, you can adapt the IgnoreArrayTypeConverter from make Json.NET ignore $type if it's incompatible to use with a resizable collection:
public class IgnoreCollectionTypeConverter : JsonConverter
{
public IgnoreCollectionTypeConverter() { }
public IgnoreCollectionTypeConverter(Type ItemConverterType)
{
this.ItemConverterType = ItemConverterType;
}
public Type ItemConverterType { get; set; }
public override bool CanConvert(Type objectType)
{
// TODO: test with read-only collections.
return objectType.GetCollectItemTypes().Count() == 1 && !objectType.IsDictionary() && !objectType.IsArray;
}
public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
{
if (!CanConvert(objectType))
throw new JsonSerializationException(string.Format("Invalid type \"{0}\"", objectType));
if (reader.TokenType == JsonToken.Null)
return null;
var token = JToken.Load(reader);
var itemConverter = (ItemConverterType == null ? null : (JsonConverter)Activator.CreateInstance(ItemConverterType, true));
if (itemConverter != null)
serializer.Converters.Add(itemConverter);
try
{
return ToCollection(token, objectType, existingValue, serializer);
}
finally
{
if (itemConverter != null)
serializer.Converters.RemoveLast(itemConverter);
}
}
private static object ToCollection(JToken token, Type collectionType, object existingValue, JsonSerializer serializer)
{
if (token == null || token.Type == JTokenType.Null)
return null;
else if (token.Type == JTokenType.Array)
{
// Here we assume that existingValue already is of the correct type, if non-null.
existingValue = serializer.DefaultCreate<object>(collectionType, existingValue);
token.PopulateObject(existingValue, serializer);
return existingValue;
}
else if (token.Type == JTokenType.Object)
{
var values = token["$values"];
if (values == null)
return null;
return ToCollection(values, collectionType, existingValue, serializer);
}
else
{
throw new JsonSerializationException("Unknown token type: " + token.ToString());
}
}
public override bool CanWrite { get { return false; } }
public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer)
{
throw new NotImplementedException();
}
}
You need to upgrade your Owner to a LeaseOwner.
You can write a JsonConverter for this purpose that loads the relevant portion of JSON into a JObject, then checks to see whether the object looks like one from the old data model, or the new. If the JSON looks old, map fields as necessary using Linq to JSON. If the JSON object looks new, you can just populate your LeaseOwner with it.
Since you are setting PreserveReferencesHandling = PreserveReferencesHandling.Objects the converter will need to handle the "$ref" properties manually:
public class OwnerToLeaseOwnerConverter : JsonConverter
{
public override bool CanConvert(Type objectType)
{
return typeof(LeaseOwner).IsAssignableFrom(objectType);
}
public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
{
if (reader.TokenType == JsonToken.Null)
return null;
var item = JObject.Load(reader);
if (item["$ref"] != null)
{
var previous = serializer.ReferenceResolver.ResolveReference(serializer, (string)item["$ref"]);
if (previous is LeaseOwner)
return previous;
else if (previous is Owner)
{
var leaseOwner = serializer.DefaultCreate<LeaseOwner>(objectType, existingValue);
leaseOwner.Owner = (Owner)previous;
return leaseOwner;
}
else
{
throw new JsonSerializationException("Invalid type of previous object: " + previous);
}
}
else
{
var leaseOwner = serializer.DefaultCreate<LeaseOwner>(objectType, existingValue);
if (item["Name"] != null)
{
// Convert from Owner to LeaseOwner. If $id is present, this stores the reference mapping in the reference table for us.
leaseOwner.Owner = item.ToObject<Owner>(serializer);
}
else
{
// PopulateObject. If $id is present, this stores the reference mapping in the reference table for us.
item.PopulateObject(leaseOwner, serializer);
}
return leaseOwner;
}
}
public override bool CanWrite { get { return false; } }
public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer)
{
throw new NotImplementedException();
}
}
These use the extensions:
public static class JsonExtensions
{
public static T DefaultCreate<T>(this JsonSerializer serializer, Type objectType, object existingValue)
{
if (serializer == null)
throw new ArgumentNullException();
if (existingValue is T)
return (T)existingValue;
return (T)serializer.ContractResolver.ResolveContract(objectType).DefaultCreator();
}
public static void PopulateObject(this JToken obj, object target, JsonSerializer serializer)
{
if (target == null)
throw new NullReferenceException();
if (obj == null)
return;
using (var reader = obj.CreateReader())
serializer.Populate(reader, target);
}
}
public static class TypeExtensions
{
/// <summary>
/// Return all interfaces implemented by the incoming type as well as the type itself if it is an interface.
/// </summary>
/// <param name="type"></param>
/// <returns></returns>
public static IEnumerable<Type> GetInterfacesAndSelf(this Type type)
{
if (type == null)
throw new ArgumentNullException();
if (type.IsInterface)
return new[] { type }.Concat(type.GetInterfaces());
else
return type.GetInterfaces();
}
public static IEnumerable<Type> GetCollectItemTypes(this Type type)
{
foreach (Type intType in type.GetInterfacesAndSelf())
{
if (intType.IsGenericType
&& intType.GetGenericTypeDefinition() == typeof(ICollection<>))
{
yield return intType.GetGenericArguments()[0];
}
}
}
public static bool IsDictionary(this Type type)
{
if (typeof(IDictionary).IsAssignableFrom(type))
return true;
foreach (Type intType in type.GetInterfacesAndSelf())
{
if (intType.IsGenericType
&& intType.GetGenericTypeDefinition() == typeof(IDictionary<,>))
{
return true;
}
}
return false;
}
}
public static class ListExtensions
{
public static bool RemoveLast<T>(this IList<T> list, T item)
{
if (list == null)
throw new ArgumentNullException();
var comparer = EqualityComparer<T>.Default;
for (int i = list.Count - 1; i >= 0; i--)
{
if (comparer.Equals(list[i], item))
{
list.RemoveAt(i);
return true;
}
}
return false;
}
}
You can apply the converters directly to your data model using JsonConverterAttribute, like so:
public class LeaseInstrument
{
[JsonConverter(typeof(IgnoreCollectionTypeConverter), typeof(OwnerToLeaseOwnerConverter))]
public ObservableCollection<LeaseOwner> OriginalLessees { get; set; }
}
If you don't want to have a dependency on Json.NET in your data model, you can do this in your custom contract resolver:
public class WritablePropertiesOnlyResolver : DefaultContractResolver
{
protected override JsonProperty CreateProperty(MemberInfo member, MemberSerialization memberSerialization)
{
var result = base.CreateProperty(member, memberSerialization);
if (typeof(LeaseInstrument).IsAssignableFrom(result.DeclaringType) && typeof(ICollection<LeaseOwner>).IsAssignableFrom(result.PropertyType))
{
var converter = new IgnoreCollectionTypeConverter { ItemConverterType = typeof(OwnerToLeaseOwnerConverter) };
result.Converter = result.Converter ?? converter;
result.MemberConverter = result.MemberConverter ?? converter;
}
return result;
}
}
Incidentally, you might want to cache your custom contract resolver for best performance.
You might find our library Migrations.Json.Net helpful
https://github.com/Weingartner/Migrations.Json.Net
A Simple example. Say you start with a class
public class Person {
public string Name {get;set}
}
and then you want to migrate to
public class Person {
public string FirstName {get;set}
public string SecondName {get;set}
public string Name => $"{FirstName} {SecondName}";
}
you would perhaps do the following migration
public class Person {
public string FirstName {get;set}
public string SecondName {get;set}
public string Name => $"{FirstName} {SecondName}";
public void migrate_1(JToken token, JsonSerializer s){
var name = token["Name"];
var names = names.Split(" ");
token["FirstName"] = names[0];
token["SecondName"] = names[1];
return token;
}
}
The above glosses over some details but there is a full example on the homepage of the project. We use this extensively in two of our production projects. The example on the homepage has 13 migrations to a complex object that has changed over several years.

Categories

Resources