I have a Serializer/Deserializer that use a references PreserveReferencesHandling = PreserveReferencesHandling.All.
The issue, is that I have circular references.
Here is a very simple example.
class Node
{
public Node(object value)
{
Value = value;
}
public object Value { get; set; }
public Node Left { get; set; }
public Node Right { get; set; }
}
My test scenario is:
var obj = new Node("o")
{
Left = new Node("oL"),
Right = new Node("oR")
};
obj.Right.Right = obj; // Circular reference!
When I deserialize, i have the following IReferenceResolver
private class InternalReferenceResolver : IReferenceResolver
{
private readonly Deserializer _par;
public InternalReferenceResolver(Deserializer par)
{
_par = par;
}
public object ResolveReference(object context, string reference)
{
object refObject;
if (!_par._processed.TryGetValue(reference, out refObject))
{
refObject = _par.DeserializeObject(reference);
}
return refObject;
}
public string GetReference(object context, object value)
{
throw new NotSupportedException("Only for Serialization");
}
public bool IsReferenced(object context, object value)
{
return false;
}
public void AddReference(object context, string reference, object value)
{
_par._processed.Add(reference, value);
}
}
As you can see, when JSON.NET inform me of a new ref->object (via AddReference()) I add it to a dictionary.
When JSON.NET request an object for a specific reference (via ResolveReference()) I recurse, and deserialize that reference.
The issue is, JSON.NET calls ResolveReference() for each of the object references, before it calls it's AddReference().
I would expect the flow of Deserialization to be:
Identify Object Type
Construct the object
AddReference(id, newObj)
Resolve References + Populate Properties
What I see happens is:
Identify Object Type
Resolve References
Construct the object
AddReference(id, newObj)
Populate Properties
My Questions:
Why is it made the latter, am i missing something with my suggested flow?
how can I overcome this issue, having a "Bare" object just for referencing and only then actually resolve the references ?
The Two-Phase deserialization you are seeing is arises because your Node class only has a parameterized constructor. As explained in Issue with serializing/deserializing object graph with self-referencing objects in combination with JSON constructor. #715:
JamesNK commented on Nov 28, 2015
Non-default constructors and preserving references don't work well together because the child values of a type have to be deserialized before the parent is created, so the reference resolves to null.
Thus, since Value is a mutable property anyway, you should add a parameterless constructor to Node:
class Node
{
public Node() : this(null) { }
public Node(object value)
{
Value = value;
}
// Remainder unchanged.
}
It can be non-public if you mark it with [JsonConstructor] or deserialize using the setting ConstructorHandling.AllowNonPublicDefaultConstructor. And if Value were immutable, you would need to make it privately settable and mark it with [JsonProperty]
[JsonProperty]
public object Value { get; private set; }
(Data contract attributes can be used in place of Json.NET attributes if you prefer.)
Notes:
Since your question lacks a complete and verifiable example, your code might encounter other problems that aren't fixed by adding a parameterless constructor.
For a related question, see Usage of non-default constructor breaks order of deserialization in Json.net.
For a related issue, see PreserveReferencesHandling.Objects deserialize does not work with non-default constructor #678.
Another thing, can I tell Json.NET to only handle the constructor parameters and leave the rest... ?
Not according to Issue 715. Since a JSON object is an unordered set of name/value pairs, Json.NET needs to parse the entire object to ensure it has loaded all the constructor parameters. As it is a single-pass serializer the non-constructor parameters will get loaded into... something... before the object can be constructed. Json.NET has chosen to deserialize them in one step to the final target member type rather than to an intermediate JToken and later the final member type. This can be see in JsonSerializerInternalReader.ResolvePropertyAndCreatorValues().
Well, I Found a solution for my problem:
The first Deserialization i perform, i use a custom IContractResolver that exclude all properties that are not relevant to the constructor...
at the second pass, I use Populate and use the default IContractResolver
private class InternalOnlyCtorContractResolver : IContractResolver
{
private readonly IContractResolver _base;
public InternalOnlyCtorContractResolver(IContractResolver _base)
{
this._base = _base;
}
public JsonContract ResolveContract(Type type)
{
var contract = _base.ResolveContract(type);
var objectContract = contract as JsonObjectContract;
if (objectContract != null)
{
var creatorParameters = new HashSet<string>(objectContract.CreatorParameters.Select(p => p.PropertyName));
var irrelevantProperties = objectContract.Properties
.Where(p => !creatorParameters.Contains(p.PropertyName))
.ToArray();
foreach (var irrelevantProperty in irrelevantProperties)
{
objectContract.Properties.Remove(irrelevantProperty);
}
//TODO Can be optimized better
}
return contract;
}
}
If for some reason, the Circular Reference is needed by the Constructor,
It still cause a loop, but it is kinda impossible to create without having a second constructor anyway.
Related
I am trying to get protobuf serialization working with the RedisSessionStateProvider. I have specified the redisSerializerType as a custom class which implements Microsoft.Web.Redis.ISerializer - here is the deserialization code:
public object Deserialize(byte[] data)
{
return DeserializeDirect(data);
}
private object DeserializeDirect(byte[] data)
{
using (var memoryStream = new MemoryStream(data))
{
return Serializer.Deserialize<object>(memoryStream);
}
return null;
}
As I need to implement Microsoft.Web.Redis.ISerializer the signature for deserialize uses a return type of object and there is no way to pass in the actual type being returned. So when DeserializeDirect tries to use the Protobuf.Serializer to deserialize it (as expected) says "Type is not expected, and no contract can be inferred: System.Object". I am using a web app with .NET framework 4.6.1 and I was hoping somebody could point out what I am doing wrong.
Thanks!
Normally, protobuf-net really wants to know the exact type. You can, however, cheat using DynamicType. This tells protobuf-net to include additional type metadata - something it doesn't usually include.
Note that this can make you code brittle - it may fail if the type changes in you code!
I will be implementing Any soon (as part of 2.3.0), which is another option here.
public static void Main()
{
// the actual object we care about
object obj = new Foo { X = 1 };
// serialize and deserialize via stub
var stub = new Stub { Data = obj };
var clone = Serializer.DeepClone(stub);
// prove it worked
Console.WriteLine(clone.Data);
// prove it is a different instance
Console.WriteLine(ReferenceEquals(obj, clone.Data));
}
[ProtoContract]
public class Foo
{
[ProtoMember(1)]
public int X { get; set; }
public override string ToString() => $"X={X}";
}
[ProtoContract]
public sealed class Stub
{
[ProtoMember(1, DynamicType = true)]
public object Data { get; set; }
}
When serializing an object, I have a List<T> property where T is an abstract type which I'd like to serialize naturally. However, when deserializing the object, I want/need to manually deserialize this abstract list. I'm doing the latter part with a custom JsonConverter which looks at a property on each item
in the list and then deserializes it to the correct type and puts it in the list.
But I also need to deserialize the rest of the object, without doing it property-by-property. Pseudo-code follows.
MyObject
class MyObject {
public Guid Id;
public string Name;
public List<MyAbstractType> Things;
}
MyObjectConverter
class MyObjectConverter : JsonConverter {
public override object ReadJson(reader, ..., serializer) {
// Line below will fail because the serializer will attempt to deserlalize the list property.
var instance = serializer.Deserialize(reader);
var rawJson = JObject.Load(reader); // Ignore reading from an already-read reader.
// Now loop-over and deserialize each thing in the list/array.
foreach (var item in rawJson.Value<JArray>(nameof(MyObject.Things))) {
var type = item.Value<string>(nameof(MyAbstractType.Type));
MyAbstractType thing;
// Create the proper type.
if (type == ThingTypes.A) {
thing = item.ToObject(typeof(ConcreteTypeA));
}
else if (type == ThingTypes.B) {
thing = item.ToObject(typeof(ConcreteTypeB));
}
// Add the thing.
instance.Things.Add(thing);
}
return instance;
}
}
To recap, I want to manually handle the deserialization of Things, but allow them to naturally serialize.
I've got a class which has been serialized into JSON, and which I'm trying to deserialize into an object.
e.g.
public class ContentItemViewModel
{
public string CssClass { get; set; }
public MyCustomClass PropertyB { get; set; }
}
the simple property (CssClass) will deserialize with:
var contentItemViewModels = ser.Deserialize<ContentItemViewModel>(contentItems);
But PropertyB gets an error...
We added a JavaScriptConverter:
ser.RegisterConverters(new List<JavaScriptConverter>{ publishedStatusResolver});
But when we added 'MyCustomClass' as a 'SupportedType', the Deserialize method was never called. However when we have ContentItemViewModel as the SupportedType, then Deserialize is called.
We've got a current solution which looks something like this:
class ContentItemViewModelConverter : JavaScriptConverter
{
public override object Deserialize(IDictionary<string, object> dictionary, Type type, JavaScriptSerializer serializer)
{
var cssClass = GetString(dictionary, "cssClass"); //I'm ommitting the GetString method in this example...
var propertyB= GetString(dictionary, "propertyB");
return new ContentItemViewModel{ CssClass = cssClass ,
PropertyB = new MyCustomClass(propertyB)}
}
public override IDictionary<string, object> Serialize(object obj, JavaScriptSerializer serializer)
{
throw new Exception("Only does the Deserialize");
}
public override IEnumerable<Type> SupportedTypes
{
get
{
return new List<Type>
{
typeof(ContentItemViewModel)
};
}
}
}
But we'd prefer a simpler solution of only deserializing MyCustomClass, as there are a number of other fields which are on the ViewModel, and it seems a waste to have to edit this converter every time we change/add a property....
Is there a way to Deserialize JUST PropertyB of type MyCustomClass?
Thanks for your help!
Have you considered using DatacontractJsonSerializer
[DataContract]
public class MyCustomClass
{
[DataMember]
public string foobar { get; set; }
}
[DataContract]
public class ContentItemViewModel
{
[DataMember]
public string CssClass { get; set; }
[DataMember]
public MyCustomClass PropertyB { get; set; }
}
class Program
{
static void Main(string[] args)
{
ContentItemViewModel model = new ContentItemViewModel();
model.CssClass = "StackOver";
model.PropertyB = new MyCustomClass();
model.PropertyB.foobar = "Flow";
//Create a stream to serialize the object to.
MemoryStream ms = new MemoryStream();
// Serializer the User object to the stream.
DataContractJsonSerializer ser = new DataContractJsonSerializer(typeof(ContentItemViewModel));
ser.WriteObject(ms, model);
byte[] json = ms.ToArray();
ms.Close();
string s= Encoding.UTF8.GetString(json, 0, json.Length);
Console.ReadLine();
}
}
Add all possible classes to DatacontractJsonSerializer.KnownTypes if MyCustomClass has derivations.
For whatever it may be worth after all this time, but I stumbled over the same problem and the solution is that the Deserializer hasn't got a clue about the classes you are deserializing unless you give him the necessary information.
On the top level, it knows the type from the type parameter of Deserialize<>(). That's why your converter for ContentItemViewModel works. For nested objects, it needs __type properties and a JavaScriptTypeResolver.
var ser = new JavaScriptSerializer(new SimpleTypeResolver());
ser.RegisterConverters(myconverters);
MyClass myObject = new MyClass();
string json = ser.Serialize(myObject);
// set a breakpoint here to see what has happened
ser.Deserialize<MyClass>(json);
A TypeResolver adds a __type property to each serialized object. You can write a custom type resolver that uses short names. In this sample, I use the SimpleTypeResolver from .net that "simply" stores the fully qualified type name as __type. When deserializing, the JavaScriptDeserializer finds __type and asks the TypeResolver for the correct type. Then it knows a type and can call a registered JavaScriptConverter.Deserialize method.
Without a TypeResolver, objects are deserialized to a Dictionary because JavaScriptSerializer doesn't have any type information.
If you can't provide a __type property in your json string, I think you'll need to deserialize to Dictionary first and then add a "guessing-step" that interprets the fields to find the right type. Then, you can use the ConvertToType method of JavaScriptSerializer to copy the dictionary into the object's fields and properties.
If you need to use the JavaScriptSerializer that is provides by ASP.NET and can't create your own, consider this section from the .ctor help of JavaScriptSerializer:
The instance of JavaScriptSerializer that is used by the asynchronous communication layer for invoking Web services from client script uses a special type resolver. This type resolver restricts the types that can be deserialized to those defined in the Web service’s method signature, or the ones that have the GenerateScriptTypeAttribute applied. You cannot modify this built-in type resolver programmatically.
Perhaps the GenerateScriptType Attribute can help you. But I don't know what kind of __type Properties are be needed here.
I have a list of objects that implement a common interface. If I try to simply serialize it I get a nice exception that tells me that the serializer cannot serialize interfaces:
private readonly ObservableCollection<ICanHasInterface> children = new ObservableCollection<ICanHasInterface>();
public ObservableCollection<ICanHasInterface> Children
{
get { return children; }
}
=> "Cannot serialize member ... of type ... because it is an interface"
Apparently asking the serializer to get the type of the objects and mark the XmlElement with the attribute xsi:type (which is done if an object inherits from another class) is too much.
So because I do not want to implement IXmlSerializable, I thought up a workaround which looked promising initially:
private readonly ObservableCollection<ICanHasInterface> children = new ObservableCollection<ICanHasInterface>();
[XmlIgnore()]
public ObservableCollection<ICanHasInterface> Children
{
get { return children; }
}
[XmlElement("Child")]
public List<object> ChildrenSerialized
{
get
{
return new List<object>(Children);
}
set
{
Children.Clear();
foreach (var child in value)
{
if (child is ICanHasInterface) AddChild(child as ICanHasInterface);
}
}
}
With this at least the serialisation works just fine (Note: Either specify XmlInclude attributes for the types that can be in the original list or hand over an array of types in the constructor of the serializer), however if the object is deserialized the Children collection ends up empty because the set block is never reached during deserialization, I am quite clueless as to why this is; any ideas?
On deserialization the serializer uses your property getter to get the collection instance and then calls Add() on it for each item. It does not call your property setter. Something like this:
YourClass c = new YourClass();
c.ChildrenSerialized.Add(ReadValue());
...
In order to keep the collections synchronized you'd need to customize the Add() behavior of the collection you return from the property getter.
A better option is to change the ChildrenSerialized property to use an object[]. For arrays, the serializer reads the value into an array and then calls your property setter with the value.
I am trying to migrate existing code that uses XmlSerializer to protobuf-net due to the increased performance it offers, however I am having problems with this specific case.
I have an object[] that includes parameters that are going to be sent to a remote host (sort of a custom mini rpc facility). I know the set of types from which these parameters can be, but I cannot tell in advance in which order they are going to be sent.
I have three constraints. The first is that I am running in Compact Framework, so I need something that works there. Second, as I mentioned performance is a big concern (on the serializing side) so I would rather avoid using a lot of reflection there if possible. And the most important is that I care about the order in which this parameters were sent.
Using XmlSerializer it was easy just adding XmlInclude, but for fields there is nothing equivalent as far as I know in Protobuf-net. So, is there a way to do this? Here is a simplified example.
[Serializable]
[XmlInclude(typeof(MyType1)),
XmlInclude(typeof(MyType2)),
XmlInclude(typeof(MyType3))
public class Message()
{
public object[] parameters;
public Message(object[] parms)
{
parameters = parms;
}
}
Message m = new Message(new object[] {MyType1(), 33, "test",
new MyType3(), new MyType3()});
MemoryStream ms = new MemoryStream();
XmlSerializer xml = new XmlSerializer(typeof(Message));
xml.Serialize(ms,xml);
That will just work with XmlSerializer, but if I try to convert it to protobuf-net I will get a "No default encoding for Object" message.
The best I came up with is to use generics and [ProtoInclude] as seen in this example. Since I can have different object types within the array this doesn't quite make it. I added a generic List for each potential type and a property with [ProtoIgnore] with type object[] to add them and get them. I have to use reflection when adding them (to know in which array to put each item) which is not desirable and I still can't preserve the ordering as I just extract all the items on each list one by one and put them into a new object[] array on the property get.
I wonder if there is a way to accomplish this?
I tried what Marc suggested below, but I couldn't get it to work. I think I may have misunderstood something.
Using the code you wrote. I thought I should use MessageParam Create to generate MessageParam objects to add to the list. So basically I added a constructor to Message like this:
public Message(object[] parms)
{
foreach (object o in parms)
{
parameters.Add(MessageParam.Create(o));
}
}
But, if i do that I will get "Unexpected type found during serialization; types must be included with ProtoIncludeAttribute; found MessageParam`1 passed as MessageParam" because I assume the serializer is expecting the non-generic version. Did I misunderstand your suggestion? If so, what is the right thing to do?
object is going to be problematic. I would try something more like:
[ProtoContract]
class Message
{
private readonly List<MessageParam> parameters = new List<MessageParam>();
[ProtoMember(1)]
public List<MessageParam> Parameters { get { return parameters; } }
}
[ProtoContract]
[ProtoInclude(3, typeof(MessageParam<int>))]
[ProtoInclude(4, typeof(MessageParam<float>))]
[ProtoInclude(5, typeof(MessageParam<DateTime>))]
//...known types...
abstract class MessageParam {
public abstract object UntypedValue { get; set; }
public static MessageParam<T> Create<T>(T value) {
return new MessageParam<T> { Value = value };
}
public static MessageParam CreateDynamic(object value)
{
Type type = value.GetType();
switch (Type.GetTypeCode(value.GetType()))
{
// special cases
case TypeCode.Int32: return Create((int)value);
case TypeCode.Single: return Create((float)value);
case TypeCode.DateTime: return Create((DateTime)value);
// fallback in case we forget to add one, or it isn't a TypeCode
default:
MessageParam param = (MessageParam)Activator.CreateInstance(
typeof(MessageParam<>).MakeGenericType(type));
param.UntypedValue = value;
return param;
}
}
}
[ProtoContract]
sealed class MessageParam<T> : MessageParam
{
[ProtoMember(1)]
public T Value { get; set; }
public override object UntypedValue
{
get { return Value; }
set { Value = (T)value; }
}
}
Note that the unreleased "v2" code offers much more ability to define the relationships at runtime rather than through attributes (which is quite limiting here).