Develop Reference

Inconsistencies after deserialization using protobuf-net

I am facing a problem of inconsistency after deserialization using protobuf-net.
The class I would like to serialize/deserialize looks like:
[ProtoContract]
public class TSS
{
[ProtoMember(1, AsReference = true)]
public EventManager eventManager { get; private set; }
[ProtoMember(2)]
public DateTime referenceDateTime { get; private set; }
[ProtoMember(3)]
public Mode mode;
}
And inside EventManager class, it looks like:
[ProtoContract]
public class EventManager
{
[ProtoMember(1)]
public InputQueue inputQueue = new InputQueue();
[ProtoMember(2)]
public InputQueue InputQueue
{
get { return this.inputQueue; }
set { this.inputQueue = value; }
}
[ProtoMember(7, AsReference = true)]
public TSS tss;
}
The tss in class EventManager is a reference of TSS object, and eventManager in class TSS is a reference of EventManager object. This is the reason I put AsReference = true there (is this the right way?)
I do serialization like:
public void StateSaving(int time, TSS tss)
{
Stream memoryStream = new MemoryStream();
Serializer.Serialize(memoryStream, tss);
states.Add(time, memoryStream);
}
and do deserialization like:
public void Deserialize(int time, ref TSS tss)
{
Stream memoryStream = states[time];
memoryStream.Position = 0;
tss = Serializer.Deserialize<TSS>(memoryStream);
}
The problem is that whenever I do deserialization, the data structures like inputQueue in the EventManager is populated with NULL values instead of actual values at that point. I am a newbie to protobuf-net, so please point out any mistakes (I believe there are a lot).
Thanks in advance!!

(from comments)
I have located the problem, basically there's a list that needs to be deserialized. And this list is a list of events, in which the constructors of the events have parameters, and when it tries to deserialize, the program will run the parameterless constructors (I manually added these constructors in order to eliminate the exceptions) instead of the right ones (with parameters). I know this is how serialization/deserialization work, but is there a way I can serialize and deserialize this list correctly?
Ah, indeed. There are various approaches when it comes to object construction:
use the parameterless constructor
look for a constructor which matches all the defined members
skip the constructor completely
use a custom object factory
use a surrogate object and custom conversion
Things like XmlSerializer use the first; things like DataContractSerializer and BinaryFormatter use the 3rd; the good news is that protobuf-net supports all 5. I suggest that in your case the best option is to use the 3rd option for this type, which you can do by:
[ProtoContract(SkipConstructor=true)]

XmlSerializer - Ignoring inherited unserializable member

Assume you have two classes, one inherits the other and the child needs to be serialized / deserialized with XmlSerializer. However, the parent contains a member that is not serializeable, say a dictionary.
public class Parent {
public Dictionary<string, int> dictionary;
}
The parent class is a library used for many other scripts. It cannot be modified. Now the child class contains only serializable members:
public class Child : Parent {
[XmlElement]
public int foo;
}
When trying to call the serializer, I receive an error saying that the dictionary is not serializable. When trying to serialize in JSON, I managed to get away by the price of a warning. I just created another member with the same name and type, and used ScriptIgnore:
public class Child : Parent {
public int foo;
[ScriptIgnore]
public Dictionary<string, int> dictionary;
}
I tried the same trick again here (by using XmlIgnore) but that didn't work on very well, the error was the same. The only way I managed to go through this is create separate classes that will only serve xml de/serializing and then copy the values back into the appropriate place.
Does anyone know a better way around this? Can I make XmlSerializer forget about the parent dictionary in any way?

The very first thing I would say, and always say: if serializing an existing model gets tricky - even remotely awkward, then stop doing that. Take 2 minutes to create a separate DTO model, i.e. a model created solely for the purposes of serialization (and indeed, perhaps even tailored to a specific serializer). Now you put the exact right types, right members, right attributes, and right layout. All you need to do is add some conversion methods - static conversion operators work great here. So what I would say is: create a ParentDto and ChildDto (your names may vary); it'll take 3 minutes, and it'll work great.
Now, back to the question...
XmlSerializer looks at the declaring class for input; for both attributes and conditional serialization, no: we can't add those into the type model at this point. But there is another option - you can use XmlAttributeOverrides to pretend that there was an [XmlIgnore] on the dictionary member. However, some important caveats:
the XmlAttributeOverrides API is a bit of a faff to use (see MSDN for an example)
it is critical that you only do this once, and then store and re-use the XmlSerializer that you create this way; basically, if you don't do this, it will create a new dynamic assembly every time you new a serializer, and assemblies never unload, so you will haemorrhage memory; note that the simple usage (new XmlSerializer(someType) etc) has an inbuilt cache for this; but the XmlAttributeOverrides usage does not
But again, all this messing with XmlAttributeOverrides is a lot more work than just creating a basic DTO
Example of using XmlAttributeOverrides:
using System;
using System.Collections.Generic;
using System.Xml.Serialization;
public class Parent {
public Dictionary<string, int> WantToIgnoreThis { get; set; }
}
public class Child : Parent {
public int Foo { get; set; }
}
static class Program
{
static readonly XmlSerializer customSerializer;
static Program()
{
var xao = new XmlAttributeOverrides();
xao.Add(typeof(Parent), "WantToIgnoreThis", new XmlAttributes {
XmlIgnore = true
});
customSerializer = new XmlSerializer(typeof(Child), xao);
}
static void Main()
{
//var ser = new XmlSerializer(typeof(Child));
// ^^ this would fail
customSerializer.Serialize(Console.Out, new Child {
Foo = 123
});
}
}
Note in particular how the static field is used to cache the serializer.

You could implement IXmlSerializable yourself and handle the specifics in ReadXml(XmlReader reader) and WriteXml(XmlWriter writer). The XmlSerializer will call those methods if your class implements them instead of generating its own serializer.
public class Child : Parent, IXmlSerializable
{
public int Foo { get; set; }
public Dictionary<string, int> Dictionary { get; set; }
public void WriteXml(XmlWriter writer)
{
writer.WriteStartElement("Foo");
writer.WriteValue(this.Foo);
writer.WriteEndElement();
}
void ReadXml(XmlReader reader)
{
var wasEmpty = reader.IsEmptyElement;
reader.Read();
if (wasEmpty)
{
return;
}
reader.ReadStartElement("Foo");
this.Foo = reader.ReadContentAsInt();
reader.ReadEndElement();
}
}

c# - DataContract Serialization - Base Classes, Inheritance and Overrides

Good day everyone,
I'm an independent game developer who has, in the past, primarily worked with XNA and, at the other extreme, commercial toolsets. The reach of XNA is pretty limited, however, and I'm building a cross-platform abstraction layer to target multiple platforms.
To cut a long story short, I've needed xml serialization that's accessible more broadly than [Serializable], and I've been pointed to data contracts. I've been doing a lot of research, but can't find any good information about some of the basics of the system, pertaining to inheritance and overrides.
The crux of my question is...
[DataContract]
public class Node
{
private string name;
public string Name { get { return name; } set { name = value; } }
public virtual float Rotation { get { return 0f; } set { } }
}
[DataContract]
public class FancyNode : Node
{
private float rotation;
public override float Rotation { get { return rotation; } set { rotation = value; } }
}
If I serialize a 'FancyNode', will 'Rotation' be properly serialized, and will 'Name' be serialised?
Follow-up Question:
I meant to ask earlier, but couldn't recall at the time. How does the serializer handler overriden [IgnoreDataMember] properties? For example...
[DataContract]
public class Simple
{
[IgnoreDataMember]
public virtual string Value { get { return ""; } set { } }
}
[DataContract]
public class Complex : Simple
{
private string value;
public override string Value { get { return value; } set { this.value = value; } }
}
Would 'Value' in 'Complex' be serialized? Some answers are suggesting that if no [DataMember] tags are used, all properties will be serialized. If so, does the [IgnoreDataMember] attribute of the base class have any bearing?

Some thoughts presented in a few answers are unclear. Please let me clear them up. First, the serializer that uses DataContract can be used to serialize types that are not decorated with DataContract. You only need to inform the serailizer about them.
For example, the DataContractJsonSerializer class will serialize a type into a JSON object (usually a string.) All you need to tell it is the type of object you're serializing, and then any other types it may reference:
var serializer = new System.Runtime.Serialization.Json.DataContractJsonSerializer(typeof(MyClass), new Type[] { Type1, Type2, Type3 });
The serializer will automatically serialize each public field and property of the type. For more information, see http://msdn.microsoft.com/en-us/ms733127
DataContractAttribute Marked Types
Once you mark a type with the DataContractAttribute, you turn that type into an automatically known type (that you don't need to provide as a child type) and you turn it into the properties and fields into opt-in mode.
Therefore, you must decorate each field or property you want to be serialized with the DataMemberAttribute. This means that the IgnoreDataMemberAttribute is useless. It is, the serializer will not look for it, since anything not marked with the DataMemberAttribute is automatically ignored.
Unmarked DataContractAttribute Types
When serializing a type that does not have the DataContractAttribute applied, as previously stated, each public property or field will be serialized. Therefore, the IgnoreDataMemberAttribute is used here to prevent a property or field from being serialized.

As far as I know, DataContract is an 'opt-in' serialization method, i.e. stuff isn't serialized unless you decorate it (tell the serializer you want to serialize it)
So for the above example, you would need to add [DataMember] to the properties you wanted to serialize
With the standard Serializable attribute, the serializer looks at all fields and only ignores those which you mark as NonSerialized
Some examples here:
http://jamescbender.azurewebsites.net/?p=651
Check the notes section on this for info on what gets serialized and a rough outline of what happens:
http://msdn.microsoft.com/en-us/library/ms733127.aspx
Edit: Also I can't see any reason why any of the fields once marked as [DataMember] wouldn't be serialized properly. The DataContract method of serialization can also deal with circular references - something that other serialization sometimes has trouble with:
http://msdn.microsoft.com/en-us/library/hh241056.aspx

You should include the [DataMember] attribute on properties that should be exposed by the contract.
[DataContract]
public class FancyNode : Node
{
private float rotation;
[DataMember]
public override float Rotation { get { return rotation; } set { rotation = value; } }
}
Note that Windows Communication Foundation (WCF) uses the Data Contract Serializer to serialize and deserialize data (convert it to and from XML). So actually you are still using Xml Serialization.

No it wont serialize because of Opt-In approach, you have to explicitly apply DataMember to your base class. Data Serialization does not work automatically in case you Inherit in Child Class

How is it that the abstract class XmlWriter can be instantiated using XmlWriter.Create(...?

Just looking to clarify my understanding of the workings of the XmlWriter and abstract classes in general.
My thinking is (was) that an abstract class can not be instantiated, although it can contain base methods that can be used by an inheriting class.
So, while investigating XmlWriter, I find that to instantiate the XmlWriter, you call XmlWriter.Create(.... , which returns an instance of... XmlWriter, which can then be used:
FileStream fs = new FileStream("XML.xml", FileMode.Create);
XmlWriter w = XmlWriter.Create(fs);
XmlSerializer xmlSlr = new XmlSerializer(typeof(TestClass));
xmlSlr.Serialize(fs, tsIn);
This clearly works, as tested. Can anyone help me understand what is going on here. As far as I can see there is or should be no 'instance' to work with here??

You can't create an instance using new, but Create as it is used here is what is called a static factory method; it is NOT a constructor. You will find that in fact, the object returned by Create does not belong to abstract class XmlWriter, but some other concrete subclass.
See also
Wikipedia/Factory method pattern

There's nothing abstract about the object you get back. There are 13 classes inside the .NET framework that implement XmlWriter. They are all internal, you could only see their names if you'd peek at the source code with Reflector.
Not having to know the names of those 13 classes yourself is very valuable both to you and Microsoft. To you because you don't have to learn the details of picking the right one. To Microsoft because they can completely change the implementation, even the name, of those classes and your code would never notice.
This is called the Factory Pattern.

This is what is known as the factory pattern.
In this case the abstract class also acts as the factory responsible for creating concrete instances of classes that extend itself.
This way the responsibility for creating the correct class is handed over to the factory, quite often the factory will make decisions on what class to create depending on some parameters you pass in or other things such as config/environment etc..

Thanks. Once I did a test on the type of the returned object, I got a type which derives from XmlWriter. What was confusing me, was that I didn't expect the abstract base class to be able to reference subclasses of it's self.
.NET must determine the type of concrete XmlWriter to return based on the input arguments.
The XmlWriter seems (to me) to work a little less intuitively than other implementations I have seen since reading all the comments here. Examples such as in the link below use the Create method from concrete classes.
http://www.codeproject.com/KB/architecture/CSharpClassFactory.aspx
I knocked out some code here to prove to my self that implementing the functionality via the Create method on the abstract class was possible. The create method of the abstract class can indeed reference derived types as shown here:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
namespace TestFactoryPattern
{
public abstract class Input
{
public int Val;
}
public class InputObjA : Input
{
public InputObjA()
{
Val = 1;
}
}
public class InputObjB : Input
{
public InputObjB()
{
Val = 2;
}
}
public abstract class MyXmlWriter
{
public static int InputVal;
public static MyXmlWriter Create(Input input)
{
InputVal = input.Val;
if (input is InputObjA)
{
return new MyObjAXmlWriter();
}
else if (input is InputObjB)
{
return new MyObjBXmlWriter();
}
else
{
return new MyObjAXmlWriter();
}
}
public abstract void WriteMyXml();
}
public class MyObjAXmlWriter : MyXmlWriter
{
public override void WriteMyXml()
{
Console.WriteLine("Input A Written: " + InputVal);
}
}
public class MyObjBXmlWriter : MyXmlWriter
{
public override void WriteMyXml()
{
Console.WriteLine("Input B Written: " + InputVal);
}
}
public class Program
{
public static void Main()
{
InputObjA a = new InputObjA();
MyXmlWriter myXml1 = MyXmlWriter.Create(a);
myXml1.WriteMyXml();
InputObjB b = new InputObjB();
MyXmlWriter myXml2 = MyXmlWriter.Create(b);
myXml2.WriteMyXml();
}
}
}
Thanks all for the answers and input.

Calling GetObjectData on serialization

I am confused about the serialization sample from MSDN.
My confusion is in method GetObjectData (which is called during serialization), will the method,
serialize both the additional data (in method GetObjectData from AddValue) and the fields/properties of the class;
or just write the data in method GetObjectData without writing fields/properties of the class?
I have debugged seems (2) is correct -- no fields/properties data are serialized if GetObjectData method is used? Is that correct? (I am not an expert and just want to confirm here, but 100% confident about myself.)

Im not sure what you want to achieve but isn't easier to let C# do the work for you:
using System;
using System.Collections.Generic;
using System.Xml.Serialization;
namespace Test
{
[Serializable]
public class TestObject
{
private String name;
private String note;
#region Getters/setters
public String Name
{
get { return name; }
set { name = value; }
}
public String Note
{
get { return note; }
set { note = value; }
}
#endregion
}
}
Now you can use the XmlSerializer or BinaryFormatter to (de)serialize the object

If you implement ISerializable, you are reasponsible for all data (i.e. scenario "2" in your question); nothing extra is serialized automatically. What is your requirement? Things like DataContractSerializer can be property-based, allowing you to decorate both the regular fields and your custom property (that has some logic) and have them serialized properly. If you need binary (for space etc), then perhaps consider things like protobuf-net, which mixes the two while being space efficient.
So: what are your requirements?
Data Contract example:
[DataContract]
public class Foo {
[DataMember]
public int Bar {get;set;} // simple data
[DataMember]
private string DoSomeThinking {
get {.... serialize the complex data ....}
set {.... deserialize the complex data ....}
}
}

If you implement ISerializable you must add all data (at least the data needed to deserialize) including all fields to the SerializationInfo using AddValue.