Develop Reference

Crash after serializing JObject with DataContractSerializer and XmlDictionaryWriter

I have to serialize an Newtonsoft JObject with DataContractSerializer, it crashes with stack overflow.
How to make it work?
My code is.
var serializer = new DataContractSerializer(typeof(JObject));
MemoryStream stream1 = new MemoryStream();
var writer = XmlDictionaryWriter.CreateBinaryWriter(stream1);
var obj = new JObject();
serializer.WriteObject(writer, obj);
writer.Flush();
The following example is converting JObject to common type using ISerializationSurrogateProvider functionality. It will crash with stack overflow.
using System;
using System.IO;
using Newtonsoft.Json.Linq;
using System.Runtime.Serialization;
using System.Xml;
class Program
{
[DataContract(Name = "JTokenReference", Namespace = "urn:actors")]
[Serializable]
public sealed class JTokenReference
{
public JTokenReference()
{
}
[DataMember(Name = "JType", Order = 0, IsRequired = true)]
public JTokenType JType { get; set; }
[DataMember(Name = "Value", Order = 1, IsRequired = true)]
public string Value { get; set; }
public static JTokenReference From(JToken jt)
{
if (jt == null)
{
return null;
}
return new JTokenReference()
{
Value = jt.ToString(),
JType = jt.Type
};
}
public object To()
{
switch (JType)
{
case JTokenType.Object:
{
return JObject.Parse(Value);
}
case JTokenType.Array:
{
return JArray.Parse(Value);
}
default:
{
return JToken.Parse(Value);
}
}
}
}
internal class ActorDataContractSurrogate : ISerializationSurrogateProvider
{
public static readonly ISerializationSurrogateProvider Instance = new ActorDataContractSurrogate();
public Type GetSurrogateType(Type type)
{
if (typeof(JToken).IsAssignableFrom(type))
{
return typeof(JTokenReference);
}
return type;
}
public object GetObjectToSerialize(object obj, Type targetType)
{
if (obj == null)
{
return null;
}
else if (obj is JToken jt)
{
return JTokenReference.From(jt);
}
return obj;
}
public object GetDeserializedObject(object obj, Type targetType)
{
if (obj == null)
{
return null;
}
else if (obj is JTokenReference reference &&
typeof(JToken).IsAssignableFrom(targetType))
{
return reference.To();
}
return obj;
}
}
[DataContract(Name = "Test", Namespace = "urn:actors")]
[Serializable]
public class Test
{
[DataMember(Name = "obj", Order = 0, IsRequired = false)]
public JObject obj;
}
static void Main(string[] args)
{
var serializer = new DataContractSerializer(typeof(Test),
new DataContractSerializerSettings()
{
MaxItemsInObjectGraph = int.MaxValue,
KnownTypes = new Type[] { typeof(JTokenReference), typeof(JObject), typeof(JToken) },
});
serializer.SetSerializationSurrogateProvider(ActorDataContractSurrogate.Instance);
MemoryStream stream1 = new MemoryStream();
var writer = XmlDictionaryWriter.CreateBinaryWriter(stream1);
var obj = new JObject();
var test = new Test()
{
obj = obj,
};
serializer.WriteObject(writer, test);
writer.Flush();
Console.WriteLine(System.Text.Encoding.UTF8.GetString(stream1.GetBuffer(), 0, checked((int)stream1.Length)));
}
}
I am trying to define a new type JTokenReference to replace JObject/JToken when serializing, but it crashed before replace happens. It seems it failed to resolve the type.

TL;DR
Your approach is reasonable, and ought to work, but fails due to what seems to be a bug in the ISerializationSurrogateProvider functionality with recursive collection types. You're going to need to change your design to use surrogate properties whenever you need to serialize a JToken, e.g. as follows:
[IgnoreDataMember]
public JObject obj { get; set; }
[DataMember(Name = "obj", Order = 0, IsRequired = false)]
string objSurrogate { get { return obj?.ToString(Newtonsoft.Json.Formatting.None); } set { obj = (value == null ? null : JObject.Parse(value)); } }
Explanation
The crash you are experiencing is a stack overflow, and can be reproduced more simply as follows. When the data contract serializer writes a generic such as List<string>, it constructs a data contract name by combining the generic class and parameter names like so:
List<string>: ArrayOfstring
List<List<string>: ArrayOfArrayOfstring
List<List<List<string>>>: ArrayOfArrayOfArrayOfstring
And so on. As the generic nesting gets deeper the name gets longer. Well then, what happens if we define a self-recursive collection type like the following?
public class RecursiveList<T> : List<RecursiveList<T>>
{
}
Well, if we try to serialize one of these list with the data contract serializer, it crashes with a stack overflow exception trying to figure out the contract name. Demo fiddle #1 here -- you will need to uncomment the line //Test(new RecursiveList<string>()); to see the crash:
Stack overflow.
at System.ModuleHandle.ResolveType(System.Runtime.CompilerServices.QCallModule, Int32, IntPtr*, Int32, IntPtr*, Int32, System.Runtime.CompilerServices.ObjectHandleOnStack)
at System.ModuleHandle.ResolveTypeHandleInternal(System.Reflection.RuntimeModule, Int32, System.RuntimeTypeHandle[], System.RuntimeTypeHandle[])
at System.Reflection.RuntimeModule.ResolveType(Int32, System.Type[], System.Type[])
at System.Reflection.CustomAttribute.FilterCustomAttributeRecord(System.Reflection.MetadataToken, System.Reflection.MetadataImport ByRef, System.Reflection.RuntimeModule, System.Reflection.MetadataToken, System.RuntimeType, Boolean, ListBuilder`1<System.Object> ByRef, System.RuntimeType ByRef, System.IRuntimeMethodInfo ByRef, Boolean ByRef)
at System.Reflection.CustomAttribute.IsCustomAttributeDefined(System.Reflection.RuntimeModule, Int32, System.RuntimeType, Int32, Boolean)
at System.Reflection.CustomAttribute.IsDefined(System.RuntimeType, System.RuntimeType, Boolean)
at System.Runtime.Serialization.CollectionDataContract.IsCollectionOrTryCreate(System.Type, Boolean, System.Runtime.Serialization.DataContract ByRef, System.Type ByRef, Boolean)
at System.Runtime.Serialization.CollectionDataContract.IsCollectionHelper(System.Type, System.Type ByRef, Boolean)
at System.Runtime.Serialization.DataContract.GetNonDCTypeStableName(System.Type)
at System.Runtime.Serialization.DataContract.GetStableName(System.Type, Boolean ByRef)
at System.Runtime.Serialization.DataContract.GetCollectionStableName(System.Type, System.Type, System.Runtime.Serialization.CollectionDataContractAttribute ByRef)
at System.Runtime.Serialization.DataContract.GetNonDCTypeStableName(System.Type)
at System.Runtime.Serialization.DataContract.GetStableName(System.Type, Boolean ByRef)
at System.Runtime.Serialization.DataContract.GetCollectionStableName(System.Type, System.Type, System.Runtime.Serialization.CollectionDataContractAttribute ByRef)
at System.Runtime.Serialization.DataContract.GetNonDCTypeStableName(System.Type)
at System.Runtime.Serialization.DataContract.GetStableName(System.Type, Boolean ByRef)
Oops. Well, what if we create a serialization surrogate such as the following dummy surrogate for RecursiveList<string>
public class RecursiveListStringSurrogate
{
// A dummy surrogate that serializes nothing, for testing purposes.
}
public class RecursiveListStringSurrogateSelector : ISerializationSurrogateProvider
{
public object GetDeserializedObject(object obj, Type targetType)
{
if (obj is RecursiveListStringSurrogate)
return new RecursiveList<string>();
return obj;
}
public object GetObjectToSerialize(object obj, Type targetType)
{
if (obj is RecursiveList<string>)
return new RecursiveListStringSurrogate();
return obj;
}
public Type GetSurrogateType(Type type)
{
if (type == typeof(RecursiveList<string>))
return typeof(RecursiveListStringSurrogate);
return type;
}
}
Using that surrogate, an empty new RecursiveList<string>() can indeed be serialized successfully, as
<RecursiveListStringSurrogate xmlns:i="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://schemas.datacontract.org/2004/07/" />
Demo fiddle #2 here.
OK, now let's try using the surrogate when a RecursiveList<string> is embedded in a model such as:
public class Model
{
public RecursiveList<string> List { get; set; }
}
Well when I try to serialize an instance of this model with an empty list, the crash comes back. Demo fiddle #3 here - you will need to uncomment the line //Test(new Model { List = new RecursiveList<string>() }); to see the crash.
Oops again. It's not entirely clear why this fails. I can only speculate that, somewhere, Microsoft is keeping a dictionary mapping original data contract names to surrogate data contract names -- which causes a stack overflow simply generating a dictionary key.
Now what does this have to do with JObject and your Test class? Well it turns out that JObject is another example of a recursive collection type. It implements IDictionary<string, JToken?> and JToken in turn implements IEnumerable<JToken> thereby triggering the same stack overflow we saw with the simple model containing a RecursiveList<string>.
You might even want to report an issue to Microsoft about this (though I don't know whether they are fixing bugs with the data contract serializer any more.)
Workaround
To avoid this issue, you will need to modify your model(s) to use surrogate properties for JToken members as shown at the beginning of this answer:
[DataContract(Name = "Test", Namespace = "urn:actors")]
public class Test
{
[IgnoreDataMember]
public JObject obj { get; set; }
[DataMember(Name = "obj", Order = 0, IsRequired = false)]
string objSurrogate { get { return obj?.ToString(Newtonsoft.Json.Formatting.None); } set { obj = (value == null ? null : JObject.Parse(value)); } }
}
Which can be serialized successfully as follows:
var obj = new JObject();
var test = new Test()
{
obj = obj,
};
var serializer = new DataContractSerializer(test.GetType());
MemoryStream stream1 = new MemoryStream();
var writer = XmlDictionaryWriter.CreateBinaryWriter(stream1);
serializer.WriteObject(writer, test);
writer.Flush();
Console.WriteLine(System.Text.Encoding.UTF8.GetString(stream1.GetBuffer(), 0, checked((int)stream1.Length)));
Notes:
If you need to serialize a JToken as the root object you can either wrap it in some container object, or use the ActorDataContractSurrogate from your question. As we have seen, the serialization functionality does seem to work for recursive collection types when they are the root object.
Since you are serializing to binary, for efficiency I suggest formatting the JObject with Formatting.None.
The surrogate property can be private as long as it is marked with [DataMember].
Demo fiddle #4 here.

Parse exponential Value in Xml document

I have an issue when deserializing an object (which I cannot modify).
I receive an exponential value for certain xml element and for represent them in my class I used decimal value expect that when I deserialize the xml document it fails.
<fwdRate>-9.72316862724032E-05</fwdRate>
Is there any solution to represent this attribute other than create 2 attributes in my class to represent it (one string and the other a decimal value)?
Can I create a custom deserializtion class for decimal value?
private void ParseXML(string value)
{
XmlSerializer serializer = new XmlSerializer(typeof(SwapDataSynapseResult));
using (TextReader reader = new StringReader(value))
{
_result = serializer.Deserialize(reader) as SwapDataSynapseResult;
}
}
As Demand
using System;
using System.IO;
using System.Xml.Serialization;
[XmlRoot(ElementName = "result")]
public class Result
{
[XmlElement(ElementName = "fwdRate")]
public decimal FwdRate { get; set; }
}
public class Program
{
public static void Main()
{
string val = "<result><fwdRate>-9.72316862724032E-05</fwdRate></result>";
Result response = ParseXML(val);
}
static Result ParseXML(string value)
{
XmlSerializer serializer = new XmlSerializer(typeof(Result));
using (TextReader reader = new StringReader(value))
{
return serializer.Deserialize(reader) as Result;
}
}
}

In XML, decimal values are not allowed to use scientific (exponential) notation (See this link at the 'Restrictions' paragraph).
Either:
the value is indeed a floating point one: Put a float/double instead of a decimal in the code.
the XML is corrupted.
In the same way, in C#, by default, Decimal.Parse doesn't accept exponential representation.
You can override this behavior by implementing a new struct that wrap a decimal and implement IXmlSerializable and allow exponential representation when de-serialized:
public struct XmlDecimal : IXmlSerializable
{
public decimal Value { get; private set; }
public XmlDecimal(decimal value) => Value = value;
public XmlSchema GetSchema() => null;
public void ReadXml(XmlReader reader)
{
var s = reader.ReadElementContentAsString();
Value = decimal.TryParse(s, NumberStyles.Number | NumberStyles.AllowExponent,
NumberFormatInfo.InvariantInfo, out var value)
? value
: 0; // If parse fail the resulting value is 0. Maybe we can throw an exception here.
}
public void WriteXml(XmlWriter writer) => writer.WriteValue(Value);
public static implicit operator decimal(XmlDecimal v) => v.Value;
public override string ToString() => Value.ToString();
}
The flaw is that you have to use this struct instead of a decimal everywhere in your model.
And sadly you can't make this struct read-only, has explained here.

The proper way is to control how your properties are deserialized by implementing the IXmlSerializable interface:
IXmlSerializable
IXmlSerializable code project example
In the ReadXml method you should convert your number
var yourStringNumber = ...
this.fwdRate = Decimal.Parse(yourStringNumber, System.Globalization.NumberStyles.Float);
But this method will require you to parse of the whole xml manually that is a bit overhead sometimes.
A simple solution (that smells but might be useful) is just to add additional fwdRateDecimal field to your class and fulfill the value after the serialization.
private void ParseXML(string value)
{
XmlSerializer serializer = new XmlSerializer(typeof(SwapDataSynapseResult));
using (TextReader reader = new StringReader(value))
{
_result = serializer.Deserialize(reader) as SwapDataSynapseResult;
_result.fwdRateDecimal = Decimal.Parse(_result.fwdRate, System.Globalization.NumberStyles.Float)
}
}
Also conversion can be implemented in a type directly:
[XmlRoot(ElementName = "result")]
public class Result
{
[XmlElement(ElementName = "fwdRate")]
public string FwdRateStr { get; set; }
private string lastParsedValue = null;
private decimal fwdRate = 0;
[XmlIgnore]
public decimal FwdRate
{
get
{
if(FwdRateStr != lastParsedValue)
{
lastParsedValue = FwdRateStr
fwdRate = Decimal.Parse(FwdRateStr ,System.Globalization.NumberStyles.Float)
}
return fwdRate
}
}

This is the solution:
using System;
using System.IO;
using System.Xml.Serialization;
[XmlRoot(ElementName = "result")]
public class Result
{
[XmlElement(ElementName = "fwdRate")]
public double FwdRate { get; set; }
}
public class Program
{
public static void Main()
{
string val = "<result><fwdRate>-9.72316862724032E-05</fwdRate></result>";
Result response = ParseXML(val);
Console.WriteLine(response.FwdRate);
}
static Result ParseXML(string value)
{
XmlSerializer serializer = new XmlSerializer(typeof(Result));
using (TextReader reader = new StringReader(value))
{
return serializer.Deserialize(reader) as Result;
}
}
}
You can't modify the incoming xml but you can modify how you read its' data.
Read that number as a double instead of decimal and you will have the right decimal precision.
Output:
-9,72316862724032E-05

WCF: Data contract serializer with multiple modules

In one of my C# projects I use a WCF data contract serializer for serialization to XML. The framework however consists of multiple extension modules that may be loaded or not, dependent on some startup configuration (I use MEF in case it matters). In the future the list of modules may potentially grow and I fear that this situation may someday pose problems with module-specific data. As I understand I can implement a data contract resolver to bidirectionally help the serializer locate types, but what happens if the project contains data it cannot interpret because the associated module is not loaded?
I am looking for a solution that allows me to preserve existing serialized data in cases where not the full set of modules is loaded (or even available). I think of this as a way to tell the de-serializer "if you don't understand what you get, then don't try to serialize it, but please keep the data somewhere so that you can put it back when serializing the next time". I think my problem is related to round-tripping, but I wasn't very successful (yet) in finding a hint on how to deal with such a case where complex types may be added or removed between serialization actions.
Minimal example:
Suppose I start my application with the optional modules A, B and C and produce the following XML (AData, BData and CData are in a collection and may be all derived from a common base class):
<Project xmlns="http://schemas.datacontract.org/2004/07/TestApplication" xmlns:i="http://www.w3.org/2001/XMLSchema-instance">
<Data>
<ModuleData i:type="AData">
<A>A</A>
</ModuleData>
<ModuleData i:type="BData">
<B>B</B>
</ModuleData>
<ModuleData i:type="CData">
<C>C</C>
</ModuleData>
</Data>
</Project>
In case I skip module C (containing the definition of CData) and load the same project, then the serializer fails because it has no idea how to deal with CData. If I can somehow manage to convince the framework to keep the data and leave it untouched until someone opens the project again with module C, then I win. Of course I could implement dynamic data structures for storing extension data, e.g., key-value trees, but it would be neat to use the existing serialization framework also in extension modules. Any hint on how to achieve this is highly appreciated!
The example code to produce the above output is as follows:
using System;
using System.IO;
using System.Collections.Generic;
using System.Runtime.Serialization;
namespace TestApplication
{
// common base class
[DataContract]
public class ModuleData : IExtensibleDataObject
{
public virtual ExtensionDataObject ExtensionData { get; set; }
}
[DataContract]
public class AData : ModuleData
{
[DataMember]
public string A { get; set; }
}
[DataContract]
public class BData : ModuleData
{
[DataMember]
public string B { get; set; }
}
[DataContract]
public class CData : ModuleData
{
[DataMember]
public string C { get; set; }
}
[DataContract]
[KnownType(typeof(AData))]
[KnownType(typeof(BData))]
public class Project
{
[DataMember]
public List<ModuleData> Data { get; set; }
}
class Program
{
static void Main(string[] args)
{
// new project object
var project1 = new Project()
{
Data = new List<ModuleData>()
{
new AData() { A = "A" },
new BData() { B = "B" },
new CData() { C = "C" }
}
};
// serialization; make CData explicitly known to simulate presence of "module C"
var stream = new MemoryStream();
var serializer1 = new DataContractSerializer(typeof(Project), new[] { typeof(CData) });
serializer1.WriteObject(stream, project1);
stream.Position = 0;
var reader = new StreamReader(stream);
Console.WriteLine(reader.ReadToEnd());
// deserialization; skip "module C"
stream.Position = 0;
var serializer2 = new DataContractSerializer(typeof(Project));
var project2 = serializer2.ReadObject(stream) as Project;
}
}
}
I also uploaded a VS2015 solution here.

Your problem is that you have a polymorphic known type hierarchy, and you would like to use the round-tripping mechanism of DataContractSerializer to read and save "unknown" known types, specifically XML elements with an xsi:type type hint referring to a type not currently loaded into your app domain.
Unfortunately, this use case simply isn't implemented by the round-tripping mechanism. That mechanism is designed to cache unknown data members inside an ExtensionData object, provided that the data contract object itself can be successfully deserialized and implements IExtensibleDataObject. Unfortunately, in your situation the data contract object cannot be constructed precisely because the polymorphic subtype is unrecognized; instead the following exception gets thrown:
System.Runtime.Serialization.SerializationException occurred
Message="Error in line 4 position 6. Element
'http://www.Question45412824.com:ModuleData' contains data of the
'http://www.Question45412824.com:CData' data contract. The
deserializer has no knowledge of any type that maps to this contract.
Add the type corresponding to 'CData' to the list of known types - for
example, by using the KnownTypeAttribute attribute or by adding it to
the list of known types passed to DataContractSerializer."
Even if I try to create a custom generic collection marked with [CollectionDataContract] that implements IExtensibleDataObject to cache items with unrecognized contracts, the same exception gets thrown.
One solution is to take advantage of the fact that your problem is slightly less difficult than the round-tripping problem. You (the software architect) actually know all possible polymorphic subtypes. Your software does not, because it isn't always loading the assemblies that contain them. Thus what you can do is load lightweight dummy types instead of the real types when the real types aren't needed. As long as the dummy types implement IExtensibleDataObject and have the same data contract namespace and name and the real types, their data contracts will be interchangeable with the "real" data contracts in polymorphic collections.
Thus, if you define your types as follows, adding a Dummies.CData dummy placeholder:
public static class Namespaces
{
// The data contract namespace for your project.
public const string ProjectNamespace = "http://www.Question45412824.com";
}
// common base class
[DataContract(Namespace = Namespaces.ProjectNamespace)]
public class ModuleData : IExtensibleDataObject
{
public ExtensionDataObject ExtensionData { get; set; }
}
[DataContract(Namespace = Namespaces.ProjectNamespace)]
public class AData : ModuleData
{
[DataMember]
public string A { get; set; }
}
[DataContract(Namespace = Namespaces.ProjectNamespace)]
public class BData : ModuleData
{
[DataMember]
public string B { get; set; }
}
[DataContract(Namespace = Namespaces.ProjectNamespace)]
[KnownType(typeof(AData))]
[KnownType(typeof(BData))]
public class Project
{
[DataMember]
public List<ModuleData> Data { get; set; }
}
[DataContract(Namespace = Namespaces.ProjectNamespace)]
public class CData : ModuleData
{
[DataMember]
public string C { get; set; }
}
namespace Dummies
{
[DataContract(Namespace = Namespaces.ProjectNamespace)]
public class CData : ModuleData
{
}
}
You will be able to deserialize your Project object using either the "real" CData or the "dummy" version, as shown with the test below:
class Program
{
static void Main(string[] args)
{
new TestClass().Test();
}
}
class TestClass
{
public virtual void Test()
{
// new project object
var project1 = new Project()
{
Data = new List<ModuleData>()
{
new AData() { A = "A" },
new BData() { B = "B" },
new CData() { C = "C" }
}
};
// serialization; make CData explicitly known to simulate presence of "module C"
var extraTypes = new[] { typeof(CData) };
var extraTypesDummy = new[] { typeof(Dummies.CData) };
var xml = project1.SerializeXml(extraTypes);
ConsoleAndDebug.WriteLine(xml);
// Demonstrate that the XML can be deserialized with the dummy CData type.
TestDeserialize(project1, xml, extraTypesDummy);
// Demonstrate that the XML can be deserialized with the real CData type.
TestDeserialize(project1, xml, extraTypes);
try
{
// Demonstrate that the XML cannot be deserialized without either the dummy or real type.
TestDeserialize(project1, xml, new Type[0]);
Assert.IsTrue(false);
}
catch (AssertionFailedException ex)
{
Console.WriteLine("Caught unexpected exception: ");
Console.WriteLine(ex);
throw;
}
catch (Exception ex)
{
ConsoleAndDebug.WriteLine(string.Format("Caught expected exception: {0}", ex.Message));
}
}
public void TestDeserialize<TProject>(TProject project, string xml, Type[] extraTypes)
{
TestDeserialize<TProject>(xml, extraTypes);
}
public void TestDeserialize<TProject>(string xml, Type[] extraTypes)
{
var project2 = xml.DeserializeXml<TProject>(extraTypes);
var xml2 = project2.SerializeXml(extraTypes);
ConsoleAndDebug.WriteLine(xml2);
// Assert that the incoming and re-serialized XML are equivalent (no data was lost).
Assert.IsTrue(XNode.DeepEquals(XElement.Parse(xml), XElement.Parse(xml2)));
}
}
public static partial class DataContractSerializerHelper
{
public static string SerializeXml<T>(this T obj, Type [] extraTypes)
{
return obj.SerializeXml(new DataContractSerializer(obj == null ? typeof(T) : obj.GetType(), extraTypes));
}
public static string SerializeXml<T>(this T obj, DataContractSerializer serializer)
{
serializer = serializer ?? new DataContractSerializer(obj == null ? typeof(T) : obj.GetType());
using (var textWriter = new StringWriter())
{
var settings = new XmlWriterSettings { Indent = true };
using (var xmlWriter = XmlWriter.Create(textWriter, settings))
{
serializer.WriteObject(xmlWriter, obj);
}
return textWriter.ToString();
}
}
public static T DeserializeXml<T>(this string xml, Type[] extraTypes)
{
return xml.DeserializeXml<T>(new DataContractSerializer(typeof(T), extraTypes));
}
public static T DeserializeXml<T>(this string xml, DataContractSerializer serializer)
{
using (var textReader = new StringReader(xml ?? ""))
using (var xmlReader = XmlReader.Create(textReader))
{
return (T)(serializer ?? new DataContractSerializer(typeof(T))).ReadObject(xmlReader);
}
}
}
public static class ConsoleAndDebug
{
public static void WriteLine(object s)
{
Console.WriteLine(s);
Debug.WriteLine(s);
}
}
public class AssertionFailedException : System.Exception
{
public AssertionFailedException() : base() { }
public AssertionFailedException(string s) : base(s) { }
}
public static class Assert
{
public static void IsTrue(bool value)
{
if (value == false)
throw new AssertionFailedException("failed");
}
}
Another solution would be to replace your List<ModuleData> with a custom collection that implements IXmlSerializable and handles the polymorphic serialization entirely manually, caching the XML for unknown polymorphic subtypes in a list of unknown elements. I wouldn't recommend that however since even straightforward implementations of IXmlSerializable can be quite complex, as shown here and, e.g., here.

Following dbc's wonderful suggestion of using dummies to exploit the roundtripping mechanism to do the job, I made the solution more generic by generating the dummy types on the fly as needed.
The core of this solution is the following simple function that internally invokes the C# compiler:
private Type CreateDummyType(string typeName, string typeNamespace)
{
var className = $"DummyClass_{random_.Next()}";
var code = $"[System.Runtime.Serialization.DataContract(Name=\"{typeName}\", Namespace=\"{typeNamespace}\")] public class {className} : ModuleData {{}}";
using (var provider = new CSharpCodeProvider())
{
var parameters = new CompilerParameters();
parameters.ReferencedAssemblies.Add("System.Runtime.Serialization.dll");
parameters.ReferencedAssemblies.Add(GetType().Assembly.Location); // this assembly (for ModuleData)
var results = provider.CompileAssemblyFromSource(parameters, code);
return results.CompiledAssembly.GetType(className);
}
}
I combined this with a DataContractResolver that takes care of any unknown types and generates dummies as needed to preserve their data during subsequent (de)serializations.
For completeness I put the recent iteration of the sample code here:
using System;
using System.IO;
using System.Collections.Generic;
using System.Runtime.Serialization;
using System.Diagnostics;
using System.Xml;
using System.Xml.Linq;
using Microsoft.CSharp;
using System.CodeDom.Compiler;
public static class Namespaces
{
public const string BaseNamespace = "http://www.Question45412824.com";
public const string ProjectNamespace = BaseNamespace + "/Project";
public const string ExtensionNamespace = BaseNamespace + "/Extension";
}
// common base class
[DataContract(Namespace = Namespaces.ProjectNamespace)]
public class ModuleData : IExtensibleDataObject
{
public ExtensionDataObject ExtensionData { get; set; }
}
[DataContract(Namespace = Namespaces.ProjectNamespace)]
public class AData : ModuleData
{
[DataMember]
public string A { get; set; }
}
[DataContract(Namespace = Namespaces.ProjectNamespace)]
public class BData : ModuleData
{
[DataMember]
public string B { get; set; }
}
[DataContract(Namespace = Namespaces.ProjectNamespace)]
[KnownType(typeof(AData))]
[KnownType(typeof(BData))]
public class Project
{
[DataMember]
public List<ModuleData> Data { get; set; }
}
[DataContract(Namespace = Namespaces.ProjectNamespace)]
internal class CSubData : ModuleData
{
[DataMember]
public string Name { get; set; }
}
[DataContract(Namespace = Namespaces.ExtensionNamespace)]
public class CData : ModuleData
{
[DataMember]
public ModuleData C { get; set; }
}
class Program
{
static void Main(string[] args)
{
new TestClass().Test();
}
}
class TestClass
{
public virtual void Test()
{
// new project object
var project1 = new Project()
{
Data = new List<ModuleData>()
{
new AData() { A = "A" },
new BData() { B = "B" },
new CData() { C = new CSubData() { Name = "C" } }
}
};
// serialization; make CData explicitly known to simulate presence of "module C"
var extraTypes = new[] { typeof(CData), typeof(CSubData) };
ConsoleAndDebug.WriteLine("\n== Serialization with all types known ==");
var xml = project1.SerializeXml(extraTypes);
ConsoleAndDebug.WriteLine(xml);
ConsoleAndDebug.WriteLine("\n== Deserialization and subsequent serialization WITH generic resolver and unknown types ==");
TestDeserialize(project1, xml, new GenericDataContractResolver());
ConsoleAndDebug.WriteLine("\n== Deserialization and subsequent serialization WITHOUT generic resolver and unknown types ==");
try
{
// Demonstrate that the XML cannot be deserialized without the generic resolver.
TestDeserialize(project1, xml, new Type[0]);
Assert.IsTrue(false);
}
catch (AssertionFailedException ex)
{
Console.WriteLine("Caught unexpected exception: ");
Console.WriteLine(ex);
throw;
}
catch (Exception ex)
{
ConsoleAndDebug.WriteLine(string.Format("Caught expected exception: {0}", ex.Message));
}
}
public void TestDeserialize<TProject>(TProject project, string xml, Type[] extraTypes)
{
TestDeserialize<TProject>(xml, extraTypes);
}
public void TestDeserialize<TProject>(string xml, Type[] extraTypes)
{
var project2 = xml.DeserializeXml<TProject>(extraTypes);
var xml2 = project2.SerializeXml(extraTypes);
ConsoleAndDebug.WriteLine(xml2);
// Assert that the incoming and re-serialized XML are equivalent (no data was lost).
Assert.IsTrue(XNode.DeepEquals(XElement.Parse(xml), XElement.Parse(xml2)));
}
public void TestDeserialize<TProject>(TProject project, string xml, DataContractResolver resolver)
{
TestDeserialize<TProject>(xml, resolver);
}
public void TestDeserialize<TProject>(string xml, DataContractResolver resolver)
{
var project2 = xml.DeserializeXml<TProject>(resolver);
var xml2 = project2.SerializeXml(resolver);
ConsoleAndDebug.WriteLine(xml2);
// Assert that the incoming and re-serialized XML are equivalent (no data was lost).
Assert.IsTrue(XNode.DeepEquals(XElement.Parse(xml), XElement.Parse(xml2)));
}
}
public static partial class DataContractSerializerHelper
{
public static string SerializeXml<T>(this T obj, Type[] extraTypes)
{
return obj.SerializeXml(new DataContractSerializer(obj == null ? typeof(T) : obj.GetType(), extraTypes));
}
public static string SerializeXml<T>(this T obj, DataContractResolver resolver)
{
return obj.SerializeXml(new DataContractSerializer(obj == null ? typeof(T) : obj.GetType(), null, int.MaxValue, false, false, null, resolver));
}
public static string SerializeXml<T>(this T obj, DataContractSerializer serializer)
{
serializer = serializer ?? new DataContractSerializer(obj == null ? typeof(T) : obj.GetType());
using (var textWriter = new StringWriter())
{
var settings = new XmlWriterSettings { Indent = true };
using (var xmlWriter = XmlWriter.Create(textWriter, settings))
{
serializer.WriteObject(xmlWriter, obj);
}
return textWriter.ToString();
}
}
public static T DeserializeXml<T>(this string xml, DataContractResolver resolver)
{
return xml.DeserializeXml<T>(new DataContractSerializer(typeof(T), null, int.MaxValue, false, false, null, resolver));
}
public static T DeserializeXml<T>(this string xml, Type[] extraTypes)
{
return xml.DeserializeXml<T>(new DataContractSerializer(typeof(T), extraTypes));
}
public static T DeserializeXml<T>(this string xml, DataContractSerializer serializer)
{
using (var textReader = new StringReader(xml ?? ""))
using (var xmlReader = XmlReader.Create(textReader))
{
return (T)(serializer ?? new DataContractSerializer(typeof(T))).ReadObject(xmlReader);
}
}
}
public static class ConsoleAndDebug
{
public static void WriteLine(object s)
{
Console.WriteLine(s);
Debug.WriteLine(s);
}
}
public class AssertionFailedException : System.Exception
{
public AssertionFailedException() : base() { }
public AssertionFailedException(string s) : base(s) { }
}
public static class Assert
{
public static void IsTrue(bool value)
{
if (value == false)
throw new AssertionFailedException("failed");
}
}
class GenericDataContractResolver : DataContractResolver
{
private static readonly Random random_ = new Random();
private static readonly Dictionary<Tuple<string, string>, Type> toType_ = new Dictionary<Tuple<string, string>, Type>();
private static readonly Dictionary<Type, Tuple<string, string>> fromType_ = new Dictionary<Type, Tuple<string, string>>();
private Type CreateDummyType(string typeName, string typeNamespace)
{
var className = $"DummyClass_{random_.Next()}";
var code = $"[System.Runtime.Serialization.DataContract(Name=\"{typeName}\", Namespace=\"{typeNamespace}\")] public class {className} : ModuleData {{}}";
using (var provider = new CSharpCodeProvider())
{
var parameters = new CompilerParameters();
parameters.ReferencedAssemblies.Add("System.Runtime.Serialization.dll");
parameters.ReferencedAssemblies.Add(GetType().Assembly.Location); // this assembly (for ModuleData)
var results = provider.CompileAssemblyFromSource(parameters, code);
return results.CompiledAssembly.GetType(className);
}
}
// Used at deserialization; allows users to map xsi:type name to any Type
public override Type ResolveName(string typeName, string typeNamespace, Type declaredType, DataContractResolver knownTypeResolver)
{
var type = knownTypeResolver.ResolveName(typeName, typeNamespace, declaredType, null);
// resolve all unknown extension datasets; all other should be explicitly known.
if (type == null && declaredType == typeof(ModuleData) && typeNamespace == Namespaces.ExtensionNamespace)
{
// if we already have this type cached, then return the cached one
var typeNameAndNamespace = new Tuple<string, string>(typeName, typeNamespace);
if (toType_.TryGetValue(typeNameAndNamespace, out type))
return type;
// else compile the dummy type and remember it in the cache
type = CreateDummyType(typeName, typeNamespace);
toType_.Add(typeNameAndNamespace, type);
fromType_.Add(type, typeNameAndNamespace);
}
return type;
}
// Used at serialization; maps any Type to a new xsi:type representation
public override bool TryResolveType(Type type, Type declaredType, DataContractResolver knownTypeResolver, out XmlDictionaryString typeName, out XmlDictionaryString typeNamespace)
{
if (knownTypeResolver.TryResolveType(type, declaredType, null, out typeName, out typeNamespace))
return true; // known type
// is the type one of our cached dummies?
var typeNameAndNamespace = default(Tuple<string, string>);
if (declaredType == typeof(ModuleData) && fromType_.TryGetValue(type, out typeNameAndNamespace))
{
typeName = new XmlDictionaryString(XmlDictionary.Empty, typeNameAndNamespace.Item1, 0);
typeNamespace = new XmlDictionaryString(XmlDictionary.Empty, typeNameAndNamespace.Item2, 0);
return true; // dummy type
}
return false; // unknown type
}
}

Serialize a class that contains a list?

I may be doing this all wrong here, but I created a class that contains a list and I need to serialize that list. (Is there a better way or other suggestions to have multiple interfaces without a list?)
I've been doing okay with serializing custom classes, but this one for some reason isn't working out.
[XmlRoot("interfaces", Namespace = "")]
[XmlInclude(typeof(Interface))]
public class Interfaces
{
[XmlArray("interfaces")]
[XmlArrayItem("interface")]
List<Interface> _IflList = new List<Interface>();
public List<Interface> IflList
{
get { return _IflList; }
set { _IflList = value; }
}
public void Add(Interface objInterface)
{
_IflList.Add(objInterface);
}
}
[XmlType("interface")]
public class Interface
{
string _name;
public string name
{
get { return _name; }
set { _name = value; }
}
public Interface(string name)
{
this._name = name;
}
}
I get the error There was an error reflecting type 'JunOSConfig.Interfaces' when trying to serialize with:
public string SerializeObject(object objToSerialize, bool StripXmlVer, bool FormatOutput)
{
string strReturn = "";
XmlSerializerNamespaces xns = new XmlSerializerNamespaces();
xns.Add("", "");
Type objType = typeof(JunOSConfig.Interfaces);
XmlSerializer xs = new XmlSerializer(objToSerialize.GetType());
XmlWriterSettings xws = new XmlWriterSettings();
xws.OmitXmlDeclaration = StripXmlVer;
StringWriter sw = new StringWriter();
XmlWriter xw = XmlWriter.Create(sw, xws);
xs.Serialize(xw, objToSerialize, xns);
strReturn = sw.ToString();
if (FormatOutput)
{
return Convert.ToString(XElement.Parse(strReturn));
}
else
{
return strReturn;
}
}

In addition to your constructor with one parameter
public Interface(string name)
{
this._name = name;
}
you will need another, parameterless constructor, like
public Interface()
{
}
If you do not declare a constructor yourself, a constructor without
parameters will be generated.
Then it should work.

Suppress xsi:nil but still show Empty Element when Serializing in .Net

I have a c# class that has 20+ string properties. I set about a fourth of those to an actual value. I would like to serialize the class and get an output of
<EmptyAttribute></EmptyAttribute>
for a property
public string EmptyAttribute {get;set;}
I do not want the output to be
<EmptyAttribute xsi:nil="true"></EmptyAttribute>
I am using the following class
public class XmlTextWriterFull : XmlTextWriter
{
public XmlTextWriterFull(string filename) : base(filename,Encoding.UTF8) { }
public override void WriteEndElement()
{
base.WriteFullEndElement();
base.WriteRaw(Environment.NewLine);
}
}
so that I can get the full tags. I just don't know how to get rid of the xsi:nil.

The way to have the XmlSerializer serialize a property without adding the xsi:nil="true" attribute is shown below:
[XmlRoot("MyClassWithNullableProp", Namespace="urn:myNamespace", IsNullable = false)]
public class MyClassWithNullableProp
{
public MyClassWithNullableProp( )
{
this._namespaces = new XmlSerializerNamespaces(new XmlQualifiedName[] {
new XmlQualifiedName(string.Empty, "urn:myNamespace") // Default Namespace
});
}
[XmlElement("Property1", Namespace="urn:myNamespace", IsNullable = false)]
public string Property1
{
get
{
// To make sure that no element is generated, even when the value of the
// property is an empty string, return null.
return string.IsNullOrEmpty(this._property1) ? null : this._property1;
}
set { this._property1 = value; }
}
private string _property1;
// To do the same for value types, you need a "helper property, as demonstrated below.
// First, the regular property.
[XmlIgnore] // The serializer won't serialize this property properly.
public int? MyNullableInt
{
get { return this._myNullableInt; }
set { this._myNullableInt = value; }
}
private int? _myNullableInt;
// And now the helper property that the serializer will use to serialize it.
[XmlElement("MyNullableInt", Namespace="urn:myNamespace", IsNullable = false)]
public string XmlMyNullableInt
{
get
{
return this._myNullableInt.HasValue?
this._myNullableInt.Value.ToString() : null;
}
set { this._myNullableInt = int.Parse(value); } // You should do more error checking...
}
// Now, a string property where you want an empty element to be displayed, but no
// xsi:nil.
[XmlElement("MyEmptyString", Namespace="urn:myNamespace", IsNullable = false)]
public string MyEmptyString
{
get
{
return string.IsNullOrEmpty(this._myEmptyString)?
string.Empty : this._myEmptyString;
}
set { this._myEmptyString = value; }
}
private string _myEmptyString;
// Now, a value type property for which you want an empty tag, and not, say, 0, or
// whatever default value the framework gives the type.
[XmlIgnore]
public float? MyEmptyNullableFloat
{
get { return this._myEmptyNullableFloat; }
set { this._myEmptyNullableFloat = value; }
}
private float? _myEmptyNullableFloat;
// The helper property for serialization.
public string XmlMyEmptyNullableFloat
{
get
{
return this._myEmptyNullableFloat.HasValue ?
this._myEmptyNullableFloat.Value.ToString() : string.Empty;
}
set
{
if (!string.IsNullOrEmpty(value))
this._myEmptyNullableFloat = float.Parse(value);
}
}
[XmlNamespaceDeclarations]
public XmlSerializerNamespaces Namespaces
{
get { return this._namespaces; }
}
private XmlSerializerNamespaces _namespaces;
}
Now, instantiate this class and serialize it.
// I just wanted to show explicitly setting all the properties to null...
MyClassWithNullableProp myClass = new MyClassWithNullableProp( ) {
Property1 = null,
MyNullableInt = null,
MyEmptyString = null,
MyEmptyNullableFloat = null
};
// Serialize it.
// You'll need to setup some backing store for the text writer below...
// a file, memory stream, something...
XmlTextWriter writer = XmlTextWriter(...) // Instantiate a text writer.
XmlSerializer xs = new XmlSerializer(typeof(MyClassWithNullableProp),
new XmlRootAttribute("MyClassWithNullableProp") {
Namespace="urn:myNamespace",
IsNullable = false
}
);
xs.Serialize(writer, myClass, myClass.Namespaces);
After retrieving the contents of the XmlTextWriter, you should have the following output:
<MyClassWithNullableProp>
<MyEmptyString />
<MyEmptyNullableFloat />
</MyClassWithNullableProp>
I hope this clearly demonstrates how the built-in .NET Framework XmlSerializer can be used to serialize properties to an empty element, even when the property value is null (or some other value you don't want to serialize). In addition, I have shown how you can make sure that null properties are not serialized at all. One thing to note, if you apply an XmlElementAttribute and set the IsNullable property of that attribute to true, then that property will serialize with the xsi:nil attribute when the property is null (unless overriden somewhere else).

I was actually able to figure this out. I know its a bit of a hack in some ways but this is how I got it to work
System.Xml.Serialization.XmlSerializer x = new System.Xml.Serialization.XmlSerializer(header.GetType());
XmlTextWriterFull writer = new XmlTextWriterFull(FilePath);
x.Serialize(writer, header);
writer.Flush();
writer.BaseStream.Dispose();
string xml = File.ReadAllText(FilePath);
xml = xml.Replace(" xsi:nil=\"true\"", "");
File.WriteAllText(FilePath, xml);
Hope this helps someone else out