Develop Reference

Azure Service-Fabric and DocumentDB message serialization issue

So, - in my DocumentDB I may have the following document:
{
"id": 1,
"type": "A",
"content": {
"x": 1,
"y": 2
}
}
That may be backed by this model:
public class acontent
{
public int x { get; set; }
public int y { get; set; }
}
public class document
{
public int id { get; set; }
public string type { get; set; }
public object content { get; set; }
}
public class documenta : document
{
public new acontent content { get; set; }
}
The idea here is that document is a complex object where content may vary depending on type.
Now, - in my ServiceFabric application I have a stateless microservice that reads from DocumentDB and should return a document type object when called from the ServiceProxy.
The problem in this is that the DocumentQuery from the DocumentDB SDK, uses Json.NET serializer when querying the database, whilst servicefabric uses DataContractSerializer for serializing the service-messages.
So when the content part of document class is being deserialized from the DocumentDB it becomes:
Newtonsoft.Json.Linq.JObject
But when it is serialized back through the returned service-message you get the exception:
Type 'Newtonsoft.Json.Linq.JToken' is a recursive collection data
contract which is not supported. Consider modifying the definition of
collection 'Newtonsoft.Json.Linq.JToken' to remove references to
itself.
To illustrate this issue try the folowing code:
using System;
using System.IO;
using System.Text;
using System.Runtime.Serialization.Json;
using Newtonsoft.Json;
namespace jsoinissue
{
public class acontent
{
public int x { get; set; }
public int y { get; set; }
}
public class document
{
public int id { get; set; }
public string type { get; set; }
public object content { get; set; }
}
public class documenta : document
{
public new acontent content { get; set; }
}
public class Program
{
private const string JSON_A = "{\"id\":1,\"type\":\"A\",\"content\":{\"x\":1,\"y\":2}}";
private static string SerializeObject<T> (T obj)
{
try
{
DataContractJsonSerializer js = new DataContractJsonSerializer(typeof(T));
using (var ms = new MemoryStream())
{
js.WriteObject(ms, obj);
ms.Position = 0;
using (var sr = new StreamReader(ms))
return sr.ReadToEnd();
}
}
catch (Exception e)
{
return String.Format("EXCEPTION: {0}",e.Message);
}
}
public static void Main()
{
var A = JsonConvert.DeserializeObject<document>(JSON_A);
var a = SerializeObject<document>(A);//HERE BE TROUBLE
Console.WriteLine(a);
Console.ReadKey();
}
}
}
How could I best resolve this issue?

Your basic problem is that DataContractJsonSerializer does not support untyped, free-form JSON data. As explained in Working with untyped JSON in a WCF service the System.Json namespace was added to Silverlight for this purpose, but it seems that it never made it into the full .Net class library.
Instead, in your stateless microservice can do a nested serialization where the free-form JSON is represented as an escaped string literal when serializing using the data contract serializer. Thus your classes would look something like this:
[DataContract]
[JsonObject]
public abstract class documentbase
{
[DataMember]
[JsonProperty]
public int id { get; set; }
[DataMember]
[JsonProperty]
public string type { get; set; }
[IgnoreDataMember]
[JsonProperty("content")]
public abstract JToken JsonContent { get; set; }
[JsonIgnore]
[DataMember(Name = "content")]
string DataContractContent
{
get
{
if (JsonContent == null)
return null;
return JsonContent.ToString(Newtonsoft.Json.Formatting.None);
}
set
{
if (string.IsNullOrEmpty(value))
JsonContent = null;
else
JsonContent = JToken.Parse(value);
}
}
}
[DataContract]
[JsonObject]
public class document : documentbase
{
JToken content;
public override JToken JsonContent { get { return content; } set { content = value; } }
}
[DataContract]
[JsonObject]
public class document<T> : documentbase where T : class
{
[IgnoreDataMember]
[JsonIgnore]
public T Content { get; set; }
public override JToken JsonContent
{
get
{
if (Content == null)
return null;
return JToken.FromObject(Content);
}
set
{
if (value == null || value.Type == JTokenType.Null)
Content = null;
else
Content = value.ToObject<T>();
}
}
}
Then the JSON generated by SerializeObject<document>(A) will look like:
{
"content":"{\"x\":1,\"y\":2}",
"id":1,
"type":"A"
}
Then, on the receiving system, you can deserialize to a document using the data contract serializer, then query the deserialized JToken JsonContent with LINQ to JSON. Alternatively, if the receiving system knows to expect a document<acontent> it can deserialize the data contract JSON as such, since document and document<T> have identical data contracts.

Have you looked into changing away from DataContractSerializer to a serializer with better support instead? Here's how you'd plug in a different serializer.
class InitializationCallbackAdapter
{
public Task OnInitialize()
{
this.StateManager.TryAddStateSerializer(new MyStateSerializer());
return Task.FromResult(true);
}
public IReliableStateManager StateManager { get; set; }
}
class MyStatefulService : StatefulService
{
public MyStatefulService(StatefulServiceContext context)
: this(context, new InitializationCallbackAdapter())
{
}
public MyStatefulService(StatefulServiceContext context, InitializationCallbackAdapter adapter)
: base(context, new ReliableStateManager(context, new ReliableStateManagerConfiguration(onInitializeStateSerializersEvent: adapter.OnInitialize)))
{
adapter.StateManager = this.StateManager;
}
}
This could be newtonsoft or whatever. Also I believe that the method is currently marked "Deprecated" however there's no alternative, so if it solves your problem go ahead and use it.

How to deserialize specific element with DataContractJsonSerializer

I'm trying to deserialize json data with DataContractJsonSerializer class. a problem is how to set root element?
my json data is here.
{
"delete":{
"status":{
"id":696142765093072896,
"id_str":"696142765093072896",
"user_id":2223183576,
"user_id_str":"2223183576"
},
"timestamp_ms":"1454808363540"
}
}
and I wrote class for deserialization like this. but it isn't works. my Status always null.
[DataContract(Name="delete")]
public class Delete
{
[DataMember(Name="status")]
public DeletedStatus Status { get; set; }
}
public class DeletedStatus
{
[DataMember(Name = "id")]
public long Id { get; set; }
[DataMember(Name = "user_id")]
public long UserId { get; set; }
}
how can I start parse json from specific element?

Based on what I can tell from the JSON, the deserialization appears to be failing because the root property of the object is the "delete" property. I don't believe this will work with the DataContractJsonSerializer simply because the given type will not match the Delete type. One other possible issue is that I see the DeleteStatus class is missing a [DataContract] attribute.
Long story short, there is no simple way of doing what you want to do. That being said, there is a short and sweet way of deserializing the JSON without adding a lot of extra headache. I suggest creating a data type that represents the JSON in its current state, and deserialize to that type instead.
I wrote a Unit Test that you can run from a Visual Studio test project. I hope this helps.
JsonDeserializationTests.cs
using Microsoft.VisualStudio.TestTools.UnitTesting;
using System.Runtime.Serialization.Json;
using System.IO;
namespace SerializationTests {
[TestClass]
public class JsonDeserializationTests {
[TestMethod]
public void Deserialize_Delete_Type_Success() {
string json = string.Empty;
//Set the DataContractJsonSerializer target type to our wrapper type.
var ser = new DataContractJsonSerializer(typeof(DeleteWrapperJsonResult));
//Create an instance of the wrapper that reflects the JSON that you gave.
//This will help me mock the data that you gave.
var deleteWrapper = new DeleteWrapperJsonResult {
delete = new DeleteJsonResult {
status = new DeletedStatusJsonResult {
id = 696142765093072896,
user_id = 2223183576
}
}
};
//Convert the mock data to JSON to reflect the JSON that you gave.
using (var serStream = new MemoryStream()) {
using (var sr = new StreamReader(serStream)) {
ser.WriteObject(serStream, deleteWrapper);
serStream.Position = 0;
json = sr.ReadToEnd(); //Set the JSON string here.
//Output "{\"delete\":{\"status\":{\"id\":696142765093072896,\"id_str\":\"696142765093072896\",\"user_id\":2223183576,\"user_id_str\":\"2223183576\"}}}"
}
}
//Prepeare to Deserialize the JSON.
var deserialized = default(DeleteWrapperJsonResult);
using (var deserStream = new MemoryStream()) {
using (var sw = new StreamWriter(deserStream)) {
sw.Write(json); //Write the JSON to the MemoryStream
sw.Flush();
deserStream.Seek(0, SeekOrigin.Begin);
//Deserialize the JSON into an instance of our wrapper class.
//This works because of the structure of the JSON.
deserialized = (DeleteWrapperJsonResult)ser.ReadObject(deserStream);
}
}
//Initialize the actual Delete instanace with what was deserialized.
var delete = new Delete {
Status = new DeletedStatus {
//These values were populated with the JSON values.
UserId = deserialized.delete.status.user_id,
Id = deserialized.delete.status.id
}
};
//Write asserts around what was given and check for equality.
Assert.AreEqual(delete.Status.UserId, deleteWrapper.delete.status.user_id);
Assert.AreEqual(delete.Status.Id, deleteWrapper.delete.status.id);
//Test Passes for Me
}
}
}
Delete.cs
using System.Runtime.Serialization;
namespace SerializationTests {
[DataContract]
[KnownType(typeof(Delete))]
public class Delete {
[DataMember]
public DeletedStatus Status { get; set; }
}
[DataContract]
[KnownType(typeof(DeletedStatus))]
public class DeletedStatus {
[DataMember]
public long Id { get; set; }
[DataMember]
public long UserId { get; set; }
}
/**************************************************************
These types below are what comprise our wrapper class so that we can
use the JSON in its current state. The wrapper classes have properties that
are synonymous with the JSON properties.
**************************************************************/
//This structure represents the object nesting as it appears currently in your example.
[DataContract]
[KnownType(typeof(DeleteJsonResult))]
public class DeleteWrapperJsonResult {
[DataMember]
public DeleteJsonResult delete { get; set; }
}
[DataContract]
[KnownType(typeof(DeleteJsonResult))]
public class DeleteJsonResult {
[DataMember]
public DeletedStatusJsonResult status { get; set; }
}
[DataContract]
[KnownType(typeof(DeletedStatusJsonResult))]
public class DeletedStatusJsonResult {
[DataMember]
public long id { get; set; }
[DataMember]
public string id_str {
get {
return id.ToString();
}
set {
return;
}
}
[DataMember]
public long user_id { get; set; }
[DataMember]
public string user_id_str {
get {
return user_id.ToString();
}
set {
return;
}
}
}
}
As of the time of this writing, my unit test is passing! Let me know if I can assist further.

Deserializing a newer version of an object from an older version of the object

Suppose I had this class:
[Serializable]
public class SomeClass
{
public SomeClass() {//init}
public string SomeString {get;set;}
}
This class gets Serialized when the application closes, and gets deserialized on the next run.
Then, I built it and released the application, and now the class has changed:
[Serializable]
public class SomeClass
{
public SomeClass() {//init}
public string SomeString {get;set;}
public int SomeInt {get;set;}
}
Is there a way to set a property to its default on deserialization in case its not found in the old serialized object?
One way I thought about is keeping the old version of the class, then check the version that was serialized then looping properties of the old object and setting them in the new object, but this is non sense to me, any other solution that makes sense?

You can mark fields with the attribute
[OptionalField()]
as explained in Version Tolerant Serialization
The class would then look like this:
[Serializable()]
public class SomeClass
{
public SomeClass() {//init}
public string SomeString { get; set; }
[OptionalField(VersionAdded = 2)]
public int SomeInt { get; set; }
[OnDeserialized()]
private void SetValuesOnDeserialized(StreamingContext context)
{
this.SomeInt = 10;
}
}

What i would do is base the SomeInt on a field where the field has a default value. IE.
public class SomeClass
{
public SomeClass() { }
int someInt = 10;
public string SomeString { get; set; }
public int SomeInt
{
get { return someInt; }
set { someInt = value; }
}
}
Then when the serializer deserializes the object if the SomeInt value is not provided the default value is still set.
EDIT: Update
Added a sample app using the XML serializer. Now to toggle the class type simply uncomment the #define serialize statement in row 2.
//uncomment for serialization
//#define serialize
using System;
using System.IO;
using System.Xml.Serialization;
namespace TestSerializer
{
class Program
{
static void Main(string[] args)
{
#if serialize
SomeClass some = new SomeClass();
some.SomeString = "abc";
XmlSerializer serializer = new XmlSerializer(some.GetType());
using (StringWriter writer = new StringWriter())
{
serializer.Serialize(writer, some);
File.WriteAllText("D:\\test.xml", writer.ToString());
}
#else
XmlSerializer serializer = new XmlSerializer(typeof(SomeClass));
using (StringReader reader = new StringReader(File.ReadAllText("D:\\test.xml")))
{
var o = serializer.Deserialize(reader) as SomeClass;
if (o != null)
Console.WriteLine(o.SomeInt);
}
Console.ReadKey();
#endif
}
}
#if serialize
[Serializable]
public class SomeClass
{
public SomeClass() { }
public string SomeString { get; set; }
}
#else
[Serializable]
public class SomeClass
{
public SomeClass() { }
private int someInt = 10;
public string SomeString { get; set; }
public int SomeInt
{
get { return someInt; }
set { someInt = value; }
}
}
#endif
}

How can I deserialize an XML, when I only know the type of an ancestor class?

Let's assume I want to deserialize this (I've removed the namespaces to make things simpler):
<TextField>
<Caption>Location</Caption>
<Name>Location</Name>
</TextField>
TextField inherits from FormField, so in my class definition of FormField looks something like this:
[KnownType(typeof(TextField))]
[DataContract(Name = "FormField"]
public abstract class FormField
{
[DataMember]
public string Name { get; set; }
}
TextField class looks like this:
[DataContract(Name = "TextField")]
public class TextField : FormField
{
[DataMember]
public string Caption { get; set; }
}
I tried deserializing using this:
internal static FormField Deserialize(string xml)
{
var serializer = new DataContractSerializer(typeof(FormField)});
using (var backing = new StringReader(xml))
{
using (var reader = new XmlTextReader(backing))
{
return serializer.ReadObject(reader) as FormField;
}
}
}
I get a SerializationException: "Expecting element 'FormField'..."

Shouldn't your model look like this then?
public abstract class TextField: FormField
{
[DataMember]
public string Name { get; set; }
}
just a stab, I actually just started diving into the DataContractSerializer stuff list night oddly enough.

To solve my problem I added a container node to the XML, so that it looks like this:
<FormFieldContainer>
<TextField>
<Caption>Location</Caption>
<Name>Location</Name>
</TextField>
</FormFieldContainer>
I created this class:
[DataContract(Name = "FormFieldContainer")]
internal class FormFieldContainer
{
[DataMember]
internal FormField FormField { get; set; }
}
And my deserialize method looks like this:
internal static FormField Deserialize(string xml)
{
var serializer = new DataContractSerializer(typeof(FormFieldContainer));
using (var backing = new StringReader(xml))
{
using (var reader = new XmlTextReader(backing))
{
return ((FormFieldContainer)serializer.ReadObject(reader)).FormField;
}
}
}
If anyone has a better solution, please share it :)

How to make a value type nullable with .NET XmlSerializer?

Let's suppose I have this object:
[Serializable]
public class MyClass
{
public int Age { get; set; }
public int MyClassB { get; set; }
}
[Serializable]
public class MyClassB
{
public int RandomNumber { get; set; }
}
The XmlSerializer will serialize the object like that:
<MyClass>
<Age>0</age>
<MyClassB>
<RandomNumber>4234</RandomNumber>
</MyClassB>
</MyClass>
How can I made the property Age nullable? IE: to not serialize the property Age when it's under 0?
I tried with the Nullable, but it serialize my object like that:
<MyClass>
<Age d5p1:nil="true" />
<MyClassB>
<RandomNumber>4234</RandomNumber>
</MyClassB>
</MyClass>
By reading the MSDN documentation I found this:
You cannot apply the IsNullable property to a member typed as a value type because a value type cannot contain nullNothingnullptra null reference (Nothing in Visual Basic). Additionally, you cannot set this property to false for nullable value types. When such types are nullNothingnullptra null reference (Nothing in Visual Basic), they will be serialized by setting xsi:nil to true.
source: http://msdn.microsoft.com/en-us/library/system.xml.serialization.xmlelementattribute.isnullable.aspx
I understand a value type can't be set to null. A valuetype is always set to something. The serialization can't make the decision to serialize it or not based on it's current value.
I tried with the attributes, but it didn't work out. I tried creating an agecontainer object and manipulate it's serialization with attributes, but it didn't work out.
What I really want is:
<MyClass>
<MyClassB>
<RandomNumber>4234</RandomNumber>
</MyClassB>
</MyClass>
When the property Age is below 0 (zero).
Looks like you'll have to implement custom serialization.
Yeah, that's what I though too, but I'd like to get away without it.
In the application, the object is much more complex, and I would like to not handle the serialization myself.

I just discovered this. XmlSerialier looks for a XXXSpecified boolean property to determine if it should be included. This should solve the problem nicely.
[Serializable]
public class MyClass
{
public int Age { get; set; }
[XmlIgnore]
public bool AgeSpecified { get { return Age >= 0; } }
public int MyClassB { get; set; }
}
[Serializable]
public class MyClassB
{
public int RandomNumber { get; set; }
}
Proof:
static string Serialize<T>(T obj)
{
var serializer = new XmlSerializer(typeof(T));
var builder = new StringBuilder();
using (var writer = new StringWriter(builder))
{
serializer.Serialize(writer, obj);
return builder.ToString();
}
}
static void Main(string[] args)
{
var withoutAge = new MyClass() { Age = -1 };
var withAge = new MyClass() { Age = 20 };
Serialize(withoutAge); // = <MyClass><MyClassB>0</MyClassB></MyClass>
Serialize(withAge); // = <MyClass><Age>20</Age><MyClassB>0</MyClassB></MyClass>
}
Edit: Yes, it is a documented feature. See the MSDN entry for XmlSerializer
Another option is to use a special pattern to create a Boolean field recognized by the XmlSerializer, and to apply the XmlIgnoreAttribute to the field. The pattern is created in the form of propertyNameSpecified. For example, if there is a field named "MyFirstName" you would also create a field named "MyFirstNameSpecified" that instructs the XmlSerializer whether to generate the XML element named "MyFirstName".

Extending Samuel's answer and Greg Beech's comment to the case of a boolean property: if the property is of type bool then you can't write a simple test in the propertySpecified property.
A solution is to use a Nullable<bool> type, then the test in the propertySpecified property is simply property.HasValue. e.g.
using System.Xml.Serialization;
public class Person
{
public bool? Employed { get; set; }
[XmlIgnore]
public bool EmployedSpecified { get { return Employed.HasValue; } }
}
An alternative to using a nullable type for a numeric property (suggested by Greg Beech) is to set the value property to an invalid default value, such as -1, as follows:
using System.ComponentModel;
using System.Xml.Serialization;
public class Person
{
[DefaultValue(-1)]
public int Age { get; set; }
[XmlIgnore]
public bool AgeSpecified { get { return Age >= 0; } }
}

You can use XmlElementAttribute.IsNullable:
[Serializable]
public class MyClass
{
[XmlElement(IsNullable = true)]
public int? Age { get; set; }
public int MyClassB { get; set; }
}

This should help
Make Age int? and..
public bool ShouldSerializeAge() { return Age.HasValue; }
..it does mean adding the ShouldSerializeXXX methods to your class!

You need to do custom XML serialization; see IXmlSerializer.
public class MyClass : IXmlSerializable
{
public int Age { get; set; }
public MyClassB MyClassB { get; set; }
public System.Xml.Schema.XmlSchema GetSchema()
{
// http://msdn.microsoft.com/en-us/library/system.xml.serialization.ixmlserializable.aspx
return null;
}
public void ReadXml(XmlReader reader)
{
if (reader.IsStartElement("Age"))
Age = reader.ReadContentAsInt();
var serializer = new XmlSerializer(typeof(MyClassB));
MyClassB = (MyClassB)serializer.Deserialize(reader);
}
public void WriteXml(XmlWriter writer)
{
if (Age > 0)
{
writer.WriteStartElement("Age");
writer.WriteValue(Age);
writer.WriteEndElement();
}
var serializer = new XmlSerializer(typeof(MyClassB));
serializer.Serialize(writer, MyClassB);
}
}

Forget about Nullable ... ShouldSerializeXXX is a pretty solution.
Here Age will be serialized upon your condition.
[Serializable]
public class MyClass
{
public int Age { get; set; }
public int MyClassB { get; set; }
#region Conditional Serialization
public bool ShouldSerializeAge() { return age > 0; }
#endregion
}
[Serializable]
public class MyClassB
{
public int RandomNumber { get; set; }
}

xsd.exe will autogenerate the XXXSpecified property and accessors if you set the 'minoccurs' attribute as 'minoccurs="0"' for an element ... if you are using a schema to define your xml/class