The default values for classes generated with protogen don't seem to be serialized when UseImplicitZeroDefaults = false.
I have a small .proto file:
package protobuf;
option java_package = "com.company.protobuf";
option java_outer_classname = "Test";
message TestMessage{
optional string Message = 1;
optional bool ABool = 2;
optional int32 AnInt = 3;
}
Using protogen.exe, I've generated a TestMessage class that I'm trying to send back and forth across the wire to a Java app. I can't seem to get protobuf-net to serialize a value of zero for AnInt or false for ABool, including setting UseImplicitZeroDefaults=false. However, using annotated classes for serialization with that setting works. Here's an equivalent class to the one I generated:
[ProtoContract]
class Test2
{
[ProtoMember(1)]
public string Message { get; set; }
[ProtoMember(2)]
public bool ABool { get; set; }
[ProtoMember(3)]
public int AnInt { get; set; }
}
Initializing the two classes with the same data and serializing to byte[] shows that four extra bytes are coming from the annotated class.
...
private static readonly RuntimeTypeModel serializer;
static Program()
{
serializer = TypeModel.Create();
serializer.UseImplicitZeroDefaults = false;
Console.WriteLine(serializer.UseImplicitZeroDefaults); //prints false
}
static void SendMessages(ITopic topic, ISession session)
{
Console.WriteLine(serializer.UseImplicitZeroDefaults);
TestMessage t = new TestMessage();
t.ABool = false;
t.AnInt = 0;
t.Message = "Test Message";
using (var o = new MemoryStream())
{
serializer.Serialize(o, t);
Console.WriteLine(string.Format("Tx: Message={0} ABool={1} AnInt={2}", t.Message, t.ABool, t.AnInt));
Console.WriteLine(o.ToArray().Length);
}
Test2 t2 = new Test2();
t2.ABool = false;
t2.AnInt = 0;
t2.Message = "Test Message";
using (var o = new MemoryStream())
{
serializer.Serialize(o, t2);
Console.WriteLine(string.Format("Tx: Message={0} ABool={1} AnInt={2}", t.Message, t.ABool, t.AnInt));
Console.WriteLine(o.ToArray().Length);
}
}
Output:
False
Tx: Message=Test Message ABool=False AnInt=0
14
Tx: Message=Test Message ABool=False AnInt=0
18
Is there a setting I'm missing? Or do classes generated from .proto files use a different mechanism for serialization? In an ideal world, I would expect the UseImplicitZeroDefaults setting to get picked up by both the annotated and generated classes on their way through the serializer.
If you add -p:detectMissing to your call to protogen, it should emit code following a different pattern that allows for better tracking of these. Basically, it should do what you want then.
Related
I am creating an gRPC service and we decided to choose the code first approach with protobuf-net.
Now I am running into a scenario where we have a couple of classes that need to be wrapped.
We do not want to define KnownTypes in the MyMessage class (just a sample name to illustrate the problem).
So I am trying to use the Any type which currently gives me some struggle with packing.
The sample code has the MyMessage which defines some header values and has to possiblity to deliver any type as payload.
[ProtoContract]
public class MyMessage
{
[ProtoMember(1)] public int HeaderValue1 { get; set; }
[ProtoMember(2)] public string HeaderValue2 { get; set; }
[ProtoMember(3)] public Google.Protobuf.WellknownTypes.Any Payload { get; set; }
}
[ProtoContract]
public class Payload1
{
[ProtoMember(1)] public bool Data1 { get; set; }
[ProtoMember(2)] public string Data2 { get; set; }
}
[ProtoContract]
public class Payload2
{
[ProtoMember(1)] public string Data1 { get; set; }
[ProtoMember(2)] public string Data2 { get; set; }
}
Somewhere in the code I construct my message with a payload ...
Payload2 payload = new Payload2 {
Data1 = "abc",
Data2 = "def"
};
MyMessage msg = new MyMessage
{
HeaderValue1 = 123,
HeaderValue2 = "iAmHeaderValue2",
Payload = Google.Protobuf.WellknownTypes.Any.Pack(payload)
};
Which doesn't work because Payload1 and Payload2 need to implement Google.Protobuf.IMessage.
Since I can't figure out how and do not find a lot information how to do it at all I am wondering if I am going a wrong path.
How is it intedend to use Any in protobuf-net?
Is there a simple (yet compatible) way to pack a C# code first class into Google.Protobuf.WellknownTypes.Any?
Do I really need to implement Google.Protobuf.IMessage?
Firstly, since you say "where we have a couple of classes that need to be wrapped" (emphasis mine), I wonder if what you actually want here is oneof rather than Any. protobuf-net has support for the oneof concept, although it isn't obvious from a code-first perspective. But imagine we had (in a contract-first sense):
syntax = "proto3";
message SomeType {
oneof Content {
Foo foo = 1;
Bar bar = 2;
Blap blap = 3;
}
}
message Foo {}
message Bar {}
message Blap {}
This would be implemented (via the protobuf-net schema tools) as:
private global::ProtoBuf.DiscriminatedUnionObject __pbn__Content;
[global::ProtoBuf.ProtoMember(1, Name = #"foo")]
public Foo Foo
{
get => __pbn__Content.Is(1) ? ((Foo)__pbn__Content.Object) : default;
set => __pbn__Content = new global::ProtoBuf.DiscriminatedUnionObject(1, value);
}
public bool ShouldSerializeFoo() => __pbn__Content.Is(1);
public void ResetFoo() => global::ProtoBuf.DiscriminatedUnionObject.Reset(ref __pbn__Content, 1);
[global::ProtoBuf.ProtoMember(2, Name = #"bar")]
public Bar Bar
{
get => __pbn__Content.Is(2) ? ((Bar)__pbn__Content.Object) : default;
set => __pbn__Content = new global::ProtoBuf.DiscriminatedUnionObject(2, value);
}
public bool ShouldSerializeBar() => __pbn__Content.Is(2);
public void ResetBar() => global::ProtoBuf.DiscriminatedUnionObject.Reset(ref __pbn__Content, 2);
[global::ProtoBuf.ProtoMember(3, Name = #"blap")]
public Blap Blap
{
get => __pbn__Content.Is(3) ? ((Blap)__pbn__Content.Object) : default;
set => __pbn__Content = new global::ProtoBuf.DiscriminatedUnionObject(3, value);
}
public bool ShouldSerializeBlap() => __pbn__Content.Is(3);
public void ResetBlap() => global::ProtoBuf.DiscriminatedUnionObject.Reset(ref __pbn__Content, 3);
optionally with an enum to help:
public ContentOneofCase ContentCase => (ContentOneofCase)__pbn__Content.Discriminator;
public enum ContentOneofCase
{
None = 0,
Foo = 1,
Bar = 2,
Blap = 3,
}
This approach may be easier and preferable to Any.
On Any:
Short version: protobuf-net has not, to date, had any particular request to implement Any. It probably isn't a huge amount of work - simply: it hasn't yet happened. It looks like you're referencing both protobuf-net and the Google libs here, and using the Google implementation of Any. That's fine, but protobuf-net isn't going to use it at all - it doesn't know about the Google APIs in this context, so: implementing IMessage won't actually help you.
I'd be more than happy to look at Any with you, from the protobuf-net side. Ultimately, time/availability is always the limiting factor, so I prioritise features that are seeing demand. I think you may actually be the first person asking me about Any in protobuf-net.
My Any implementation.
[ProtoContract(Name = "type.googleapis.com/google.protobuf.Any")]
public class Any
{
/// <summary>Pack <paramref name="value"/></summary>
public static Any Pack(object? value)
{
// Handle null
if (value == null) return new Any { TypeUrl = null!, Value = Array.Empty<byte>() };
// Get type
System.Type type = value.GetType();
// Write here
MemoryStream ms = new MemoryStream();
// Serialize
RuntimeTypeModel.Default.Serialize(ms, value);
// Create any
Any any = new Any
{
TypeUrl = $"{type.Assembly.GetName().Name}/{type.FullName}",
Value = ms.ToArray()
};
// Return
return any;
}
/// <summary>Unpack any record</summary>
public object? Unpack()
{
// Handle null
if (TypeUrl == null || Value == null || Value.Length == 0) return null;
// Find '/'
int slashIx = TypeUrl.IndexOf('/');
// Convert to C# type name
string typename = slashIx >= 0 ? $"{TypeUrl.Substring(slashIx + 1)}, {TypeUrl.Substring(0, slashIx)}" : TypeUrl;
// Get type (Note security issue here!)
System.Type type = System.Type.GetType(typename, true)!;
// Deserialize
object value = RuntimeTypeModel.Default.Deserialize(type, Value.AsMemory());
// Return
return value;
}
/// <summary>Test type</summary>
public bool Is(System.Type type) => $"{type.Assembly.GetName().Name}/{type.FullName}" == TypeUrl;
/// <summary>Type url (using C# type names)</summary>
[ProtoMember(1)]
public string TypeUrl = null!;
/// <summary>Data serialization</summary>
[ProtoMember(2)]
public byte[] Value = null!;
/// <summary></summary>
public static implicit operator Container(Any value) => new Container(value.Unpack()! );
/// <summary></summary>
public static implicit operator Any(Container value) => Any.Pack(value.Value);
/// <summary></summary>
public struct Container
{
/// <summary></summary>
public object? Value;
/// <summary></summary>
public Container()
{
this.Value = null;
}
/// <summary></summary>
public Container(object? value)
{
this.Value = value;
}
}
}
'System.Object' can be used as a field or property in a surrounding Container record:
[DataContract]
public class Container
{
/// <summary></summary>
[DataMember(Order = 1, Name = nameof(Value))]
public Any.Container Any { get => new Any.Container(Value); set => Value = value.Value; }
/// <summary>Object</summary>
public object? Value;
}
Serialization
RuntimeTypeModel.Default.Add(typeof(Any.Container), false).SetSurrogate(typeof(Any));
var ms = new MemoryStream();
RuntimeTypeModel.Default.Serialize(ms, new Container { Value = "Hello world" });
Container dummy = RuntimeTypeModel.Default.Deserialize(typeof(Container), ms.ToArray().AsMemory()) as Container;
I have some data which I serialized with protobuf.net. The data is a map, and contains some duplicates (which happened as my key didn't implement IEquatable)
I want to deserialize the data into a dictionary and ignore duplicates.
There seems to be an attribute for that, i.e. [ProtoMap(DisableMap=false)], which the documentation says:
Disable "map" handling; dictionaries will use .Add(key, value) instead
of [key] = value. ...
Basically I want to behavior to be [key] = value, but apparently, the attribute is ignored.
Am I doing anything wrong? Is there any way to achieve the desired (and documented) behavior of ignoring duplicates?
Example code:
1. Produce data with duplicates:
// ------------- ------------- ------------- ------------- ------------- ------------- -------------
// The following part generated the bytes, which requires the key NOT to implement IEquatable
// ------------- ------------- ------------- ------------- ------------- ------------- -------------
var cache = new MyTestClass() { Dictionary = new Dictionary<MyTestKey, string>() };
cache.Dictionary[new MyTestKey { Value = "X" }] = "A";
cache.Dictionary[new MyTestKey { Value = "X" }] = "B";
var bytes = cache.Serialize();
var bytesStr = string.Join(",", bytes); // "10,8,10,3,10,1,88,18,1,65,10,8,10,3,10,1,88,18,1,66";
//..
[DataContract]
public class MyTestKey
{
[DataMember(Order = 1)]
public string Value { get; set; }
}
[DataContract]
public class MyTestClass
{
[DataMember(Order = 1)]
[ProtoMap(DisableMap = false)]
public Dictionary<MyTestKey, string> Dictionary { get; set; }
}
´´´
2. Try deserialize the data, with property IEquatable, which fails..:
[DataContract]
public class MyTestKey : IEquatable<MyTestKey>
{
[DataMember(Order = 1)]
public string Value { get; set; }
public bool Equals(MyTestKey other)
{
if (ReferenceEquals(null, other)) return false;
if (ReferenceEquals(this, other)) return true;
return Value == other.Value;
}
public override bool Equals(object obj)
{
if (ReferenceEquals(null, obj)) return false;
if (ReferenceEquals(this, obj)) return true;
if (obj.GetType() != this.GetType()) return false;
return Equals((MyTestKey) obj);
}
public override int GetHashCode()
{
return (Value != null ? Value.GetHashCode() : 0);
}
}
//..
var bytesStr2 = "10,8,10,3,10,1,88,18,1,65,10,8,10,3,10,1,88,18,1,66";
var bytes2 = bytesStr2.Split(',').Select(byte.Parse).ToArray();
var cache = bytes2.DeserializeTo<MyTestClass>();
´´´
Exception An item with the same key has already been added.
public static class SerializationExtensions
{
public static T DeserializeTo<T>(this byte[] bytes)
{
if (bytes == null)
return default(T);
using (var ms = new MemoryStream(bytes))
{
return Serializer.Deserialize<T>(ms);
}
}
public static byte[] Serialize<T>(this T setup)
{
using (var ms = new MemoryStream())
{
Serializer.Serialize(ms, setup);
return ms.ToArray();
}
}
There's a few different things going on here; "map" mode is actually the one you want here - so it isn't that you're trying to disable map, but actually you're trying to force it on (it is now on by default in most common dictionary scenarios).
There are some complications:
the library only processes [ProtoMap(...)] when processing a [ProtoMember(...)] for a [ProtoContract(...)]
even then, it only processes [ProtoMap(...)] for key-types that are valid as "map" keys in the proto specification
you can turn it on manually (not via the attributes), but in v2.* it enforces the same check as #2 at runtime, which means it will fail
The manual enable from #3 works in v3.* (currently in alpha):
RuntimeTypeModel.Default[typeof(MyTestClass)][1].IsMap = true;
however, this is obviously inelegant, and today requires using an alpha build (we've been using it in production here at Stack Overflow for an extended period; I just need to get a release together - docs, etc).
Given that it works, I'm tempted to propose that we should soften #2 in v3.*, such that while the default behavior remains the same, it would still check for [ProtoMap(...)] for custom types, and enable that mode. I'm on the fence about whether to soften #1.
I'd be interested in your thoughts on these things!
But to confirm: the following works fine in v3.* and outputs "B" (minor explanation of the code: in protobuf, append===merge for root objects, so serializing two payloads one after the other has the same effect as serializing a dictionary with the combined content, so the two Serialize calls spoofs a payload with two identical keys):
static class P
{
static void Main()
{
using var ms = new MemoryStream();
var key = new MyTestKey { Value = "X" };
RuntimeTypeModel.Default[typeof(MyTestClass)][1].IsMap = true;
Serializer.Serialize(ms, new MyTestClass() { Dictionary =
new Dictionary<MyTestKey, string> { { key, "A" } } });
Serializer.Serialize(ms, new MyTestClass() { Dictionary =
new Dictionary<MyTestKey, string> { { key, "B" } } });
ms.Position = 0;
var val = Serializer.Deserialize<MyTestClass>(ms).Dictionary[key];
Console.WriteLine(val); // B
}
}
I think what I'd like is if, in v3.*, it worked without the IsMap = true line, with:
[ProtoContract]
public class MyTestClass
{
[ProtoMember(1)]
[ProtoMap] // explicit enable here, because not a normal map type
public Dictionary<MyTestKey, string> Dictionary { get; set; }
}
There seems to be a bug / inconsistency in the Microsoft XmlSerializer: If you have a property marked with a System.ComponentModel.DefaultValue attribute, this does not get serialized. Fair enough - this could be seen as an expected behavior.
The problem is the same attribute is not respected when deserializing. The code below illustrates the issue.
Question is how could I bypass this? I have potentially hundreds of business classes with default values used in the UI tier (Views), so default value initialization in constructor is not an option. It has to be something generic. I could create a completely new default attribute, but it seems like duplicate work. Do you see a way to override the XmlSerializer behavior or should I use just another serializer that does the job better?
The example code:
public class DefaultValueTestClass
{
[System.ComponentModel.DefaultValue(10000)]
public int Foo { get; set; }
}
[TestMethod]
public void SimpleDefaultValueTest()
{
// Create object and set the property value TO THE DEFAULT
var before = new DefaultValueTestClass();
before.Foo = 10000;
// Serialize => xml
var serializer = new System.Xml.Serialization.XmlSerializer(typeof(DefaultValueTestClass));
string xml;
using (var stream = new System.IO.StringWriter())
{
serializer.Serialize(stream, before);
xml = stream.ToString();
}
// Deserialize the same object
DefaultValueTestClass after;
using (var reader = new System.IO.StringReader(xml))
{
after = (DefaultValueTestClass)serializer.Deserialize(reader);
}
// before.Foo = 10000
// after.Foo = 0
Assert.AreEqual(before.Foo, after.Foo);
}
It is your job to implement the defaults; [DefaultValue] merely says "this is the default, you don't need to worry about this" - it doesn't apply it. This applies not just to XmlSerializer, but to the core System.ComponentModel API to which [DefaultValue] belongs (which drives things like the bold / not-bold in PropertyGrid, etc)
Basically:
public class DefaultValueTestClass
{
public DefaultValueTestClass()
{
Foo = 10000;
}
[DefaultValue(10000)]
public int Foo { get; set; }
}
will work in the way you expect. If you want it to serialize whether or not it is that particular value, then the correct implementations is:
public class DefaultValueTestClass
{
public DefaultValueTestClass()
{
Foo = 10000;
}
public int Foo { get; set; }
}
If you want to preserve the [DefaultValue], but want it to always serialize, then:
public class DefaultValueTestClass
{
[DefaultValue(10000)]
public int Foo { get; set; }
[Browsable(false), EditorBrowsable(EditorBrowsableState.Never)]
public bool ShouldSerializeFoo() { return true; }
}
Where ShouldSerialize* is another pattern from System.ComponentModel that is recognised by several serializers.
And here's some UI code to show that XmlSerializer is actually doing exactly the same things that the UI code (built on System.ComponentModel) has always done:
using System.ComponentModel;
using System.Windows.Forms;
static class Program
{
[System.STAThread]
static void Main()
{
Application.EnableVisualStyles();
using (var form = new Form())
using (var grid = new PropertyGrid())
{
grid.Dock = DockStyle.Fill;
var obj = new DefaultValueTestClass
{ // TODO - try with other numbers to
// see bold / not bold
Foo = 10000
};
// note in the grid the value is shown not-bold; that is
// because System.ComponentModel is saying
// "this property doesn't need to be serialized"
// - or to show it more explicitly:
var prop = TypeDescriptor.GetProperties(obj)["Foo"];
bool shouldSerialize = prop.ShouldSerializeValue(obj);
// ^^^ false, because of the DefaultValueAttribute
form.Text = shouldSerialize.ToString(); // win title
grid.SelectedObject = obj;
form.Controls.Add(grid);
Application.Run(form);
}
}
}
public class DefaultValueTestClass
{
[System.ComponentModel.DefaultValue(10000)]
public int Foo { get; set; }
}
I am checking if protobuf-net can be an in place replacement for DataContracts. Besides the excellent performance it is really a neat library. The only issue I have is that the .NET serializers do not make any assumptions what they are currently de/serializing. Especially objects which do contain reference to the typed object are a problem.
[DataMember(Order = 3)]
public object Tag1 // The DataContract did contain a object which becomes now a SimulatedObject
{
get;
set;
}
I tried to mimic object with protocol buffers with a little generic helper which does store for each possible type in a different strongly typed field.
Is this an recommended approach to deal with fields which de/serialize into a number of different not related types?
Below is the sample code for a SimulatedObject which can hold up to 10 different types.
using System;
using System.Collections.Generic;
using System.IO;
using System.Runtime.Serialization;
using ProtoBuf;
using System.Diagnostics;
[DataContract]
public class SimulatedObject<T1,T2,T3,T4,T5,T6,T7,T8,T9,T10>
{
[DataMember(Order = 20)]
byte FieldHasValue; // the number indicates which field actually has a value
[DataMember(Order = 1)]
T1 I1;
[DataMember(Order = 2)]
T2 I2;
[DataMember(Order = 3)]
T3 I3;
[DataMember(Order = 4)]
T4 I4;
[DataMember(Order = 5)]
T5 I5;
[DataMember(Order = 6)]
T6 I6;
[DataMember(Order = 7)]
T7 I7;
[DataMember(Order = 8)]
T8 I8;
[DataMember(Order = 9)]
T9 I9;
[DataMember(Order = 10)]
T10 I10;
public object Data
{
get
{
switch(FieldHasValue)
{
case 0: return null;
case 1: return I1;
case 2: return I2;
case 3: return I3;
case 4: return I4;
case 5: return I5;
case 6: return I6;
case 7: return I7;
case 8: return I8;
case 9: return I9;
case 10: return I10;
default:
throw new NotSupportedException(String.Format("The FieldHasValue field has an invlaid value {0}. This indicates corrupt data or incompatible data layout chagnes", FieldHasValue));
}
}
set
{
I1 = default(T1);
I2 = default(T2);
I3 = default(T3);
I4 = default(T4);
I5 = default(T5);
I6 = default(T6);
I7 = default(T7);
I8 = default(T8);
I9 = default(T9);
I10 = default(T10);
if (value != null)
{
Type t = value.GetType();
if (t == typeof(T1))
{
FieldHasValue = 1;
I1 = (T1) value;
}
else if (t == typeof(T2))
{
FieldHasValue = 2;
I2 = (T2) value;
}
else if (t == typeof(T3))
{
FieldHasValue = 3;
I3 = (T3) value;
}
else if (t == typeof(T4))
{
FieldHasValue = 4;
I4 = (T4) value;
}
else if (t == typeof(T5))
{
FieldHasValue = 5;
I5 = (T5) value;
}
else if (t == typeof(T6))
{
FieldHasValue = 6;
I6 = (T6) value;
}
else if (t == typeof(T7))
{
FieldHasValue = 7;
I7 = (T7) value;
}
else if (t == typeof(T8))
{
FieldHasValue = 8;
I8 = (T8) value;
}
else if (t == typeof(T9))
{
FieldHasValue = 9;
I9 = (T9) value;
}
else if (t == typeof(T10))
{
FieldHasValue = 10;
I10 = (T10) value;
}
else
{
throw new NotSupportedException(String.Format("The type {0} is not supported for serialization. Please add the type to the SimulatedObject generic argument list.", t.FullName));
}
}
}
}
}
[DataContract]
class Customer
{
/*
[DataMember(Order = 3)]
public object Tag1 // The DataContract did contain a object which becomes now a SimulatedObject
{
get;
set;
}
*/
[DataMember(Order = 3)]
public SimulatedObject<bool, Other, Other, Other, Other, Other, Other, Other, Other, SomethingDifferent> Tag1 // Can contain up to 10 different types
{
get;
set;
}
[DataMember(Order = 4)]
public List<string> Strings
{
get;
set;
}
}
[DataContract]
public class Other
{
[DataMember(Order = 1)]
public string OtherData
{
get;
set;
}
}
[DataContract]
public class SomethingDifferent
{
[DataMember(Order = 1)]
public string OtherData
{
get;
set;
}
}
class Program
{
static void Main(string[] args)
{
Customer c = new Customer
{
Strings = new List<string> { "First", "Second", "Third" },
Tag1 = new SimulatedObject<bool, Other, Other, Other, Other, Other, Other, Other, Other, SomethingDifferent>
{
Data = new Other { OtherData = "String value "}
}
};
const int Runs = 1000 * 1000;
var stream = new MemoryStream();
var sw = Stopwatch.StartNew();
Serializer.Serialize<Customer>(stream, c);
sw = Stopwatch.StartNew();
for (int i = 0; i < Runs; i++)
{
stream.Position = 0;
stream.SetLength(0);
Serializer.Serialize<Customer>(stream, c);
}
sw.Stop();
Console.WriteLine("Data Size with Protocol buffer Serializer: {0}, {1} objects did take {2}s", stream.ToArray().Length, Runs, sw.Elapsed.TotalSeconds);
stream.Position = 0;
var newCustw = Serializer.Deserialize<Customer>(stream);
sw = Stopwatch.StartNew();
for (int i = 0; i < Runs; i++)
{
stream.Position = 0;
var newCust = Serializer.Deserialize<Customer>(stream);
}
sw.Stop();
Console.WriteLine("Read object with Protocol buffer deserializer: {0} objects did take {1}s", Runs, sw.Elapsed.TotalSeconds);
}
}
No, this solution is hard to maintain in a long term.
I recommend that you prepend the full name of the serialized type to the serialized data in the serialization process and read the type name in the beginning of the deserialization process (no need to change the protobuf source-code)
As a side note, you should try to avoid mixing object types in the deserialization process. I'm assuming you are updating an existing .net application and can't re-design it.
Update: Sample code
public byte[] Serialize(object myObject)
{
using (var ms = new MemoryStream())
{
Type type = myObject.GetType();
var id = System.Text.ASCIIEncoding.ASCII.GetBytes(type.FullName + '|');
ms.Write(id, 0, id.Length);
Serializer.Serialize(ms, myObject);
var bytes = ms.ToArray();
return bytes;
}
}
public object Deserialize(byte[] serializedData)
{
StringBuilder sb = new StringBuilder();
using (var ms = new MemoryStream(serializedData))
{
while (true)
{
var currentChar = (char)ms.ReadByte();
if (currentChar == '|')
{
break;
}
sb.Append(currentChar);
}
string typeName = sb.ToString();
// assuming that the calling assembly contains the desired type.
// You can include aditional assembly information if necessary
Type deserializationType = Assembly.GetCallingAssembly().GetType(typeName);
MethodInfo mi = typeof(Serializer).GetMethod("Deserialize");
MethodInfo genericMethod = mi.MakeGenericMethod(new[] { deserializationType });
return genericMethod.Invoke(null, new[] { ms });
}
}
I'm working on something similar now and I provided first version of the lib already:
http://bitcare.codeplex.com/
The current version doesn't support generics yet, but I plan to add it in the nearest time.
I uploaded source code only there-when I'm ready with generics I prepare bin version also...
It assumes both sides (client and server) know what they serialize/deserialize so there is no any reason to embed there full metadata info. Because of this serialization results are very small and generated serializers work very fast. It has data dictionaries, uses smart data storage (stores only important bits in short) and does final compression when necessary. If you need it just try if it solves your problem.
The license is GPL, but I will change it soon to less restrictive one(free for commercial usage also, but on your own risk like in GPL)
The version I uploaded to codeplex is working with some of my product. It's tested with different set of unit tests of course. They are not uploaded there, because I ported it to vs2012 and .net 4.5 and decided to create new sets of test cases for the incoming release.
I don't deal with abstract (so called opened) generics. I process parametrized generics. From data contract point of view parametrized generics are just specialized classes so I can process them as usual (as other classes) - the difference is in objects construction only and storage optimizations.
When I store information about null value on Nullable<> it takes one bit in storage stream only, if it's not null value I do serialization according to type provided as generics parameter (so I do serialization of DateTime for instance that can take from one bit for so called default value to a few bytes). The goal was to generate serialization code according to the current knowledge about data contracts on classes instead of doing it on the fly and wasting memory and processing power. When I see the property in some class based on some generic during code generation I know all the properties of that generic and I know the type of every property :) From this point of view it's concrete class.
I will change the license soon. I have to figure out first how to do it :) , because I see it's possible to choose from list of the provided license types but I can't provide my own license text. I see the license of Newtonsoft.Json is what I would have also, but I don't know yet how to change it...
The documentation has not been provided there yet, but in short it's easy to prepare your own serialization tests. You have to compile assembly with your types you want to store/serialize effective way, then create *.tt files in your serialization library (like for person class-it checks dependencies and generates code for other dependent classes also) and save the files (when you save them then it generates all the code for cooperation with serialization library). You can also create the task in your build config to regenerate source code from tt files every time you build the solution(probably Entity Framework generates the code similar way during the build).
You can compile your serialization library now and measure the performance and size of the results.
I need this serialization library for my framework for effective usage of entities with Azure Table and Blob storage so I plan to finish initial release soon...
I have a custom Fraction class, which I'm using throughout my whole project. It's simple, it consists of a single constructor, accepts two ints and stores them. I'd like to use the DataContractSerializer to serialize my objects used in my project, some of which include Fractions as fields. Ideally, I'd like to be able to serialize such objects like this:
<Object>
...
<Frac>1/2</Frac> // "1/2" would get converted back into a Fraction on deserialization.
...
</Object>
As opposed to this:
<Object>
...
<Frac>
<Numerator>1</Numerator>
<Denominator>2</Denominator>
</Frac>
...
</Object>
Is there any way to do this using DataContracts?
I'd like to do this because I plan on making the XML files user-editable (I'm using them as input for a music game, and they act as notecharts, essentially), and want to keep the notation as terse as possible for the end user, so they won't need to deal with as many walls of text.
EDIT: I should also note that I currently have my Fraction class as immutable (all fields are readonly), so being able to change the state of an existing Fraction wouldn't be possible. Returning a new Fraction object would be OK, though.
If you add a property that represents the Frac element and apply the DataMember attribute to it rather than the other properties you will get what you want I believe:
[DataContract]
public class MyObject {
Int32 _Numerator;
Int32 _Denominator;
public MyObject(Int32 numerator, Int32 denominator) {
_Numerator = numerator;
_Denominator = denominator;
}
public Int32 Numerator {
get { return _Numerator; }
set { _Numerator = value; }
}
public Int32 Denominator {
get { return _Denominator; }
set { _Denominator = value; }
}
[DataMember(Name="Frac")]
public String Fraction {
get { return _Numerator + "/" + _Denominator; }
set {
String[] parts = value.Split(new char[] { '/' });
_Numerator = Int32.Parse(parts[0]);
_Denominator = Int32.Parse(parts[1]);
}
}
}
DataContractSerializer will use a custom IXmlSerializable if it is provided in place of a DataContractAttribute. This will allow you to customize the XML formatting in anyway you need... but you will have to hand code the serialization and deserialization process for your class.
public class Fraction: IXmlSerializable
{
private Fraction()
{
}
public Fraction(int numerator, int denominator)
{
this.Numerator = numerator;
this.Denominator = denominator;
}
public int Numerator { get; private set; }
public int Denominator { get; private set; }
public XmlSchema GetSchema()
{
throw new NotImplementedException();
}
public void ReadXml(XmlReader reader)
{
var content = reader.ReadInnerXml();
var parts = content.Split('/');
Numerator = int.Parse(parts[0]);
Denominator = int.Parse(parts[1]);
}
public void WriteXml(XmlWriter writer)
{
writer.WriteRaw(this.ToString());
}
public override string ToString()
{
return string.Format("{0}/{1}", Numerator, Denominator);
}
}
[DataContract(Name = "Object", Namespace="")]
public class MyObject
{
[DataMember]
public Fraction Frac { get; set; }
}
class Program
{
static void Main(string[] args)
{
var myobject = new MyObject
{
Frac = new Fraction(1, 2)
};
var dcs = new DataContractSerializer(typeof(MyObject));
string xml = null;
using (var ms = new MemoryStream())
{
dcs.WriteObject(ms, myobject);
xml = Encoding.UTF8.GetString(ms.ToArray());
Console.WriteLine(xml);
// <Object><Frac>1/2</Frac></Object>
}
using (var ms = new MemoryStream(Encoding.UTF8.GetBytes(xml)))
{
ms.Position = 0;
var obj = dcs.ReadObject(ms) as MyObject;
Console.WriteLine(obj.Frac);
// 1/2
}
}
}
This MSDN article describes IDataContractSurrogate Interface which:
Provides the methods needed to substitute one type for another by the
DataContractSerializer during serialization, deserialization, and
export and import of XML schema documents.
Although way too late, still may help someone. Actually, allows to change XML for ANY class.
You can do this with the DataContractSerializer, albeit in a way that feels hacky to me. You can take advantage of the fact that data members can be private variables, and use a private string as your serialized member. The data contract serializer will also execute methods at certain points in the process that are marked with [On(De)Serializ(ed|ing)] attributes - inside of those, you can control how the int fields are mapped to the string, and vice-versa. The downside is that you lose the automatic serialization magic of the DataContractSerializer on your class, and now have more logic to maintain.
Anyways, here's what I would do:
[DataContract]
public class Fraction
{
[DataMember(Name = "Frac")]
private string serialized;
public int Numerator { get; private set; }
public int Denominator { get; private set; }
[OnSerializing]
public void OnSerializing(StreamingContext context)
{
// This gets called just before the DataContractSerializer begins.
serialized = Numerator.ToString() + "/" + Denominator.ToString();
}
[OnDeserialized]
public void OnDeserialized(StreamingContext context)
{
// This gets called after the DataContractSerializer finishes its work
var nums = serialized.Split("/");
Numerator = int.Parse(nums[0]);
Denominator = int.Parse(nums[1]);
}
}
You'll have to switch back to the XMLSerializer to do that. The DataContractSerializer is a bit more restrictive in terms of being able to customise the output.