Efficient (Space) Serialization for Network Transfer - c#

I am attempting to write some infrastructure to facilitate updating objects between a server and client(s). This will likely be used in a game, however, I feel that the question is not at all specific to the game (so I have asked it here).
For security and efficiency reasons I would like the server to selectively update object properties. For example, a specific property of an object may only be useful to the client which controls that object, as such the server will only update the 'owner' with this information. Alternatively, some properties may need to be sent to all clients. To implement this I have defined a custom attribute which specifies the manner in which the network should handle the property:
[AttributeUsage(AttributeTargets.Property)]
public class NetworkParameterAttribute : System.Attribute
{
public enum NetworkParameterType
{
ServerToOwner,
ServerToAll,
ServerToOwnerView,
OwnerToServer
}
private NetworkParameterType type;
public NetworkParameterType Type
{
get
{
return type;
}
}
public NetworkParameterAttribute(NetworkParameterType Type)
{
this.type = Type;
}
}
Now in an object class I can define properties like so:
public class TestObject
{
[NetworkParameter(NetworkParameterAttribute.NetworkParameterType.ServerToAll)]
public int ID { get; set; }
[NetworkParameter(NetworkParameterAttribute.NetworkParameterType.ServerToOwner)]
public string Name { get; set; }
}
I can then write a simple function which automatically grabs a certain set of properties from an object:
public byte[] GetBytes(NetworkParameterAttribute.NetworkParameterType type)
{
MemoryStream stream = new MemoryStream();
BinaryFormatter formatter = new BinaryFormatter();
foreach (PropertyInfo info in this.GetType().GetProperties())
{
foreach (object attribute in info.GetCustomAttributes(true))
{
if (attribute is NetworkParameterAttribute &&
((NetworkParameterAttribute)attribute).Type == type)
{
formatter.Serialize(stream, info.GetValue(this, null));
}
}
}
byte[] buf = new byte[stream.Length];
Array.Copy(stream.GetBuffer(), buf, stream.Length);
return buf;
}
A similar function can put the object back together on the receiving side. The issue that I am having is that the serialization is very inefficient in terms of space used. For example, grabbing the ServerToAll properties from a TestObject results in 54 bytes (whereas it could be as little as 4).
So the question: Is there a more efficient way of serializing objects to a byte stream that will work for my intended purpose? Note that I would prefer not to write a lot of serialization related code.
Thank you!

NetworkParameterAttribute should have additional field that denotes the corresponding property as "dirty". Every change to a property should effectively set this flag, and during serialization the flag should be reset. Only dirty properties should actually be serialized.
Additionally, now that the object is only partially serialized, during serialization now you need to provide the information about what properties are being serialized. Maintain one bitvector of dirty properties while you populate the stream, and put this bitvector in the beginning of returned byte array.
EDIT: Instead of having a flag inside an attribute we can have the actual value that was serialized last. The advantage is that we don't need additional code for each property to keep the flag synchronized with the property. During serialization we compare two values and serialize property if the values are not equal.

Related

C# Serialize WinForm

I am trying to serialize a winform, with the end goal of being able to recreate the values in the various controls of the form. My form contains the typical controls, buttons/radio buttons/checkboxes/textboxes/listbox/tab control.
I am receiving this error:
An exception of type 'System.InvalidOperationException' occurred
in System.Xml.dll but was not handled in user code
Additional information: There was an error reflecting type
'Receptionist_Program.Objects.Client.Client_NCQ'.
I setup properties for each value I want to save:
public bool CbMedTreat
{
get { return cbMedTreat.Checked; }
set { cbMedTreat.Checked = value; }
}
public List<Client_AddDoctor> TxtDocExplain // Client_AddDoctor is another form
{
get { return listDoctors; }
set { listDoctors = value; }
}
// etc, variety of string and bool properties
At the top of the class I have the decoration:
[Serializable]
public partial class Client_NCQ : Form
Finally, here is my code doing the serialization:
Client_NCQ badname = new Client_NCQ();
badname.Initialize();
badname.ShowDialog();
string result = "";
XmlSerializer xmlSerializer = new XmlSerializer(typeof(Client_NCQ));
// Error occurs here on above line: new XmlSerializer(typeof(Client_NCQ))
using (StringWriter textWriter = new StringWriter())
{
xmlSerializer.Serialize(textWriter, badname);
result = textWriter.ToString();
}
I tried two different things so far, first, I added the decoration [XmlIgnore] to the List<> property, this made no difference. Second, I tried ensuring that the constructor was empty and had no parameters.
Serializing an entire Form is a bad idea because it is not meant to be serialized:
it has a lot of properties that should not be serialized (e.g. displaying related properties)
even if it works properly, you will have a lot of data that is not relevant for your application's state
The correct solution is to keep all state information in your custom objects and bind to those objects using WinForm's databinding capabilities. If this means great changes to your application, try to serialize only the data that is relevant to constructing the state.
How can you know which data is relevant for application state?
Before constructing and showing the form, I expect that you are loading the data from a database, file etc. All that information should be contained in clearly defined objects of types marked with [Serializable] attribute. This way, it is easy to serialize and deserialize it at will.
Also, it is important to take into consideration version tolerant serialization or what happens when the form / state information is changed (e.g. a field is added) and an older XML is used to restore state.
Every form has its own mechanism to store and retrieve (serialize and deserialize) data and it does this automatically. However, the following conditions are to be met in order to use this feature automatically.
- All properties which are desired to be serialized must have public get and set accesor.
- If a certain property represents custom object such as user defined class or struct, the object must be adorned with [Serializable] attribute.
- Property which is desired to be serialized must not have [DesignerSerializationVisibility] attribute set to Hidden. By default it is Visible so not specifying this attribute at all is sufficiently serves the purpose.
Consider this example:
namespace MnM.Drawing
{
[Serializable, TypeConverter(typeof(ExpandableObjectConverter))]
public class Coordinates
{
public int X { get; set; }
[DesignerSerializationVisibility(DesignerSerializationVisibility.Hidden)]
public int Y { get; set; }
public int Z { get; protected set; }
}
public class MyForm : Form
{
[Browsable(true), EditorBrowsable(EditorBrowsableState.Always)]
public Coordinates MyCoordinates { get; set; }
}
}
Now, MyForm will automatically serialize MyCoordinates object but...
Only property X will get serialized because it fits the requisite
status to qualify for auto serialization.
Property Y can not be serialized because of DesignerSerializationVisibility(DesignerSerializationVisibility.Hidden)
Property Z can not get serialized automatically because it does
not have public set accesor.
In order to serialize Y and Z, custom serialization code is required. In most cases, need of custom serialization does not arise and custom serialization can be done in many ways but its a vast topic.

Deserializing different object types from a MemoryStream efficiently

I am trying to manually deserialize objects from a MemoryStream, but the MemoryStream will have different object types in it and I need to call an appropriate method depending on the object type. For example, this is how the MemoryStream is formatted (object type then data, repeating):
[object type (uint)][object data (variable length)][object type (uint)][object data (variable length)][object type (uint)][object data (variable length)]
Given the object type, I know how much data to expect and read for that object type. The problem that I have is figuring out an efficient way to read that data.
One way would be to use a switch statement like this:
switch (objectType) {
case 0:
SomeClass.LoadFromMemoryStream(memoryStream);
break;
case 1:
SomeOtherClass.LoadFromMemoryStream(memoryStream);
break;
case 2:
EvenAnotherClass.LoadFromMemoryStream(memoryStream);
break;
...
}
Each of the different classes will obviously need their own method to load the data from the memory stream and advance the pointer however many bytes is appropriate for that object type.
This seems difficult to maintain when there are hundreds of types of objects that can be loaded this way, and somewhat inefficient if for each object in the stream, it has to iterate through a case statement with hundreds of types to determine what to call.
I suspect that I can do this with generics but don't understand how to set up the classes and methods to support this. Is a switch statement the best approach? If not, what is?
In my opinion, I will do the following.
1.Define an interface,
public interface ISerializableType
{
uint TypeIdentity { get; }
void InitFromStream(Stream stream);
void ToStream(Stream stream);
}
2.Let all the types that you need to support to implement this interface.
3.Create a factory class, register all the object types that you need to support
private IDictionary<uint, System.Type> _objectMap = new Dictionary<uint, System.Type>();
4.When to are reading information from the stream, first of all check the type identity, and then check against your container to see whether you can support it or not, if you can , then grab the type and create an new object.
5.Cast the new object to an ISerializableType, and call the InitFromStream method.
var typeIdentity = ReadIdentityFromStream(stream);
if(_objectMap.ContainsKey(typeIdentity)){
var type = _objectMap[typeIdentity];
var obj = System.Activator.CreateInstance(type);
var iSerializable = obj as ISerializableType;
if (null != iSerializable) iSerializable.InitFromStream(stream);
};
I hope it is helpful.
Regarding efficiency. Check protobuf.
And regarding the problem. You should not operate with array of objects alone when saving/loading a state. It should be a complete type, which performs all necessary steps upon deserialization on its own.
To example, deserializing this
object[] mydata;
would required from you a switch case or running a method to detect types of objects (long and hard to maintain switch), you could actually utilize attributes to tell what to do with which object, but..
Consider this
public class Level
{
public IItem[] Item {get; set;}
public IRoom[] Room {get; set;}
public IMonster[] Monster {get; set;}
...
}
public class ItemBook: IItem
{
public string Name
{
set { /* this code will be automatically executed during deserialization */ }
}
public ItemBook()
{
/* and this code */
}
}
...
Deserializing Level is more efficient.

Dynamically ignore data members from getting serialized

We have an existing WCF service which uses several DataContracts. We want to modify the serialization based on the device, so that when accessed from mobile devices, the service should serialize only some important data members(not all)
We have 2 options here
Create separate operation and data contracts for different types of
devices
Mess with the actual xml serialization and suppress creating
unnecessary elements based on the device
We don't want to go with the first option since it introduces a lot of redundant code problems in the future
Small research showed that we need to use IXmlSerializable and override the readXML() and writeXML() methods. But at the same time, I have seen somewhere that DataContract and IXmlSerializable should not be used together
Any example to mess with actual serialization is greatly appreciated .
[DataContract]
public class TokenMessage
{
string tokenValue;
string extraValue;
[DataMember]
public string Token
{
get { return tokenValue; }
set { tokenValue = value; }
}
[DataMember]
public string Extra
{
get { return extraValue; }
set { extraValue = value; }
}
}
Now when i access the service which returns a typical TokenMessage data contract, from a mobile device, i don't want the "Extra" data member to be serialized i.e. When I supply a different argument to the operation contract, it should be able to serialize some/all the data members(depending on the action)
PS: For now please ignore the device detection part. Lets assume we have an argument in the operation contract, which helps us identify the device
I'm not convinced that some variant of #Pranav Singh's answer isn't a better design, but that's not your question...
As you mentioned in a comments attributes in .NET are static by design. This means dynamically adding/removing [DataMember] isn't a good option. It is possible. There are options like using Reflection.Emit to recreate the instance with the meta data changes (see all the answers to Can attributes be added dynamically in C#?) but all of those routes are complicated.
I see two reasonable options:
1) Implement an IParameterInspector for the service. In the AfterCall() method you could inspect and alter the parameters being returned to the client before they are serialized. There is some work to use reflection to dynamically determine the parameter types and set their values, but its not complicated. This is the better design that enables reuse of the behavior across many contracts or services. Carlos Figueira's blog is the best source for WCF extension examples.
2) Use the [OnSerializing] and [OnSerialized] events. In the [DataContract] you could temporarily alter what the properties are returning during serialization. The events are actually designed to enable initialization and as such this solution is a bit of a hack. This solution is also not thread safe. But it does keep the code contained to the DataContract class and solves the problem quickly (and I think you are looking for quick).
Solution #2 mights look something like:
[DataContract]
public class TokenMessage
{
string tokenValue;
string extraValue;
bool enableExtraValue = true;
[DataMember]
public string Extra
{
get {
if (enableExtraValue)
return extraValue;
return null;
}
set { extraValue = value; }
}
[OnSerializing()]
internal void OnSerializingMethod(StreamingContext context)
{
enableExtraValue = false;
}
[OnSerialized()]
internal void OnSerializedMethod(StreamingContext context)
{
enableExtraValue = true;
}
}
Solution #2 is a quick fix (which is what I think you are looking for).
Solution #1 is the better design.
Try using IgnoreDataMemberAttribute
There is a approach, but I think this will require extra DataContract to be generated but still no need for separate operation and data contracts for different types of devices.
It can classic implementation to run-time polymorphism. I am just giving idea:
Say you have a generic DataContract like :
[DataContract]
[KnownType(typeof(Extra))]
[KnownType(typeof(Extra2))]
public class TokenMessage
{
string tokenValue;
string extraValue;
[DataMember]
public string Token
{
get { return tokenValue; }
set { tokenValue = value; }
}
}
Other device specific contracts can inherit TokenMessage as base class like:
[DataContract]
public class Extra:TokenMessage
{
[DataMember]
public string Extra
{
get ;set;
}
}
[DataContract]
public class Extra2:TokenMessage
{
[DataMember]
public string Extra2
{
get ;set;
}
}
Now at run-time as you say you know an argument in the operation contract, which helps us identify the device. Say based on device type, you can instantiate base class with derived class like:
TokenMessage tm= new Extra();
OR
TokenMessage tm= new Extra2();
So at run-time you will decide which device contract will be part of genric response.
Note: Adding KnownType will generate the separate xsd within wsdl for all known types within base class, but saves serialization for data at run-time as this should depend on actual inheritance chosen.
In your model add a property 'ShouldSerializeYOUR_PROPERTY_NAME', set it to false when you do not want the property serialized.
See more here: http://msdn.microsoft.com/en-us/library/system.windows.dependencyobject.shouldserializeproperty(v=vs.110).aspx

Serialize and Deserialize Property Based on DB Value

I have a simple class that has bool property. The 'Get' logic for this property executes a stored procedure to return back a bit field from a database.
I then serialize this class and save it to an XML field in a database. It saves the class and the bool property just fine, no problem.
The problem I seem to be having is when I deserialize this class. The class deserilizes just fine, but when the data that drives the bool field has been updated, it seems that the class only recognizes what was serialized in XML, and it not looking back to the database to get the new bool value (does not execute my procedure to get the newly update bit field).
My solution has been to add the XmlIgnoreAttribute attribute to this field so it isn't serialized to begin with. But I'm wondering if anyone noticed this and/or can help me understand the inner working of .NET XmlSerializer class.
Thanks!
[XmlIgnoreAttribute]
public bool IsUpdated
{
get
{
DataTable dtResults = mclsSQLServerTool.LoadDataTable("exec stp_RL_SEL_NameIsUpdated '" + mstrName + "'");
bool blnIsUpdated = Convert.ToBoolean(dtResults.Rows[0]["RU_bitIsUpdated"]);
return blnIsUpdated;
}
}
The first thing to note here is that [XmlIgnore] is redundant; XmlSerializer is simply not interested in get-only properties (except for lists), because it knows it can't deserialize them. For example:
public class SomeType
{
public string Foo { get; set; }
public string Bar { get { Console.WriteLine("get_Bar"); return "abc"; } }
static void Main()
{
var ser = new XmlSerializer(typeof (SomeType));
ser.Serialize(Console.Out, new SomeType { Foo = "def" });
}
}
outputs (minus the namespaces aliases etc):
<SomeType>
<Foo>def</Foo>
</SomeType>
(note that Bar was not called)
For deserialization, the process (for simple values, not lists) is simple: as values are found in the the incoming xml stream, resolve them to members, and assign them - i.e. a xml-deserializer is basically a glorified switch statement based on incoming xml nodes.
It will never randomly call a "set" unless the data is in the incoming xml (and the property is read/write); and when it does, it expects to assign a value.
The interesting thing in your scenario is that your "get" doesn't assign the value anywhere - there is no cache. So actually, it doesn't matter that XmlSerializer doesn't touch it - every time you access IsUpdated it will do the query. Personally I suspect that is a mistake, and could lead to aggressive and unpredictable data querying.
Many serializers support the concept of serialization callbacks, which would allow you to perform some code at the end of serialization; however, XmlSerializer does not support this. So that isn't an option.
It isn't very clear what you want to achieve, but I'd just call a method at some point.
IMHO this is a mis-use of properties. Properties should have little or no code behind them. If this code was ever used in a client-server application you could potentially be making database calls from the client. I would recommend changing this to a method call. If you want to serialize the results then store the results of the "Convert.ToBoolean" in a property. Now it is a bit clearer as to what the property value is.
Something like this...
public bool IsUpdated { get; private set; }
public bool IsDataUpdated()
{
DataTable dtResults = mclsSQLServerTool.LoadDataTable("exec stp_RL_SEL_NameIsUpdated '" + mstrName + "'");
IsUpdated = Convert.ToBoolean(dtResults.Rows[0]["RU_bitIsUpdated"]);
return IsUpdated;
}

Find out whether property setter is called in DeSerialization process

Is there a way to find out whether an object property is called as part of the DeSerialization process (e.g. by the XmlSerializationReaderXXX).
Background: A typical scenario is to disable events and complex operations in that case, until the initialization is complete.
One approach I have found, is to "interpret" the stack and look up whether the call is triggered by XmlSerializationReaderXXX, which is not so elegant IMHO. Is there anything better?
public SomeClass SomeProperty
{
get { ..... }
set
{
this._somePropertyValue = value;
this.DoSomeMoreStuff(); // Do not do this during DeSerialization
}
}
-- Update --
As Salvatore has mentioned, somehow similar to How do you find out when you've been loaded via XML Serialization?
I have a possible solution.
public class xxx
{
private int myValue;
[XmlElement("MyProperty")]
public int MyPropertyForSerialization
{
get { return this.myValue; }
set
{
Console.WriteLine("DESERIALIZED");
this.myValue = value;
}
}
[XmlIgnore]
public int MyProperty
{
get { return this.myValue; }
set
{
Console.WriteLine("NORMAL");
this.myValue = value;
}
}
}
class Program
{
static void Main(string[] args)
{
xxx instance = new xxx();
instance.MyProperty = 100; // This should print "NORMAL"
// We serialize
var serializer = new XmlSerializer(typeof(xxx));
var memoryStream = new MemoryStream();
serializer.Serialize(memoryStream, instance);
// Let's print our XML so we understand what's going on.
memoryStream.Position = 0;
var reader = new StreamReader(memoryStream);
Console.WriteLine(reader.ReadToEnd());
// Now we deserialize
memoryStream.Position = 0;
var deserialized = serializer.Deserialize(memoryStream) as xxx; // This should print DESERIALIZED
Console.ReadLine();
}
}
The trick is using the XmlIgnore, it will force the xml serializer to ignore our property, then we use XmlElement to rename the property for serialization with the name of the property we want.
The problem with this technique is that you have to expose a public property for serialization, and is in some way bad because it can virtually be called by everyone.
It will not work if the member is private, unfortunally.
It works, is not totally clean, but is thread safe and don't rely on any flag.
Another possibility is to use something like the Memento pattern.
Using the same trick you can add a property called for example Memento that returns another object that contains properties suitable only for serialization, it can makes things a little cleaner.
Did you think instead of changing approach and using DataContractSerializer? It is much more powerful and produces pure XML. It supports the OnDeserializationCallback mechanism.
Since you got a pretty complex scenario you might want to consider creating a "data core" class which will be actually serialized/deserialized using simple direct way. Then your complex object is constructed from that object and you fire all events/operations as normal. It will make sequence of deserialize -> fire events/operations more explicit and easier to understand.
There's an OnDeserializingAttribute/OnDeserializedAttribute attributes pair. You can set isDeserializing flag while object is being deserialized. I don't know if they play well with XML serialization, though.
For XML Serialization solution could be implementing IXmlSerializable and embedding such logic into the ReadXml()/WriteXml() method
To have finer control of the deserialization process you could implement IXmlSerializable interface for SomeClass - in ReadXML you can then for example have some field set a flag that you are in deserialization... this flag can then be checked in the respective methods... and on completion it needs to be reset.
Another option (though not for XML IIRC) is to implement the above via OnDeserializingAttribute and OnDeserializedAttribute .
I misunderstood the question at first, but you want to ask from within setter if you are called during deserialization. To do that, use a static flag:
[serializable]
class SomeClass
{
public static IsSerializing = false;
SomeProperty
{
set
{
if(IsSerializing) DoYouStuff();
}
}
}
and then set the flag just before the serialization:
try
{
SomeClass.IsSerializing = true;
deserializedClass = (SomeClass)serializer.Deserialize(reader);
}
finaly
{
SomeClass.IsSerializing = false; //make absolutely sure you set it back to false
}
Note that same approach can work even if you deserialize a class that contains a member of your class...
Set a breakpoint on the property, and run in debug mode. It will break at the point of access for the getter/setter that you set the breakpoint on.

Categories

Resources