I am looking for a serializer that will match my requirements,
the serializer can be in the .Net framework, an Open-Sorce or a pay-for product (as long as it can b used directly from the code).
now, my requirements are these:
mandatory
Able to handle a cyclic reference.
Automatic, usues either attribute or inheritance in the target class, and then simply writes to the file.
Positive filtering, meaning that in the target class the fields are marked as what to serialize, and not what not to serialize (like [DataMember] in DataContractSerializer and not like [XmlIgnore] in XmlSerializer).
Must use a default constructor.
Supports polymorphism (no things like 'KnownTypes' in DataContractSerializer).
preferable
Generates file as light-wight as possible.
Serialize as fast as possible.
Works on non-public fields.
I checked most of the .Net serializers and tryied to find more online, and came out short,
all of wiche either not supports cyclic reference, polymorphism, or dose not use any constructor.
so right now i'm prettey out of ideas, and i will be glad for some halp.
Thank you!
The closest in the BCL is BinaryFormatter but it is not interoperable.
I would look at Google's Protocol Buffers They are available for a wide range of languages C++, Java, Python and .NET C#.
The problem withe BinaryFormatter is that it is negative filtering (marking the fildes not to serialze) and that it does not use a constractor.
about google Protocol Buffers (or ProtoBuff) i had a chance to work with it and its very complicated and can hardly be refered as automatic
Related
Based on my understanding, SerializableAttribute provides no compile time checks, as it's all done at runtime. If that's the case, then why is it required for classes to be marked as serializable?
Couldn't the serializer just try to serialize an object and then fail? Isn't that what it does right now? When something is marked, it tries and fails. Wouldn't it be better if you had to mark things as unserializable rather than serializable? That way you wouldn't have the problem of libraries not marking things as serializable?
As I understand it, the idea behind the SerializableAttribute is to create an opt-in system for binary serialization.
Keep in mind that, unlike XML serialization, which uses public properties, binary serialization grabs all the private fields by default.
Not only this could include operating system structures and private data that is not supposed to be exposed, but deserializing it could result in corrupt state that can crash an application (silly example: a handle for a file open in a different computer).
This is only a requirement for BinaryFormatter (and the SOAP equivalent, but nobody uses that). Diego is right; there are good reasons for this in terms of what it does, but it is far from the only option - indeed, personally I only recommend BinaryFormatter for talking between AppDomains - it is not (IMO) a good way to persist data (to disk, in cache, to a database BLOB, etc).
If this behaviour causes you trouble, consider using any of the alternatives:
XmlSerializer, which works on public members (not just the fields), but demands a public parameterless constructor and public type
DataContractSerializer, which can work fully opt-in (using [DataContract]/[DataMember]), but which can also (in 3.5 and above) work against the fields instead
Also - for a 3rd-party option (me being the 3rd party); protobuf-net may have options here; "v2" (not fully released yet, but available as source) allows the model (which members to serialize, etc) to be described independently of the type, so that it can be applied to types that you don't control. And unlike BinaryFormatter the output is version-tolerant, known public format, etc.
I'm trying to create content via a small C# desktop app, and have it appear inside a Silverlight application. (I'm creating plain, ordinary C# objects, and trying to make them easily persist.) The context is a game of some sort, where I have a desktop tool that lets me create and edit the content I want, and then the Silverlight binaries consume it.
How can I serialize something in (desktop) C# and deserialize it in Silverlight?
I have a small library I created for serialization; it uses Mike Talbot's amazing serializer for Silverlight, and a simple BinaryFormatter for desktop. Within each platform, these are OK; but across platforms, these two are obviously incompatible.
Is it possible to do this? I would not like to revert back to manually serializing by saving data as text and then parsing it, and I would not like to use an embedded database if possible. I may have lists of lists and other complex data, and manually parsing it is too painful.
If it's not possible, what alternatives do I have?
Edit: ProtoBuf .NET looks OK, but as I mentioned in Marc's comment, I'm using the serializer inside my own library. This means that requiring users of my persistence library to add attribution to classes to serialize them will break encapsulation. I don't want to do that.
What do I mean by breaking encapsulation?
The target user of my library (Persistent Storage) is a game developer. They will use the library to persist information within their games.
Hence, they only consume PersistentStorage.dll. Internally, Persistent Storage uses a serializer (currently, Mike Talbot's for Silverlight, and a simple Binary one for non-Silverlight) to persist data.
For me to say "to use my library, put [ProtoContract] or [Serializable] on all your classes" breaks encapsulation. It means the user knows about the internals of my library usage, which they shouldn't. I can change serializers tomorrow, and they shouldn't care.
I am aware that as a work-around, I can ask them to attribute everything with [PersistMe] and have that as a plain empty attribute that, in turn, extends whatever attribute my serializer needs. But I'm hoping that other serializers, like Mike Talbot's, will not require any attribution to use.
You can try to use Silverlight Serializer
From the author's page:
Serializing Classes Between .NET 4 and Silverlight
You may want to use SilverlightSerializer to share objects between Silverlight and .NET 4, perhaps across a WCF link.
The vital thing to do in these circumstances is to define the classes you want to share in a Silverlight assembly that only references System, System.Core and mscorlib. These are the requirements for assemblies that can be used in both types of project. If you define your classes in this way then they can be deserialized on the back end without a problem, reference anything else and it won’t work. Fortunately most of what you will need is included in those system assemblies!
You need to use the same format in this scenario. Since BinaryFormatter isn't OK for Silverlight, that is out. Personally I'd use protobuf-net, which works on both and can be configured to work on vanilla objects (but is easier if you can add attributes), but if your linked serialiser works on desktop that is a viable option too.
With an example of your model I can be more specific.
Why not try old school xml serialization with the XmlSerializer, both the .Net framework on the desktop and silverlight should have that class. This way there is no addition library to include, its in the framework.
You could also look at Sharp serializer. It allows you to either use xml based formatting or binary formatting for serialization.
I have used it in a similar scenario to share data between a Silverlight and a non Silverlight application and it works beautifully.
Just for reference:
you can build a single SilverlightSerializer DLL and reference it in .net and Silverlight. this works even though the DLL targets Silverlight
Silverlight doesn't run with enough security permissions to enable the inspection of private class members. SilverlightSerializer let's you write support classes to serialize third party components with non-standard requirements, and this can work for private members, but in that particular case it's manual and requires that the serialization class and the serialized class are one in the same.
Have you tried a JSON Serializer like JSON.net ( http://json.codeplex.com/ )?
JSON Specification: http://json.org
We use WCF to do all our serialization to the Silverlight client. We have a dll shared between the client and the server that has all the data transfer objects and interfaces. This allows us to not use the wsdl to generate service ref in silverlight.
To do the searlization we use the DataContractSerializer with a BinaryMessageEncoding. Also you do have to watch out for private setters of objects (which cant be done in silverlight, as you cant set a property with a private setter in the partial trust enviroment of silverlight). If you want to use generics and other things like that, use the NetDataContractSerializer, but that will break compatability with Java and other standards based web services outside of .net (but should work fine for silverlight).
All our DTO's are POCO, other then we add a [ItemKey] Attribute to one of the properties (no other attributes or interfaces), so that our system knows which property is the primary key (this isnt required but it makes things easier to do updates in the persistance layer if things change in the objects).
We have been using BinarySerialization with our C# app, but the size and complexity of the classes which need to be serialized results in sloooooow (de)serialization, and large files.
We suspect that we should just write our own custom serializers; but protobuf-net claims significant speed and size advantages over standard .Net binary serialization, and may be easier to add to our app than a large number of bespoke serializers.
Before spending significant time and effort trying to get it to work for us, I would love to know whether there are any deal-breakers. We are using properties defined with interfaces, generic lists of abstract sub-classes, custom bit flag enums, etc etc etc. What would stop protobuf-net working for us?
protobuf-net does what it can to adhere to the core protobuf spec, and then some (for example, it includes inheritance), however:
v1 is not very good at interface-based properties (i.e. ICustomer etc); I'm working on getting this improved in v2
v1 likes there to be a parameterless constructor (this requirement is lifted in v2)
you need to tell it how to map the model to fields; in v1 this needs to be decorated on the type (or there is an option to infer some things from the names etc); in v2 this can be done externally
in v1, flags enums are a pain; in v2 there is an option to pass-thru enums as raw integers, making it much more suitable for falgs
abstracts and inheritance are fine, but you must be able to determine all the concrete types ahead of time (to map them to integer keys)
generics should be fine
jagged arrays / nested lists without intermediate types aren't OK - you can shim this by introducing an intermediate type in the middle
not all core types have inbuilt support (the new date/time offset types, for example); in "v2" you can introduce your own shims for this if necessary
it is a tree serializer, not a graph serializer; I have some thoughts there, but nothing implemented yet
If there is some limited example of what you want to serialize, I'll happily take a look to see if it is likely to work (I'm the author).
It's not appropriate when you have to interact with existing software / an existing standard. For example, you can't use it to communicate with an SMTP server.
Please read this here on a blog about protobuf-net, to quote
What’s the catch?
In the most part, that’s it. WCF will use protobuf-net for any suitable
objects (data-contracts etc). Note that this is a coarser brush than the
per-operation control, though (you could always split the interface into
different endpoints, of course).
Also, protobuf-net does have some subtle differences (especially regarding empty
objects), so run your unit tests etc.
Note that it only works on the full-fat WCF; it won’t help Silverlight etc, since
it lacks the extension features – but that isn’t new here.
Finally, the resolver in WCF is a pain, and AFAIK wants the full assembly details
including version number; so one more thing to maintain when you get new versions.
If anyone knows how to get around this?
What's the difference between the two? Why would you use one over the other?
Found here: http://aaron-powell.spaces.live.com/blog/cns!91A824220E2BF369!150.entry
DataContractJsonSerializer
The primary purpose of the DataContractJsonSerializer is to be used with WCF, since one serialization is a big focus of WCF. Also, it is also better equipped to handle complex classes which have only certain properties available for serialization.
This class is more strongly typed, has more knowledge about the type(s) it's handling and better error handling for badly-formed JSON.
JavaScriptSerializer
This class on the other hand is much better equipped for quick serialization, it's a more cowboy approach. There's less error checking and less control over what properties which are serialized.
Update
As the above link is dead, here is another link: http://kb.cnblogs.com/a/1454030.
Personally, I'd look at Json.NET - this has the advantage of being .NET 2.0 compatible
The JavaScriptSerializer is marked as obsolete in framework 3.5. You shouldn't use it for that reason. However, back to your question. The JavaScriptSerializer doesn't require classes to be marked as [Serializable] or as [DataContract] but the DataContractJsonSerializer does. If you have compiled classes that you can't mark with attributes, you may want to use the older JSON serializer.
I am heavily using byte array to transfer objects, primitive data, over the network and back. I adapt java's approach, by having a type implement ISerializable, which contains two methods, as part of the interface, ReadObjectData and WriteObjectData. Any class using this interface, would write date into the byte array. Something Like that
class SerializationType:ISerializable
{
void ReadObjectData (/*Type that manages the write/reads into the byte array*/){}
void WriteObjectData(/*Type that manages the write/reads into the byte array*/){}
}
After write is complete for all object, I send an array of the network.
This is actually two-fold question. Is it a right way to send data over the network for the most efficiency (in terms of speed, size)?
Would you use this approach to write objects into the file, as opposed to use typically xml serialization?
Edit #1
Joel Coehoorn mentioned BinaryFormatter. I have never used this class. Would you elaborate, provide good example, references, recommendations, current practices -- in addition to what I currently see on msdn?
This should be fine, but you're doing work that is already done for you. Look at the System.Runtime.Serialization.Formatters.Binary.BinaryFormatter class.
Rather than needing to implement your own Read/WriteOjbectData() methods for each specific type you can just use this class that can already handle most any object. It basically takes an exact copy of the memory representation of almost any .Net object and writes it to or reads it from a stream:
BinaryFormatter bf = new BinaryFormatter();
bf.Serialize(outputStream, objectToSerialize);
objectToDeserialize = bf.Deserialize(inputStream) as DeserializedType;
Make sure you read through the linked documents: there can be issues with unicode strings, and an exact memory representation isn't always appropriate (things like open Sockets, for example).
If you are after simple, lightweight and efficient binary serialization, consider protobuf-net; based on google's protocol buffers format, but implemented from scratch for typical .NET usage. In particular, it can be used either standalone (via protobuf-net's Serializer), or via BinaryFormatter by implementing ISerializable (and delegating to Serializer).
Apart from being efficient, this format is designed to be extensible and portable (i.e. compatible with java/php/C++ "protocol buffers" implementations), unlike BinaryFormatter that is both implementation-specific and version-intolerant. And it means you don't have to mess around writing any serialization code...
Creating your own ISerializable interface when there's already one in the framework sounds like a bit of a recipe for disaster. At least give it a different name.
You'll have a bit of a problem when it comes to reading - you won't have an instance to call the method on. You might want to make it a sort of "factory" instead:
public interface ISerializationFactory<T>
{
T ReadObjectData(Stream input);
void WriteObjectData(Stream output);
}
As for XML vs binary... it entirely depends on the situation: how much data will there be, do you need backwards and forwards compatibility, does the XML serialization in .NET give you enough control already etc.
Yes this will be faster than sending XML as you will be sending less data over the wire. Even if you compressed the XML (which would drastically reduce its size) you would still have the overhead of compression and decompression. So I would say that between what you are currently doing and XML serialization you are currently using the most efficient solution.
However I am curious as to how much of a performance hit you would incur by using XML instead of a marshaled object. The reason that I would encourage you to look into XML serialization is because you will be storing the data in an application-neutral format that is also human readable. If you are able to serialize the data to XML in a way that does not incur performance penalties in your application I would recommend that you look into it.
Regarding writing to file, generally you want to serialize an object to XML if you want to be able to read the serialization or perhaps alter it. If you have no desire for the serialization to be human readable, you might as well reuse your binary serialization.
If you do want it to be human readable, then XML is something to consider, but it depends on the type of data you need to serialize. XML is inherently recursive and is therefore good for serializing likewise recursive data. It's less of a good fit on other types of data.
In other words, pick a persistent serialization that suits your needs. There's no one-way-fits-all solution here.
As for network, generally you'll want to keep size to a minimum, so XML is usually never a good choice due to its verbosity.
Serialization (in Java) is deceptively simple. As long as you do simple stuff (like never change the class) it is easy - but there are a number of "fun" things with it too.
Foe a good discussion on Java serialization look at Effective Java (specifically chapter 10).
For C#, not sure, but likely the core issues are the same.
There is an example here on C# serialization: http://www.codeproject.com/KB/cs/objserial.aspx.
XStream library provide an exceptionally good way of dealing with serialisation including support for XML, JSON and supporting custom converters. Specifically, the use of custom converters allowed us to reduce XML verbosity and to serialise strictly what is needed.
XStream has no requirement to declare everything as Serializable, which is very important when one utilises a third-party lib and needs to serialise an instance of a class from that lib, which is not declared as Serializable.
The answer is already accepted, but for the sake of completeness of this discussion here is a link to a good comparison between different serialisation approaches/libraries:
http://code.google.com/p/thrift-protobuf-compare/wiki/Benchmarking
The kryo library looks very compelling for Java serialisation. Similarly to XStream is supports custom converters.