So my concern here is when I'm targeting AOT platforms so no support for dynamic code generation. protobuf-net allows to compile the serializers before hand solving the problem. But I'm not quite sure how to use this features...
So I create a RuntimeTypeModel with all my types and members and then call model.Compile(name, path); - OK... what to do with this generated dll? include it and use its serialize methods and forget about my model object? or can I just serialize from the same model object? (i.e. if I do model.Compile if I then say model.Seiralize, will the model use the serializers in the compiled dll? or...?)
Any enlightenment on the subject would be great. I found bits and bytes on the Internets but no complete answer.
This depends a bit on:
whether you need cross platform support
whether you need custom configuration
It sounds like "no" and "yes" in that order, from the question. In which case you should be able to use the overload of Compile that accepts a path and serializer type name. This emits a DLL that you can reference from your project; do that, and then simply use the serializer type:
TypeModel serializer = new YourCustomSerializer();
This then had the methods to serialize / deserialize etc.
If you need cross-platform support, the tool to compile is more complicated, and requires using a special build step. The simplest way to do this is via the "protogen" tool, which uses attribute-based configuration. If you need cross-platform compilation and custom configuration, you need to write a small tool (not big) that references the ikvm protobuf-net; this takes a bit more explaining - let me know if you need an example!
Related
Trying to get my mind around google protobuf. I found some implementation of protobuf in C# but they seems to lack one feature: the ability to generate .proto files automatically from an existing C# class decorated with attributes.
The reason I want to do it this way instead of going from auto-generated C# classes from .proto file is because I already have the C# classes defined in my project and I don't want to duplicate them just to satisfy ProtoBuf.
Does anyone have encountered such a scenario?
Update
Is this possible to just decorate a C# class and not use a .proto file to use protobuf?
Good news; what you have described (having existing C# classes) is the expected use-case of protobuf-net. All the .proto stuff ("protogen", the VS add-in, etc) were all added as afterthoughts. The core of protobuf-net doesn't know about them or care about them.
protocol buffers defines a DSL (.proto, as you mention) that is shared between implementations, and is (sometimes) used for code generation. When I first wrote protobuf-net, the code-generation aspect wasn't my biggest concern - simply that .NET developers are generally guilty (myself included) of "implementation first" rather than "contract first".
As a consequence, protobuf-net doesn't need .proto files to work; an attributed class is sufficient to unambiguously serialize/deserialize. Just use Serializer.Serialize , .Merge and .Deserialize (etc).
That said; it does include some very under-developed and experimental support for this:
string proto = Serializer.GetProto<YourType>();
This is far from complete, but may work for simple types. If you have some specific cases where it fails, then let me know (add a comment or log an issue). However; most of the time, people interested in .proto would write the .proto first and work from there.
Examples of working decorated types are shown on the project home page; it is entirely up to you whether you use WCF attributes, xml attributes or protobuf-net attributes (although the latter provide more control over some specific serialization points, such as inheritance and numeric layouts).
Before Skeet Marc runs in here and gets massive ups, let me point out protobuf.net.
It seems like the documentation around Roslyn is a bit lacking?
I am not able to find good comprehensive documentation.
What I am trying to do essentially is copy the public surface of an existing API (.dll)
into a new assembly (need to create source code .cs files!) and at the same time make a variety of tranformations to the resulting code (think making wrapper classes).
Would really appreciate any help in how I can use Rolsyn to load the initial SyntaxTree from an existing assembly and how to do those basic tranforms (for example exclude internal classes etc)
In the current Roslyn CTP there is a Roslyn.Services.MetadataAsSource namespace which can be used to convert an type's public interface to source code. This is what we implement the F12 "metadata as source" feature with. Now, it generates only a shell of source code which won't actually compile, so you'd have to use further APIs to munge the syntax tree into what you want. Alternatively, you could use the Roslyn.Services.CodeGeneration namespace to generate source from these symbols automatically. I should warn the MetadataAsSource namespace may go away in future versions of the API.
You can import symbols from metadata by creating an otherwise empty compilation with the metadata references you care about added, and then from that compilation browsing the type hierarchy from GlobalNamespace property, or calling Compilation.GetReferencedAssemblySymbol() and then digging through that. This is actually far better than using reflection, since it'll properly express the symbol model from the "C# perspective" instead of the "CLR perspective" -- reflection won't give you information for uses of dynamic, some default parameter values, etc.
It seems like the documentation around Roslyn is a bit lacking? I am not able to find good comprehensive documentation.
Roslyn is at the Community Technology Preview stage, so it's not surprising that its documentation is lacking. You can find some sources at Roslyn API documentation.
What I am trying to do essentially is copy the public surface of an existing API (.dll) into a new assembly (need to create source code .cs files!) and at the same time make a variety of transformations to the resulting code (think making wrapper classes).
Working with assemblies this way is not something Roslyn can do. But it seems for what you want, reflection for reading the assembly combined with Roslyn for writing the new code would work. But you would need to write all the code to translate from the reflection model to Roslyn's model (e.g. Type → TypeDeclarationSyntax, MethodInfo → MethodDeclarationSyntax, etc.).
I'm trying to create content via a small C# desktop app, and have it appear inside a Silverlight application. (I'm creating plain, ordinary C# objects, and trying to make them easily persist.) The context is a game of some sort, where I have a desktop tool that lets me create and edit the content I want, and then the Silverlight binaries consume it.
How can I serialize something in (desktop) C# and deserialize it in Silverlight?
I have a small library I created for serialization; it uses Mike Talbot's amazing serializer for Silverlight, and a simple BinaryFormatter for desktop. Within each platform, these are OK; but across platforms, these two are obviously incompatible.
Is it possible to do this? I would not like to revert back to manually serializing by saving data as text and then parsing it, and I would not like to use an embedded database if possible. I may have lists of lists and other complex data, and manually parsing it is too painful.
If it's not possible, what alternatives do I have?
Edit: ProtoBuf .NET looks OK, but as I mentioned in Marc's comment, I'm using the serializer inside my own library. This means that requiring users of my persistence library to add attribution to classes to serialize them will break encapsulation. I don't want to do that.
What do I mean by breaking encapsulation?
The target user of my library (Persistent Storage) is a game developer. They will use the library to persist information within their games.
Hence, they only consume PersistentStorage.dll. Internally, Persistent Storage uses a serializer (currently, Mike Talbot's for Silverlight, and a simple Binary one for non-Silverlight) to persist data.
For me to say "to use my library, put [ProtoContract] or [Serializable] on all your classes" breaks encapsulation. It means the user knows about the internals of my library usage, which they shouldn't. I can change serializers tomorrow, and they shouldn't care.
I am aware that as a work-around, I can ask them to attribute everything with [PersistMe] and have that as a plain empty attribute that, in turn, extends whatever attribute my serializer needs. But I'm hoping that other serializers, like Mike Talbot's, will not require any attribution to use.
You can try to use Silverlight Serializer
From the author's page:
Serializing Classes Between .NET 4 and Silverlight
You may want to use SilverlightSerializer to share objects between Silverlight and .NET 4, perhaps across a WCF link.
The vital thing to do in these circumstances is to define the classes you want to share in a Silverlight assembly that only references System, System.Core and mscorlib. These are the requirements for assemblies that can be used in both types of project. If you define your classes in this way then they can be deserialized on the back end without a problem, reference anything else and it won’t work. Fortunately most of what you will need is included in those system assemblies!
You need to use the same format in this scenario. Since BinaryFormatter isn't OK for Silverlight, that is out. Personally I'd use protobuf-net, which works on both and can be configured to work on vanilla objects (but is easier if you can add attributes), but if your linked serialiser works on desktop that is a viable option too.
With an example of your model I can be more specific.
Why not try old school xml serialization with the XmlSerializer, both the .Net framework on the desktop and silverlight should have that class. This way there is no addition library to include, its in the framework.
You could also look at Sharp serializer. It allows you to either use xml based formatting or binary formatting for serialization.
I have used it in a similar scenario to share data between a Silverlight and a non Silverlight application and it works beautifully.
Just for reference:
you can build a single SilverlightSerializer DLL and reference it in .net and Silverlight. this works even though the DLL targets Silverlight
Silverlight doesn't run with enough security permissions to enable the inspection of private class members. SilverlightSerializer let's you write support classes to serialize third party components with non-standard requirements, and this can work for private members, but in that particular case it's manual and requires that the serialization class and the serialized class are one in the same.
Have you tried a JSON Serializer like JSON.net ( http://json.codeplex.com/ )?
JSON Specification: http://json.org
We use WCF to do all our serialization to the Silverlight client. We have a dll shared between the client and the server that has all the data transfer objects and interfaces. This allows us to not use the wsdl to generate service ref in silverlight.
To do the searlization we use the DataContractSerializer with a BinaryMessageEncoding. Also you do have to watch out for private setters of objects (which cant be done in silverlight, as you cant set a property with a private setter in the partial trust enviroment of silverlight). If you want to use generics and other things like that, use the NetDataContractSerializer, but that will break compatability with Java and other standards based web services outside of .net (but should work fine for silverlight).
All our DTO's are POCO, other then we add a [ItemKey] Attribute to one of the properties (no other attributes or interfaces), so that our system knows which property is the primary key (this isnt required but it makes things easier to do updates in the persistance layer if things change in the objects).
Trying to get my mind around google protobuf. I found some implementation of protobuf in C# but they seems to lack one feature: the ability to generate .proto files automatically from an existing C# class decorated with attributes.
The reason I want to do it this way instead of going from auto-generated C# classes from .proto file is because I already have the C# classes defined in my project and I don't want to duplicate them just to satisfy ProtoBuf.
Does anyone have encountered such a scenario?
Update
Is this possible to just decorate a C# class and not use a .proto file to use protobuf?
Good news; what you have described (having existing C# classes) is the expected use-case of protobuf-net. All the .proto stuff ("protogen", the VS add-in, etc) were all added as afterthoughts. The core of protobuf-net doesn't know about them or care about them.
protocol buffers defines a DSL (.proto, as you mention) that is shared between implementations, and is (sometimes) used for code generation. When I first wrote protobuf-net, the code-generation aspect wasn't my biggest concern - simply that .NET developers are generally guilty (myself included) of "implementation first" rather than "contract first".
As a consequence, protobuf-net doesn't need .proto files to work; an attributed class is sufficient to unambiguously serialize/deserialize. Just use Serializer.Serialize , .Merge and .Deserialize (etc).
That said; it does include some very under-developed and experimental support for this:
string proto = Serializer.GetProto<YourType>();
This is far from complete, but may work for simple types. If you have some specific cases where it fails, then let me know (add a comment or log an issue). However; most of the time, people interested in .proto would write the .proto first and work from there.
Examples of working decorated types are shown on the project home page; it is entirely up to you whether you use WCF attributes, xml attributes or protobuf-net attributes (although the latter provide more control over some specific serialization points, such as inheritance and numeric layouts).
Before Skeet Marc runs in here and gets massive ups, let me point out protobuf.net.
I am looking for a serializer that will match my requirements,
the serializer can be in the .Net framework, an Open-Sorce or a pay-for product (as long as it can b used directly from the code).
now, my requirements are these:
mandatory
Able to handle a cyclic reference.
Automatic, usues either attribute or inheritance in the target class, and then simply writes to the file.
Positive filtering, meaning that in the target class the fields are marked as what to serialize, and not what not to serialize (like [DataMember] in DataContractSerializer and not like [XmlIgnore] in XmlSerializer).
Must use a default constructor.
Supports polymorphism (no things like 'KnownTypes' in DataContractSerializer).
preferable
Generates file as light-wight as possible.
Serialize as fast as possible.
Works on non-public fields.
I checked most of the .Net serializers and tryied to find more online, and came out short,
all of wiche either not supports cyclic reference, polymorphism, or dose not use any constructor.
so right now i'm prettey out of ideas, and i will be glad for some halp.
Thank you!
The closest in the BCL is BinaryFormatter but it is not interoperable.
I would look at Google's Protocol Buffers They are available for a wide range of languages C++, Java, Python and .NET C#.
The problem withe BinaryFormatter is that it is negative filtering (marking the fildes not to serialze) and that it does not use a constractor.
about google Protocol Buffers (or ProtoBuff) i had a chance to work with it and its very complicated and can hardly be refered as automatic