I have a complex model I want to persist to Mongo DB. This model has some circular reference I don't want to break.
Apparently I can't do that with the official Mongo C# driver.
Json.Net can serialize properly in Bson, and handle my model without any problem.
Can I change the mongo driver serializer ? Or do you have any other suggestion to handle this case (other than just avoid circular reference or ignore property) ?
Thank you.
Customized serialization/deserialization is a good solution, but it does not have to be painful. The simplest way may be for your model to use the ISupportInitialize interface, and implement the BeginInit() and EndInit() methods.
Sergio Costa has a good example of using this interface to deal with circular references on a web page titled Serializing circular references in MongoDb.
If you can't break the connections, there are a number of options for doing custom serialization.
I'd start here: http://docs.mongodb.org/ecosystem/tutorial/serialize-documents-with-the-csharp-driver/
In particular, the section entitled "Write a Custom Serializer." You'll need to decide whether the potential extra complexity is worth it.
Or, you could consider converting the objects to json, then to a BsonDocument.
BsonDocument document = BsonDocument.Parse(json);
If you do need this extensively, just note that you are doing a reasonably significant amount of busy work with the Object->JSON->BsonDocument->Bson path. It's something to consider if you need high throughput or are concerned about CPU/memory utilization.
Related
I'm currently logging (Logging application) to a CouchDB database with Serilog, and with a handful of Types being decomposed into the database.
I've got a separate application (Reporting application) that is trying to pull LogEvents out of the database and deserialize them into the original LogEvents. The Reporting application is just as aware of the same types as the logging application and the specific Types in the database are fully decomposed into it.
Json.Net's deserializer has problems with deserializing the MessageTemplate. Even with a custom converter, it has so many problems that I'm probably doing it wrong (various exceptions deserializing, but no real pattern that I can tell).
Has anyone been able to do this successfully? I was under the impression that being able to pull Types out of the logs is one of the features of Serilog, and all the data is there, so I don't see why it's not possible.
These Types are all fully serializable as well, they're regularly serialized/deserialized by Json.net.
After more research, I've found a way to partially solve the problem. Generate new classes with http://json2csharp.com/ - rename the RootObject to something (e.g., SpecificLogEvent) and use:
var logEvent = JsonConvert.DeserializeObject<SpecificLogEvent>(doc.Value);
Then convert the objects to the real objects where needed.
I'll not mark this as the answer for awhile, because I'd love an easy back and forth and avoid this extra step which creates redundant classes.
I have long held the believe that your domain model should not be responsible for serializing itself to XML. I have used the IXmlSerializable interface in the past to control how my objects are serialized but ideally I'd prefer the nuts and bolts of the serialization to live outside the object.
However I've never been able to actually implement this in a clean manner and I was wondering if there was any patterns I was overlooking to make this happen. Basically I want my object model to do it's thing and be oblivious to XML serialization (or any other serialization for that matter) and then handed off to some service that spiders the object and serializes it.
I've tried doing this with extension methods but this falls short when you want to serialize a collection of type object. I've looked at doing it with object wrappers and DTO's that then serialize but then you've got the overhead of maintaining another set of objects and having to create these objects when you want to serialize which again can get messy when you have collections of type object.
The only other thing is using reflection but I'd worry about the processing overheads.
Is there a sane way to do what I'm asking or should I just bite the bullet and make my objects xml aware?
Using the System.Xml.Serialization Attributes is putting the nuts and bolts outside of your code. You are defining metadata and with the exception of optional parameters, no extra code is required. Implementing IXmlSerializable and doing the serialization by hand is error prone and should be avoided. Why? You are defining your data 3 times.
XML Schema
Class
Serialization code
Using attributes, you can scrub step 3.
XML and C# has an impedance mismatch. Like it or not, at some point, you will need to define the serialization to create the right document model.
Arguably, the classes you are serializing should not be performing any work. They are just a data store. Try abstracting your logic away from serialized objects - it may give you a warmer feeling.
Update
If you really, really hate attributes, try using the adapter pattern to serialize your model. The XML code will be in a separate class or assembly and you can work with your model across storage mediums. You will suffer the consequence of having to update the serialization separately when you update your model.
We are building application that stores objects to isolated storage using .NET runtime serialization.
Problems occur when we update application by adding some new properties to the classes of objects we are serializing. So we want to do some kind of versioning of the objects in isolated storage so we can check if they are obsolete before they are deserialized.
Any advice and ideas how to do this on best possible way?
What do you think about custom formatter implementing IFormatter interface and can it help instead of vesioning objects?
I wrote about this issue on MS forum more detailed here.
You COULD have a serialization in the serialization. First a wrapper class telling the version, and holding the inner true class.
This however feels a bit bad smelly..
Here are a few options (at in any particular order).
Name the file based on the version
Place the file in a directory based on a version
Create a wrapper object that contains metadata about each serialized object such as the version number.
Add a property to each object that contains the persisting application's version number
If its binary serialization, you could read the bytes directly, and determine the assembly version from this. Byte number 22 onwards contains information on the assembly and object type, so you could write something that would read this, and then determine if your objects are obsolete.
Marc Gravell was propose in comment great idea to use version-tolerant serializer.
It enables enough control of deserialization for us even to make obsolete objects reusable.
More on msdn
Thanks to all for suggestions.
The answer to just about every single question about using C# with JSON seems to be "use JSON.NET", but that's not the answer I'm looking for.
The reason I say that is, from everything I've been able to read in the documentation, JSON.NET is basically just a better performing version of the DataContractSerializer built into the .NET framework...
Which means if I want to deserialize a JSON string, I have to define the full, strongly-typed class for EVERY request I might have. So if I have a need to get categories, posts, authors, tags, etc., I have to define a new class for every one of these things.
This is fine if I built the client and know exactly what the fields are, but I'm using someone else's API, so I have no idea what the contract is unless I download a sample response string and create the class manually from the JSON string.
Is that the only way it's done? Is there not a way to have it create a kind of hashtable that can be read with json["propertyname"]?
Finally, if I do have to build the classes myself, what happens when the API changes and they don't tell me (as twitter seems to be notorious for doing)? I'm guessing my entire project will break until I go in and update the object properties...
So what exactly is the general workflow when working with JSON? And by general I mean library-agnostic. I want to know how it's done in general, not specifically to a target library...
It is very hard to be library-agnostic as you request because how you work with json really depends on the library you use. As an example inside JSON.NET there are multiple ways you could work with JSON. There is the method you talk about with direct serialization into objects. That is type safe but will break if the data from your API changes. However, there is also a LINQ-to-JSON that provides a JObject (which behaves fairly similarly to XElement) that provides a way to do JObject["key"] as you requested in your question. If you are really just looking for a flexible way to work with JSON inside C#, then check out JSON.NET's LINQ-to-JSON.
In reality no matter how you do it, if the API changes your code is likely to break. Even if you are just strictly a hashtable-based approach, your code will still be likely to break if the data coming back changes.
Edit
JSON.NET Documentation
Examples
If you check out the examples, the second one should give you a good example of how LINQ-to-JSON works. It allows you to work with it without defining any classes. Everything gets converted to standard framework classes (mostly collections and strings). This avoids the need to maintain classes.
I've been a Perl developer for over a decade, and I've just recently started to work in C#. I'm surprised by how much I like it (I don't like Java at all) but one of the most difficult cognitive switches is going from "Everything can be treated as a string and the language takes care of conversions" to "Pre-define your types." In this case string-thinking might be an advantage, because it's what you need to do for the kind of API you're asking for.
You need to write a JSON parser that understands the syntax, which is fairly simple: comma-separated lists, key/value pairs, {} for hashes/objects, [] for arrays, and quoting/escaping constructs. You'll want to create a Hashtable to start because the top-level entity in JSON is always an object, then scan the JSON string character-by-character. Pull out key/value pairs; if the value starts with { then add it as a new Hashtable, if it starts with [ add it as a new ArrayList, otherwise add it as a string. If you get { or [ you'll need to recursively descend to add the child data elements.
If .NET has a good recursive descent parser, you could probably use that to make the job simpler or more robust, but JSON is simple enough to make this a good and reasonably completable exercise.
I am heavily using byte array to transfer objects, primitive data, over the network and back. I adapt java's approach, by having a type implement ISerializable, which contains two methods, as part of the interface, ReadObjectData and WriteObjectData. Any class using this interface, would write date into the byte array. Something Like that
class SerializationType:ISerializable
{
void ReadObjectData (/*Type that manages the write/reads into the byte array*/){}
void WriteObjectData(/*Type that manages the write/reads into the byte array*/){}
}
After write is complete for all object, I send an array of the network.
This is actually two-fold question. Is it a right way to send data over the network for the most efficiency (in terms of speed, size)?
Would you use this approach to write objects into the file, as opposed to use typically xml serialization?
Edit #1
Joel Coehoorn mentioned BinaryFormatter. I have never used this class. Would you elaborate, provide good example, references, recommendations, current practices -- in addition to what I currently see on msdn?
This should be fine, but you're doing work that is already done for you. Look at the System.Runtime.Serialization.Formatters.Binary.BinaryFormatter class.
Rather than needing to implement your own Read/WriteOjbectData() methods for each specific type you can just use this class that can already handle most any object. It basically takes an exact copy of the memory representation of almost any .Net object and writes it to or reads it from a stream:
BinaryFormatter bf = new BinaryFormatter();
bf.Serialize(outputStream, objectToSerialize);
objectToDeserialize = bf.Deserialize(inputStream) as DeserializedType;
Make sure you read through the linked documents: there can be issues with unicode strings, and an exact memory representation isn't always appropriate (things like open Sockets, for example).
If you are after simple, lightweight and efficient binary serialization, consider protobuf-net; based on google's protocol buffers format, but implemented from scratch for typical .NET usage. In particular, it can be used either standalone (via protobuf-net's Serializer), or via BinaryFormatter by implementing ISerializable (and delegating to Serializer).
Apart from being efficient, this format is designed to be extensible and portable (i.e. compatible with java/php/C++ "protocol buffers" implementations), unlike BinaryFormatter that is both implementation-specific and version-intolerant. And it means you don't have to mess around writing any serialization code...
Creating your own ISerializable interface when there's already one in the framework sounds like a bit of a recipe for disaster. At least give it a different name.
You'll have a bit of a problem when it comes to reading - you won't have an instance to call the method on. You might want to make it a sort of "factory" instead:
public interface ISerializationFactory<T>
{
T ReadObjectData(Stream input);
void WriteObjectData(Stream output);
}
As for XML vs binary... it entirely depends on the situation: how much data will there be, do you need backwards and forwards compatibility, does the XML serialization in .NET give you enough control already etc.
Yes this will be faster than sending XML as you will be sending less data over the wire. Even if you compressed the XML (which would drastically reduce its size) you would still have the overhead of compression and decompression. So I would say that between what you are currently doing and XML serialization you are currently using the most efficient solution.
However I am curious as to how much of a performance hit you would incur by using XML instead of a marshaled object. The reason that I would encourage you to look into XML serialization is because you will be storing the data in an application-neutral format that is also human readable. If you are able to serialize the data to XML in a way that does not incur performance penalties in your application I would recommend that you look into it.
Regarding writing to file, generally you want to serialize an object to XML if you want to be able to read the serialization or perhaps alter it. If you have no desire for the serialization to be human readable, you might as well reuse your binary serialization.
If you do want it to be human readable, then XML is something to consider, but it depends on the type of data you need to serialize. XML is inherently recursive and is therefore good for serializing likewise recursive data. It's less of a good fit on other types of data.
In other words, pick a persistent serialization that suits your needs. There's no one-way-fits-all solution here.
As for network, generally you'll want to keep size to a minimum, so XML is usually never a good choice due to its verbosity.
Serialization (in Java) is deceptively simple. As long as you do simple stuff (like never change the class) it is easy - but there are a number of "fun" things with it too.
Foe a good discussion on Java serialization look at Effective Java (specifically chapter 10).
For C#, not sure, but likely the core issues are the same.
There is an example here on C# serialization: http://www.codeproject.com/KB/cs/objserial.aspx.
XStream library provide an exceptionally good way of dealing with serialisation including support for XML, JSON and supporting custom converters. Specifically, the use of custom converters allowed us to reduce XML verbosity and to serialise strictly what is needed.
XStream has no requirement to declare everything as Serializable, which is very important when one utilises a third-party lib and needs to serialise an instance of a class from that lib, which is not declared as Serializable.
The answer is already accepted, but for the sake of completeness of this discussion here is a link to a good comparison between different serialisation approaches/libraries:
http://code.google.com/p/thrift-protobuf-compare/wiki/Benchmarking
The kryo library looks very compelling for Java serialisation. Similarly to XStream is supports custom converters.