I'm currently logging (Logging application) to a CouchDB database with Serilog, and with a handful of Types being decomposed into the database.
I've got a separate application (Reporting application) that is trying to pull LogEvents out of the database and deserialize them into the original LogEvents. The Reporting application is just as aware of the same types as the logging application and the specific Types in the database are fully decomposed into it.
Json.Net's deserializer has problems with deserializing the MessageTemplate. Even with a custom converter, it has so many problems that I'm probably doing it wrong (various exceptions deserializing, but no real pattern that I can tell).
Has anyone been able to do this successfully? I was under the impression that being able to pull Types out of the logs is one of the features of Serilog, and all the data is there, so I don't see why it's not possible.
These Types are all fully serializable as well, they're regularly serialized/deserialized by Json.net.
After more research, I've found a way to partially solve the problem. Generate new classes with http://json2csharp.com/ - rename the RootObject to something (e.g., SpecificLogEvent) and use:
var logEvent = JsonConvert.DeserializeObject<SpecificLogEvent>(doc.Value);
Then convert the objects to the real objects where needed.
I'll not mark this as the answer for awhile, because I'd love an easy back and forth and avoid this extra step which creates redundant classes.
Related
I have a complex model I want to persist to Mongo DB. This model has some circular reference I don't want to break.
Apparently I can't do that with the official Mongo C# driver.
Json.Net can serialize properly in Bson, and handle my model without any problem.
Can I change the mongo driver serializer ? Or do you have any other suggestion to handle this case (other than just avoid circular reference or ignore property) ?
Thank you.
Customized serialization/deserialization is a good solution, but it does not have to be painful. The simplest way may be for your model to use the ISupportInitialize interface, and implement the BeginInit() and EndInit() methods.
Sergio Costa has a good example of using this interface to deal with circular references on a web page titled Serializing circular references in MongoDb.
If you can't break the connections, there are a number of options for doing custom serialization.
I'd start here: http://docs.mongodb.org/ecosystem/tutorial/serialize-documents-with-the-csharp-driver/
In particular, the section entitled "Write a Custom Serializer." You'll need to decide whether the potential extra complexity is worth it.
Or, you could consider converting the objects to json, then to a BsonDocument.
BsonDocument document = BsonDocument.Parse(json);
If you do need this extensively, just note that you are doing a reasonably significant amount of busy work with the Object->JSON->BsonDocument->Bson path. It's something to consider if you need high throughput or are concerned about CPU/memory utilization.
I've seen couple of examples with serializable attribute like this:
[Serializable()]
public class sampleClass
{
public int Property1{ get; set; }
public string Proerty2{ get; set; }
public sampleClass() { }
public sampleClass(int pr1, int pr2)
{
pr1 = pr1;
Name = pr2.ToString();
}
}
I never had a good grasp on understanding of how this works, but from msdn:
Serialization allows the developer to save the state of an object
and recreate it as needed, providing storage of objects as well as
data exchange. Through serialization, a developer can perform actions
like sending the object to a remote application by means of a Web
Service, passing an object from one domain to another, passing an
object through a firewall as an XML string, or maintaining security or
user-specific information across applications.
But the problem is that in my code example I see no use for it. Just an object that is used to retrieve data from the database, nothing special. What are some other uses on when to use and when not to use serialization.
For example, should I always use serializzation because it is more secure? is it goin to be slower this way?
Update: Thanks for all nice answers
Serialization is useful any time you want to move a representation of your data into or out of your process boundary.
Saving an object to disk is a trivial example you'll see in many tutorials.
More commonly, serialization is used to transfer data to and from a web service, or to persist data to or from a database.
Several answers have covered the reasons of why you might want to use serialization in general. You seem to also want to know why a specific class has attribute [Serializable] and you are wondering why that may have been done.
With ASP.NET the default Session state storage is InProc which allows you to store any object as a reference and leave it on the heap. This is the best performing way to store session state, however, it only works if you are using a single worker thread or if all your session state could be rebuilt automatically if the worker thread were to change (unlikely). For the other state storage modes (StateServer and SQL Server) all the session state objects must be serializable as the ASP.NET engine will first serialize these objects using binary serialization before sending them to the storage medium.
In your case, you may be using InProc. One reason though to still mark all classes that are used in session state as Serializable and test them that way is that you may have a need to change this in the future (for example, to use a Web Farm). If you do not design your session state classes with this in mind it will be quite difficult to do the migration in the future.
Also, just because you can remove the Serializable attribute and the program "works" in one environment does not mean that it will work in another environment. For example, it may work fine for you under Visual Studio test web server (which always uses InProc session state mode) instance and even in a development IIS instance but then, perhaps a production IIS instance is setup to use a different storage mode.
These environmental/configuration differences are not necessarily limited to ASP.NET applications. There are other application engines that may do this or even standalone applications that do (it is not difficult to build this kind of configurable environment).
Finally, you may be working with a library which may be consumed by different applications. Some may need to store state in a serializable manner and others may not.
Because of these factors it is often a very good idea, at least when building a library, to consider marking simple value classes or state management classes with [Serializable]. Keep in mind that this increases the work for testing these classes and there are limits to what can be serialized (i.e. a class that contains a socket reference or open file reference may not be a good candidate for serialization as open external resources cannot be serialized) so do not overuse this.
You asked if using [Serializable] will be slower. No, it will not be. This attribute has no direct affect on performance. However, if the application environment is changed to serialize the object, then yes, performance will be affected. It is the act of serializing and deserializing that is slower than just storing the object on the heap. [Note that some routines could be written to look for the Serializable attribute and then choose to serialize but this is rare; usually it is like ASP.NET and left up to an administrator or user to decide if they want to change the store medium.]
The MSDN quote you provide explains when serialization is useful: for transporting or storing data. Writing to a file is serialization, and serialization is required t send an object over a network.
If you are just populating the object in a single application, perhaps from a database, then indeed: serialization is not a concern at all. Imaging a class for serialization has no impact on security or performance: if you don't need it, don't worry about it.
Note also that [Serializable] mainly relates to BinaryFormatter, but there are actually many more serializers than that. For example: you might want to expose your object via JSON or XML - both of those require serialization, but neither requires [Serializable].
Simple example: Imagine you have a custom shape to store application settings.
namespace My.Namespace
{
[Serializable]
public class Settings
{
public string Setting1 { get; set; }
public string Setting2 { get; set; }
}
}
You could then have a file an xml file as such:
<?xml version="1.0" encoding="utf-8" ?>
<Settings>
<Setting1>Foo</Setting1>
<Setting2>Bar</Setting2>
</Settings>
Using XmlSerializer you could simply serialize and deserialize your settings.
It is also necessary for your shape to be Serializable if you wish to stuff it into ASP.NET ViewState
These are very basic examples but demonstrate it's usefulness
What are some other uses on when to use and when not to use serialization.
Let me give you one practical example. In one of my application, I was given XML schemas (XSD files) for request and response XML files. I need to parse the request XML file, process and save the information back into several tables. Later I need to prepare response XML file accordingly and send it back to our client.
I used Xsd2Code to generate C# classes based on the schema. So parsing the request XML file is simply deserializing it to the generated request class object. Then I can access properties from the object the way it appears in request XML file. While generating response XML file is simply serializing from the generated response class object which I populate in my code. This way I can work with C# objects rather than XML files. I hope it makes sense.
For example, should I always use serializzation because it is more secure
I don't think this is related to security in any way.
I'm trying to create content via a small C# desktop app, and have it appear inside a Silverlight application. (I'm creating plain, ordinary C# objects, and trying to make them easily persist.) The context is a game of some sort, where I have a desktop tool that lets me create and edit the content I want, and then the Silverlight binaries consume it.
How can I serialize something in (desktop) C# and deserialize it in Silverlight?
I have a small library I created for serialization; it uses Mike Talbot's amazing serializer for Silverlight, and a simple BinaryFormatter for desktop. Within each platform, these are OK; but across platforms, these two are obviously incompatible.
Is it possible to do this? I would not like to revert back to manually serializing by saving data as text and then parsing it, and I would not like to use an embedded database if possible. I may have lists of lists and other complex data, and manually parsing it is too painful.
If it's not possible, what alternatives do I have?
Edit: ProtoBuf .NET looks OK, but as I mentioned in Marc's comment, I'm using the serializer inside my own library. This means that requiring users of my persistence library to add attribution to classes to serialize them will break encapsulation. I don't want to do that.
What do I mean by breaking encapsulation?
The target user of my library (Persistent Storage) is a game developer. They will use the library to persist information within their games.
Hence, they only consume PersistentStorage.dll. Internally, Persistent Storage uses a serializer (currently, Mike Talbot's for Silverlight, and a simple Binary one for non-Silverlight) to persist data.
For me to say "to use my library, put [ProtoContract] or [Serializable] on all your classes" breaks encapsulation. It means the user knows about the internals of my library usage, which they shouldn't. I can change serializers tomorrow, and they shouldn't care.
I am aware that as a work-around, I can ask them to attribute everything with [PersistMe] and have that as a plain empty attribute that, in turn, extends whatever attribute my serializer needs. But I'm hoping that other serializers, like Mike Talbot's, will not require any attribution to use.
You can try to use Silverlight Serializer
From the author's page:
Serializing Classes Between .NET 4 and Silverlight
You may want to use SilverlightSerializer to share objects between Silverlight and .NET 4, perhaps across a WCF link.
The vital thing to do in these circumstances is to define the classes you want to share in a Silverlight assembly that only references System, System.Core and mscorlib. These are the requirements for assemblies that can be used in both types of project. If you define your classes in this way then they can be deserialized on the back end without a problem, reference anything else and it won’t work. Fortunately most of what you will need is included in those system assemblies!
You need to use the same format in this scenario. Since BinaryFormatter isn't OK for Silverlight, that is out. Personally I'd use protobuf-net, which works on both and can be configured to work on vanilla objects (but is easier if you can add attributes), but if your linked serialiser works on desktop that is a viable option too.
With an example of your model I can be more specific.
Why not try old school xml serialization with the XmlSerializer, both the .Net framework on the desktop and silverlight should have that class. This way there is no addition library to include, its in the framework.
You could also look at Sharp serializer. It allows you to either use xml based formatting or binary formatting for serialization.
I have used it in a similar scenario to share data between a Silverlight and a non Silverlight application and it works beautifully.
Just for reference:
you can build a single SilverlightSerializer DLL and reference it in .net and Silverlight. this works even though the DLL targets Silverlight
Silverlight doesn't run with enough security permissions to enable the inspection of private class members. SilverlightSerializer let's you write support classes to serialize third party components with non-standard requirements, and this can work for private members, but in that particular case it's manual and requires that the serialization class and the serialized class are one in the same.
Have you tried a JSON Serializer like JSON.net ( http://json.codeplex.com/ )?
JSON Specification: http://json.org
We use WCF to do all our serialization to the Silverlight client. We have a dll shared between the client and the server that has all the data transfer objects and interfaces. This allows us to not use the wsdl to generate service ref in silverlight.
To do the searlization we use the DataContractSerializer with a BinaryMessageEncoding. Also you do have to watch out for private setters of objects (which cant be done in silverlight, as you cant set a property with a private setter in the partial trust enviroment of silverlight). If you want to use generics and other things like that, use the NetDataContractSerializer, but that will break compatability with Java and other standards based web services outside of .net (but should work fine for silverlight).
All our DTO's are POCO, other then we add a [ItemKey] Attribute to one of the properties (no other attributes or interfaces), so that our system knows which property is the primary key (this isnt required but it makes things easier to do updates in the persistance layer if things change in the objects).
.NET provides the JavaScriptSerializer class in the
System.Web.Script.Serialization namespace. (provided in System.Web.Extensions.dll)
It was originally intended to support AJAX web server apps, but the class can be used by any application (client, server, hybrid, anything) that serializes and deserializes .NET classes to JSON. I have a desktop app that captures screenshots and uploads to Facebook, and uses this class to deserialize the response.
would I ever want to look elsewhere for JSON deserialization from within .NET?
If so, why? and where would I Look?
If not, then why does JSON.Net exist? Is it strictly for historical purposes? (ie, because it was created by the community before the JavaScriptSerializer).
In my case there are various reasons that prevent me to use JavaScriptSerializer. Here are some of them.
1) Ugly deserialization when dealing with anonymous types
While the usage is fairly straight forward for serialization:
JavaScriptSerializer serializer = new JavaScriptSerializer();
String json = serializer.Serialize(data);
For deserialization however, there is a minor annoyance in that the deserializer accepts a generic type along with the content:
serializer.Deserialize<T>(String s)
this can be a problem if the type T is not known at compile time and needs to be dynamic. The work around is a bit ugly as I learnt because it uses reflection to create a generic method (but it works)
var result = typeof(JavaScriptSerializer).GetMethod("Deserialize")
.MakeGenericMethod(JsonDataType)
.Invoke(serializer, new object[] { inputContent });
Please note: according to Dave Ward comment on this answer there's a DeserializeObject() that can be used to prevent this.
2) Cannot handle circular references
I have seen this using Entity Framework, Linq to SQL, NHibernate, NetTiers and even when using Castle's proxy.
According to MS Connect the circular reference exception will be raised when a navigable relation is double-sided (can access both sides of the relation), so the first thing to do is disable one side of the relation. The exception will also be thrown when you use 1:1 relations (or 1:0..1 or any relation causing the creation of an EntityReference type property), in this case the exception will be of type System.Data.Metadata.Edm.AssociationType.
The solution to this is to make the serializer ignore the properties of type EntityReference, using an empty implementation of a class deriving from JavaScriptConverter and registering it using the RegisterConverters method of the JavaScriptSerializer object.
3) Useful features that leads to less testable code
A useful feature of the JavaScriptSerializer is that you can also implement a custom JavaScriptConverter and pass that in to JavaScriptSerializer for fine-grained control over the serialization/deserialization. However, for it to be really useful you need to know the types at compile time and have references to those types. This really limits the usefulness of this feature because by referencing those classes your code becomes tightly coupled so you cannot easily use it in something like an MVC filter.
For these reasons I have often ended up using Json.NET.
Hope this helps!
I use the JavaScriptSerializer on a wide variety of scenarios, it never let me down, and never needed to look elsewhere for other solutions... :)
...but i do know that JSON.net has some added values like LINQ to JSON, which i never needed, and nice JSON formatting but as Serializing goes JavaScriptSerializer does the work fine.
The answer to just about every single question about using C# with JSON seems to be "use JSON.NET", but that's not the answer I'm looking for.
The reason I say that is, from everything I've been able to read in the documentation, JSON.NET is basically just a better performing version of the DataContractSerializer built into the .NET framework...
Which means if I want to deserialize a JSON string, I have to define the full, strongly-typed class for EVERY request I might have. So if I have a need to get categories, posts, authors, tags, etc., I have to define a new class for every one of these things.
This is fine if I built the client and know exactly what the fields are, but I'm using someone else's API, so I have no idea what the contract is unless I download a sample response string and create the class manually from the JSON string.
Is that the only way it's done? Is there not a way to have it create a kind of hashtable that can be read with json["propertyname"]?
Finally, if I do have to build the classes myself, what happens when the API changes and they don't tell me (as twitter seems to be notorious for doing)? I'm guessing my entire project will break until I go in and update the object properties...
So what exactly is the general workflow when working with JSON? And by general I mean library-agnostic. I want to know how it's done in general, not specifically to a target library...
It is very hard to be library-agnostic as you request because how you work with json really depends on the library you use. As an example inside JSON.NET there are multiple ways you could work with JSON. There is the method you talk about with direct serialization into objects. That is type safe but will break if the data from your API changes. However, there is also a LINQ-to-JSON that provides a JObject (which behaves fairly similarly to XElement) that provides a way to do JObject["key"] as you requested in your question. If you are really just looking for a flexible way to work with JSON inside C#, then check out JSON.NET's LINQ-to-JSON.
In reality no matter how you do it, if the API changes your code is likely to break. Even if you are just strictly a hashtable-based approach, your code will still be likely to break if the data coming back changes.
Edit
JSON.NET Documentation
Examples
If you check out the examples, the second one should give you a good example of how LINQ-to-JSON works. It allows you to work with it without defining any classes. Everything gets converted to standard framework classes (mostly collections and strings). This avoids the need to maintain classes.
I've been a Perl developer for over a decade, and I've just recently started to work in C#. I'm surprised by how much I like it (I don't like Java at all) but one of the most difficult cognitive switches is going from "Everything can be treated as a string and the language takes care of conversions" to "Pre-define your types." In this case string-thinking might be an advantage, because it's what you need to do for the kind of API you're asking for.
You need to write a JSON parser that understands the syntax, which is fairly simple: comma-separated lists, key/value pairs, {} for hashes/objects, [] for arrays, and quoting/escaping constructs. You'll want to create a Hashtable to start because the top-level entity in JSON is always an object, then scan the JSON string character-by-character. Pull out key/value pairs; if the value starts with { then add it as a new Hashtable, if it starts with [ add it as a new ArrayList, otherwise add it as a string. If you get { or [ you'll need to recursively descend to add the child data elements.
If .NET has a good recursive descent parser, you could probably use that to make the job simpler or more robust, but JSON is simple enough to make this a good and reasonably completable exercise.