I have a legacy app that uses SoapFormatter to persist a graph of objects (maybe 50 different classes). I want to move away from using this as it is deprecated, and increasingly hard to continue to support deserializing from old files as the classes change.
I want to use DataContractSerializer going forward. Does anyone have any suggestions as to a good strategy for migration? I need to continue to be able to deserializing old files written by SoapFormatter...
Thanks
I don't think you want to be limited to a backward-compatible format.
So you will need to distinguish old and new content. And easy method would be :
Old Format: <soapdata>
New Format: <header> <newdata>
And in your new Load() method:
(Try to) Read the header
If a header is found, continue to read the new format
else re-position to begin and use the SOAP formatter to read.
The simplest code would be to try deserialize with DataContractSerializer and fallback to SoapFormatter if it fails.
The save part will always use the DataContractSerializer, so that your new objects or the updated ones will use your new supported version.
public MyContract Deserialize(string file)
{
try
{
using (var stream = loadFile())
{
return loadWithDataContractSerializer(stream);
}
}
catch (SerializationException)
{
using (var stream = openForRead(file))
{
return convertToContract(loadWithSoapFormatter(stream));
}
}
}
private MyContract loadWithDataContractSerializer(Stream s);
private MyOldObject loadWithSoapFormatter(Stream s);
private MyContract convertToContract(MyOldObject obj);
public void Serialize(string file, MyContract data)
{
using (var stream = openForWrite(file))
{
writeWithDataContractSerializer(stream, data);
}
}
Of course, it might be possible to implement a custom logic to allow DataContractSerializer to understant the SoapFormatter structure, but you will have to provide a lot more work.
Related
I searched for an answer to this question but I didn't see many specific answers to it. My question is relatively simple: given data serialized in the protobuf format, is it possible to deserialize it without knowing the data type?
It seems that I can serialize any type without knowing what it is by doing:
public static byte[] SerializeData<T>(T arg)
{
try
{
using (var testStream = new MemoryStream())
{
Serializer.Serialize(testStream, arg);
....
Is there a way to do this such that Serializer.Deserialize(stream) (or something of that sort)?
I don't like using MemoryStream objects inbetween stream interfaces. They are awkward, requiring you to re-seek to the start, and will also peak memory usage in demanding situations.
Sometimes a utility will only work a certain way. Perhaps it will output byte[]s, or write to a stream, or is a stream in a pipeline that you read from, pulling the data through.
This Newtonsoft JSON serializer is a utility which can only write to a stream.
var js = new Newtonsoft.Json.JsonSerializer();
var sw = new StreamWriter(ps);
js.Serialize(sw, o);
This is a problem for me, because I want to chain:
IEnumerable
JSON serialization
GZIP compression
HTTP to client
(Network)
HTTP from Server
GZIP decompression
JSON deserialization
IEnumerable
Apart from the difficulties getting the JSON deserializer to present a nice IEnumerable interface, the rest of the parts don't provide an interface suitable for pipelining. Even the GZIP compression side is the wrong way around.
Ideally, on the server-side I would be able to do:
IEnumerable<object> o = GetData();
var js = new Newtonsoft.Json.JsonSerialization(o);
var gz = new System.IO.Compression.GZipStream(js, System.IO.Compression.CompressionMode.Compress, true);
return new FileStreamResult(gz, "application/x-gzip");
I could possibly extend the Newtonsoft project to provide a pipeline implementation, and I may do so. But until then I need a solution, and I believe one is required for other utilities (including the BCL GZipStream).
Are there any solutions which allow one to join such utilities more efficiently?
Is there a library which contains an adapter for such situations?
I am working on such a library, not expecting there to be such a library.
The answer is the brand new StreamAdaptor project:
https://bitbucket.org/merarischroeder/alivate-stream-adaptor. It still needs a bit of work - would be nice to package it as a NuGet package, but it's all there and tested.
So the interface will look a bit like this:
var data = GetData(); //Get the source data
var sa = new StreamAdaptor(); //This is what wraps the write-only utility source
sa.UpstreamSource((ps) => //ps is the dummy stream which does most of the magic
{
//This anon. function is run on a separate thread and can therefore be blocked
var sw = new StreamWriter(ps);
sw.AutoFlush = true;
var js = new Newtonsoft.Json.JsonSerializer();
js.Serialize(sw, data); //This is the main component of the implementation
sw.Flush();
});
var sa2 = new StreamAdaptor();
sa2.UpstreamSource((ps) =>
{
using (var gz = new System.IO.Compression.GZipStream(ps, System.IO.Compression.CompressionMode.Compress, true))
sa.CopyTo(gz);
});
The reverse process is easier with natural support for a read-through pipeline
System.IO.Compression.GZipStream sw = new System.IO.Compression.GZipStream(sa2, System.IO.Compression.CompressionMode.Decompress);
var jsonTextReader = new JsonTextReader(new StreamReader(sw));
return TestA.Deserialize(jsonTextReader);
I also demonstrate there a workaround to the IEnumerable<> deserializing issue. It requires you to create your own deserializer leveraging JsonTextReader, but it works well.
The serializer supports IEnumerable natively. The GetData function above, sets up the data source for the serializer using IEnumerable functions (among other things):
public static IEnumerable<TestB> GetTestBs()
{
for (int i = 0; i < 2; i++)
{
var b = new TestB();
b.A = "A";
b.B = "B";
b.C = TestB.GetCs();
yield return b;
}
}
It's Deserialisation which requires a workaround. Keep in mind that IEnumerable<> properties need to be listed all at the end of the JSON stream/objects, because enumeration is deferred, yet JSON deserialization is linear.
The Deserialization entry point:
public static TestA Deserialize(JsonTextReader reader)
{
TestA a = new TestA();
reader.Read();
reader.Read();
if (!reader.Value.Equals("TestBs"))
throw new Exception("Expected property 'TestBs' first");
reader.Read(); //Start array
a.TestBs = DeserializeTestBs(reader); //IEnumerable property last
return a;
}
One of the IEnumerable deserializer functions:
static IEnumerable<TestB> DeserializeTestBs(JsonTextReader reader)
{
while (reader.Read())
{
if (reader.TokenType == JsonToken.EndArray)
break;
yield return TestB.Deserialize(reader);
}
reader.Read(); //End of object
}
This can of course be achieved with trial and error, although built-in support in JSON.NET is desirable.
BinaryFormatter is able to handle serialization simply:
private byte[] TokenToBytes(SessionSecurityToken token)
{
if (token == null)
{
return null;
}
using (var memoryStream = new MemoryStream())
{
var binaryFormatter = new BinaryFormatter();
binaryFormatter.Serialize(memoryStream, token);
return memoryStream.ToArray();
}
}
When I tried replacing BinaryFormatter with protobuf-net:
using (var memoryStream = new MemoryStream())
{
Serializer.Serialize(memoryStream, token);
return memoryStream.ToArray();
}
I get the following exception:
Type is not expected, and no contract can be inferred:
System.IdentityModel.Tokens.SessionSecurityToken
I tried adding:
RuntimeTypeModel.Default.Add(typeof(SessionSecurityToken), true);
Which gets past the exception but I now get a zero byte array.
How do I properly configure protobuf-net to serialize a SessionSecurityToken?
On the flipside, SessionSecurityToken does not have a parameterless constructor.
using (var memoryStream = new MemoryStream(tokenAsBytes))
{
return Serializer.Deserialize<SessionSecurityToken>(memoryStream);
}
Throws a ProtoException:
No parameterless constructor found for SessionSecurityToken
BinaryFormatter is able to do it without any fuss:
using (var memoryStream = new MemoryStream(bytes))
{
var binaryFormatter = new BinaryFormatter();
return (SessionSecurityToken)binaryFormatter.Deserialize(memoryStream);
}
How do I properly configure protobuf-net to deserialize a SessionSecurityToken?
protobuf-net does not claim to be able to serialize every single type; indeed, you would have great difficulty serializing that via most serializers (XmlSerializer, any of the json serializers, DataContractSerializer, etc). BinaryFormatter is in a different category of serializers - and in this particular case, implements custom serialization via ISerializable.GetObjectData(SerializationInfo, StreamingContext).
The constructor thing is a red herring; actually, protobuf-net can bypass constructors completely, and in this particular scenario BinaryFormatter is using a custom serialization constructor via .ctor(SerializationInfo, StreamingContext).
For simple cases, protobuf-net can be configured via attributes or runtime options; for more complex scenarios, surrogates can be used to map between representations - however, in this case I would suggest (looking at the implementation of SessionSecurityToken) that this is more complex than you probably want to maintain.
I would step back a step or two here; most serializers are designed to work with data, not implementation - and work great with DTOs etc. SessionSecurityToken is very much not a DTO, and there is no simple way of switching between them. My strong suggestion here would be: serialize what this represents, not what it is. However, if this is part of an existing complex model and is really really hard to separate out, you could switch back to BinaryFormatter for those bits. I haven't tested this, but consider:
RuntimeTypeModel.Default.Add(typeof(SessionSecurityToken), false)
.SetSurrogate(typeof(BinaryFormatterSurrogate<SessionSecurityToken>));
With:
[ProtoContract]
public class BinaryFormatterSurrogate<T>
{
[ProtoMember(1)]
public byte[] Raw { get; set; }
public static explicit operator T(BinaryFormatterSurrogate<T> value)
{
if(value==null || value.Raw == null) return default(T);
using(var ms = new MemoryStream(value.Raw))
{
return (T)new BinaryFormatter().Deserialize(ms);
}
}
public static explicit operator BinaryFormatterSurrogate<T>(T value)
{
object obj = value;
if (obj == null) return null;
using (var ms = new MemoryStream())
{
new BinaryFormatter().Serialize(ms, obj);
return new BinaryFormatterSurrogate<T> { Raw = ms.ToArray() };
}
}
}
Keep in mind that this simply embeds the output of one serializer as raw data inside another. Fortunately protobuf-net is happy talking binary, so this won't add any noticeable overhead (just the header and length-prefix for the blob) - but it also won't do anything particularly smart or clever with the SessionSecurityToken instances. If this is the only thing you are serializing, it really isn't worth it. If this is just one ugly bump in a larger DTO model, where most of it can serialize nicely - then it might get the job done for you.
I need a semi-shallow copy of an object. Under my original design I used MemberwiseClone to catch all the simple stuff and then I specifically copied the classes to the extent that they needed to be copied. (Some of them are inherently static and most of the rest are containers holding static items.) I didn't like the long list of copies but there's no way around that.
Now, however, I find myself needing to create a descendent object--do I now have to go back and copy all those fields that previously I was copying with MemberwiseClone?
Or am I missing some better workaround for this?
The easiest way to clone, I find, is to use serialization. This obviously only works with classes that are [Serializable] or that implement ISerializable.
Here is a general generic extension you can use to make any serializable class' objects cloneable:
public static T Clone<T>(this T source)
{
if (source == default(T))
{
return default(T);
} else {
IFormatter formatter = new BinaryFormatter();
Stream ms = new MemoryStream();
using (ms)
{
formatter.Serialize(ms, source);
stream.Seek(0, SeekOrigin.Begin);
return (T) formatter.Deserialize(ms);
}
}
}
What libraries are available to handle JSON in .Net? I've seen this: http://james.newtonking.com/projects/json-net.aspx but would prefer a native library, if possible.
I have been using the JavaScriptSerializer some to expose data structures from a WCF service to Ajax calls, and it has been working out quite well.
The JavaScriptSerializer has been marked as obsolete in .NET 3.5 and but you could use the DataContractJsonSerializer.
EDIT: See this question on SO about whether JavaScriptSerializer is actually obsolete going forward in the .NET BCL. It looks like JavaScriptSerializer is no longer obsolete in .NET 3.5 SP1 - so it's probably fine to use that. If in doubt, you can use the contract serializer from WCF, or JSON.NET (if you're willing to include 3rd party code).
Here's some wrapper code to make using DataContractJsonSerializer nicer.
using System.Runtime.Serialization;
using System.Runtime.Serialization.Json;
public class JSONHelper
{
public static string Serialize<T>(T obj)
{
DataContractJsonSerializer serializer =
new DataContractJsonSerializer(obj.GetType());
using( MemoryStream ms = new MemoryStream() )
{
serializer.WriteObject(ms, obj);
string retVal = Encoding.Default.GetString(ms.ToArray());
return retVal;
}
}
public static T Deserialize<T>(string json)
{
using( MemoryStream ms = new MemoryStream(Encoding.Unicode.GetBytes(json)) )
{
DataContractJsonSerializer serializer =
new DataContractJsonSerializer(typeof(T));
T obj = (T)serializer.ReadObject(ms);
ms.Close();
return obj;
}
}
}
The code above is courtesy of: http://pietschsoft.com/post/2008/02/NET-35-JSON-Serialization-using-the-DataContractJsonSerializer.aspx,
I have altered it from it's original form to use best practices for object disposal (the using pattern .NET supports).
If you can require .Net 3.5, use System.Web.Script.Serialization.JavaScriptSerializer