Deserialization of optional fields from BinaryFormatter - c#

I have an application that serializes data using BinaryFormatter. A member was added to the class that was serialized from one version to the next without changing the class name. Code was added to handle the possible absence of the added member in old serialized files:
private void readData(FileStream fs, SymmetricAlgorithm dataKey)
{
CryptoStream cs = null;
try
{
cs = new CryptoStream(fs, dataKey.CreateDecryptor(),
CryptoStreamMode.Read);
BinaryFormatter bf = new BinaryFormatter();
string string1 = (string)bf.Deserialize(cs);
// do stuff with string1
bool bool1 = (bool)bf.Deserialize(cs);
// do stuff with bool1
ushort ushort1 = (ushort)bf.Deserialize(cs);
// do stuff with ushort1
// etc. etc. ...
// this field was added later, so it may not be present
// in the serialized binary data. Check for it, and if
// it's not there, do some default behavior
NewStuffIncludedRecently newStuff = null;
try
{
newStuff = (NewStuffIncludedRecently)bf.Deserialize(cs);
}
catch
{
newStuff = null;
}
_newStuff = newStuff != null ?
new NewStuffIncludedRecently(newStuff) :
new NewStuffIncludedRecently();
}
catch (Exception e)
{
// ...
}
finally
{
// ...
}
}
The point I'm at now is that I'd really like to just rinse and repeat with another member I'd like to add, which would mean I'd add another field and try-catch block similar to that for NewStuffIncludedRecently.
I had thought of just making the entire class [Serializable] but wouldn't that break compatibility with the old serialized data?
My main concern is that I'm not clear how the deserialization works. If I add in handling for another optional field similarly to above, will it work? What are other options I have for handling these changes better?
Thanks in advance as always.

If you mark the new fields with [OptionalField] it should work, but I have heard reports of flakiness in some cases. I can't say for sure, since I avoid BinaryFormatter, because it has so many issues when versioning :) (plus, it isn't as "tight" as some alternatives, and has severe issues if you want to go cross-platform, or to CF/SL etc)
If you are implementing ISerializable, you might try:
foreach(SerializationEntry entry in info) {
switch(entry.Name) {
case "Name": Name = (string)info.Value;
case "Id": Id = (int)info.Value;
...
}
}
But again, must stress - this is doing things the hard way :p
With this approach, you only process the data that is actually there.

Related

C# check if deserialized object has specific field

So, I want to save object and then load it and take data from it. I made an class called SaveData, in there I have field isVibrationOn.
Working code below:
public class SaveData
{
public bool isVibratonOn;
}
Here is the code for serialization:
public void SaveGame()
{
SaveData saveData = new SaveData();
BinaryFormatter bf = new BinaryFormatter();
FileStream file = File.Create(Path.Combine(Application.persistentDataPath, FILE_NAME));
SaveData(saveData);
bf.Serialize(file, saveData);
file.Close();
}
private void SaveData(SaveData saveData)
{
saveData.isVibrationOn = VibrationController.controller.isVibrationOn;
}
And here is code for load data:
public void LoadGame()
{
if (File.Exists(Path.Combine(Application.persistentDataPath, FILE_NAME)))
{
FileStream file = File.Open(Path.Combine(Application.persistentDataPath, FILE_NAME), FileMode.Open);
if (file.Length > 0)
{
SaveData saveData = new SaveData();
BinaryFormatter bf = new BinaryFormatter();
saveData = (SaveData)bf.Deserialize(file);
LoadData(saveData);
file.Close();
}
}
}
public LoadData(SaveData saveData)
{
VibrationController.controller.isVibrationOn = saveData.isVibrationOn;
}
My question here is, when I decide to add or remove some fields (lists etc) to SaveData object, my LoadData would look different, but object that is saved on device whould have different fields as well. Simple expample
public LoadData(SaveData saveData)
{
VibrationController.controller.isVibrationOn = saveData.isVibrationOn;
//old save data doesn't have isMusicOn field
//LoadData method is different because I added new field on SaveData object after I saved file.
//This is simple example, but also it could be any changes like list of objects with an object that has different fields added/changed.
MusicController.controller.isMusicOn = saveData.isMusicOn;
}
How would I check if old instance has that field?
As a general recommendation, do not use binaryFormatter. It is slow, inefficient, unsafe and has poor backwardscompatibility.
So if you change the class I would not expect it to be possible to de serialize older data at all, let alone tell you what fields where missing. Switching .net versions can also be an issue with binaryformatter.
There are much better serialization libraries out there. Json.net is the standard for text-based serialization, and I have used protobuf.net for binary serialization. But there are many other libraries that can be used.
To handle missing or optional fields you would typically have some default value, like null, that you can check. It should also be possible to initialize the fields to some other default value if desired.
I would recommend separating your serialization objects from your domain objects, since serialization frameworks may require parameter less constructors or public setters. And separate serialization objects provide a chance to manage differences in object structures between versions.
If your goal is to make sure your code doesn't break because of "missing" field:
If you already have the "old" version rolled out, that's tough - you will have to implement some kind of "migration" from old data to new, probably by keeping the old class as-is, implementing your changes in a new class (possibly derived from old, to keep code duplication to a minimum), and then checking if the data you are deserializing is old (I'll refer to it as "MyClass_Old") or new ("MyClass_New"). If you are able to determine that from some metadata, file attributes or such - great. If not, you could just deserialize it as MyClass_New on purpose and wrap in in try-catch. If you caught a SerializationException, then it's probably MyClass_Old, and then you deserialize it as MyClass_Old and then use it to construct a new MyClass_New instance.
However, if you didn't yet roll out those changes, you can make use of version-tolerant serialization. You can use attributes like [OptionalFieldAttribute] to mark fields that might be missing in different version, [OnDeserializingAttribute] (that goes on method that will be called before deserializing - to maybe set some values in those missing fields), and [OnDeserializedAttribute] to "fix" or validate your deserialized object.
So for your example, knowing that you added the isMusicOn field, you'd mark it as optional (because it might be missing in deserialized data):
[OptionalField(VersionAdded = 2)]
bool isMusicOn;
and then set it to some kind of default value if it's missing. Let's say you want it to be on by default:
[OnDeserializing]
internal void OnDeserializingMethod(StreamingContext context)
{
isMusicOn = true;
}
Note that here you use the "deserializing" attribute, because you do want to keep the value if it is not missing. Since OnDeserializingMethod is called before deserializing, it will be overwritten by deserialized value if it is present. This specific case (you have a field that might be missing, and you want it to have some specific value) is also covered here.
If your goal is to check if this specific field was missing in deserialized object:
With some preparation this same principle (version-tolerant serialization) can also be used to specifically determine if the field was present in deserialized object. You can set it to some predetermined value (one that would not be allowed when serializing) before deserializing (using [OnDeserializing]). Then after deserialization check if that value is still there or it was replaced by something reasonable.
So in your case let's say that allowed values for isMusicOn are true and false, then you can make your isMusicOn a nullable bool:
[OptionalField(VersionAdded = 2)]
bool? isMusicOn;
Then you'll make sure that serialized object cannot have isMusicOn set to null, by using [OnSerializing]:
[OnSerializing()]
internal void OnSerializingMethod(StreamingContext context)
{
if (isMusicOn == null)
isMusicOn = false;
}
Then assign null in a method marked with [OnDeserializing]:
[OnDeserializing]
internal void OnDeserializingMethod(StreamingContext context)
{
isMusicOn = null;
}
and then in [OnDeserialized] check if it is still null (if it was deserialized, it will have changed to proper value like true or false, if not it'll stay null):
[OnDeserialized]
internal void OnDeserializedMethod(StreamingContext context)
{
Console.WriteLine($"isMusicOn {isMusicOn == null ? "wasn't" : "was" } present in deserialized object!");
if (isMusicOn == null)
isMusicOn = false;//set it to some "proper" default value;
}
In this example, actually, nullable value wouldn't even require the setup (it'll be null by default even without the OnDeserializing part), but i'll leave it in as an example.
All this last part is probably more trouble then it's worth. Besides all the hassle with attributes, you have to change your field type to allow for that "super special value", and that might require you to change a lot of other code that depends on it. I suspect the real question is not "how to check if the field was there", but rather "how to make sure my app doesn't break because it wasn't there", and for that you don't need to know if your field was deserialized - just to make sure that it has some reasonable value in it after deserialization.

Weird situation with lock field being null - c# probably .net 4.0 win8-32 specific

I have the following code:
public class Settings
{
private object _lock = new object();
public void Save() {
lock (_lock)
{
...
}
}
}
On windows 8 x86 / .net 4.0 lock throws exception:
Exception Type: System.ArgumentNullException
Exception Message: Value cannot be null.
Exception Target Site: ReliableEnter
I attached debugger and its null. _lock is really null :|
On windows 7 x64 it works all right. I am checking other OSes now.
EDIT:
It is realted to deserialization. After deserialization of Settings class _lock is null. _lock field didn't exist at the time of serilization of settings and its getting back deserialized as null. Deleting serilized object file and recreating it with lock field eliminated exception. I will check if this is correct deserialization behavior if field didin't exist at serialization. Overwriting object value initialized in declaration doesn't look cool to me. But thats pretty much answer to this. I admit that I didn't consider serialization, deserialization at the time I asked for help ;).
EDIT2:
Here is the code that ilustrates my scenario and what was happening to me:
Serilize class without _lock field
class Program
{
static void Main(string[] args)
{
Settings set = new Settings();
using (FileStream fs = new FileStream(#"C:\test\tst.set", FileMode.Create, FileAccess.Write, FileShare.None))
{
BinaryFormatter bf = new BinaryFormatter();
bf.Serialize(fs, set);
fs.Flush();
fs.Close();
}
}
}
[Serializable]
internal class Settings
{
private int _i = 12;
public void testMethod()
{
int i = 0;
i++;
Console.WriteLine(i);
}
}
Deserialize to class with lock field and call testMethod:
class Program
{
static void Main(string[] args)
{
Settings set = null;
using (FileStream fs = new FileStream(#"C:\test\tst.set", FileMode.Open, FileAccess.Read, FileShare.Read))
{
BinaryFormatter bf = new BinaryFormatter();
set = (Settings)bf.Deserialize(fs);
fs.Flush();
fs.Close();
}
set.testMethod();
}
}
[Serializable]
internal class Settings
{
private int _i = 12;
private readonly object _lock = new object();
public void testMethod()
{
lock (_lock)
{
int i = 0;
i++;
Console.WriteLine(i);
}
}
}
program will crash on testMethod call.
I didin't find anything related to this behavior in documentaion. I will check for null after deserialization and that will end my trouble.
Ok. How deserilization can assign null to my readonly field assigned on declaration?
To all intents and purposes, if you set it to a new Object and you don't change it anywhere else, then what you're seeing is impossible.
First thing I'd try is to move the new Object code into my constructor, if it's happy there, then leave it be. Either way, trace over the assignment and see if it ever got assigned.
Make sure your IDE is not watching anything that could effect the value of the variable as a side effect - check conditional breakpoints for the same thing.
Note that you can create a conditional break point on change!
If still broken add asserts, compile in debug - run outside of the IDE.
If the debug version crashes when the IDE is not part of the mix, take it to other PCs and try it.
If it works on other PCs, Virus scan/reinstall etc. Memory check etc.
The compiler could be broken, try reinstalling Visual Studio.
Then I'm out of ideas!
Private properties are not serialized by default. If you want them to be serialized you need to create your custom serializer methods.
You can look at the following question for extra details

List of Objects Serialization xml fault

I am making a WPF application in which I save list of object on exit of my WPF Application. And get the list of objects on system startup. Everything works fine initially. But Some times it gives the serialization Exception. After getting the exception I looked of the xml serialized file. But it seem to me that the exception was thrown because the xml file was not formed properly. When I corrected it. It again worked fine.
public static class IsolatedStorageCacheManager<T>
{
public static void store(T loc)
{
IsolatedStorageFile appStore = IsolatedStorageFile.GetStore(IsolatedStorageScope.User | IsolatedStorageScope.Assembly | IsolatedStorageScope.Domain, null, null);
using(IsolatedStorageFileStream fileStream=appStore.OpenFile("myFile21.xml",FileMode.OpenOrCreate))
{
DataContractSerializer serializer = new DataContractSerializer(typeof(T));
serializer.WriteObject(fileStream, loc);
}
}
public static T retrieve()
{
T obj = default(T);
IsolatedStorageFile appStore = IsolatedStorageFile.GetStore(IsolatedStorageScope.User | IsolatedStorageScope.Assembly | IsolatedStorageScope.Domain, null, null);
if (appStore.FileExists("myFile21.xml"))
{
using (IsolatedStorageFileStream fileStream = appStore.OpenFile("myFile21.xml", FileMode.Open))
{
DataContractSerializer serializer = new DataContractSerializer(typeof(T));
try
{
obj = (T)serializer.ReadObject(fileStream);
}
catch (SerializationException e)
{
Console.WriteLine(e.StackTrace);
}
}
}
return obj;
}
}
The first thing to do is make sure that the objects passed to store are of a type supported by the DataContractSerializer.
The easiest thing to do is check all store calls yourself.
You could also create a validation method or even better, see if anyone else has implemented one. This method could validate the loc object and return a boolean and be called at the beginning of the store method inside a System.Diagnostics.Debug.Assert call so that it will only run on a debug configuration. Note though that this method could be quite tricky because you will have to validate type T for all the cases mentioned in the specification of DataContractSerializer and if T is a generic validate T's parameter(s) as well.

Files,strings and save

I been having trouble trying to figure this out. When I think I have it I get told no. Here is a picture of it.
I am working on the save button. Now after the user adds the first name, last name and job title they can save it. If a user loads the file and it comes up in the listbox, that person should be able to click on the name and then hit the edit button and they should be able to edit it. I have code, but I did get inform it looked wackey and the string should have the first name, last name and job title.
It is getting me really confused as I am learning C#. I know how to use savefiledialog but I am not allowed to use it on this one. Here is what I am suppose to be doing:
When the user clicks the “Save” button, write the selected record to
the file specified in txtFilePath (absolute path not relative) without
truncating the values currently inside.
I am still working on my code since I got told that it will be better file writes records in a group of three strings. But this is the code I have right now.
private void Save_Click(object sender, EventArgs e)
{
string path = txtFilePath.Text;
if (File.Exists(path))
{
using (StreamWriter sw = File.CreateText(path))
{
foreach (Employee employee in employeeList.Items)
sw.WriteLine(employee);
}
}
else
try
{
StreamWriter sw = File.AppendText(path);
foreach (var item in employeeList.Items)
sw.WriteLine(item.ToString());
}
catch
{
MessageBox.Show("Please enter something in");
}
Now I can not use save or open file dialog. The user should be able to open any file on the C,E,F drive or where it is. I was also told it should be obj.Also the program should handle and exceptions that arise.
I know this might be a noobie question but my mind is stuck as I am still learning how to code with C#. Now I have been searching and reading. But I am not finding something to help me understand how to have all this into 1 code. If someone might be able to help or even point to a better web site I would appreciate it.
There are many, many ways to store data in a file. This code demonstrates 4 methods that are pretty easy to use. But the point is that you should probably be splitting up your data into separate pieces rather than storing them as one long string.
public class MyPublicData
{
public int id;
public string value;
}
[Serializable()]
class MyEncapsulatedData
{
private DateTime created;
private int length;
public MyEncapsulatedData(int length)
{
created = DateTime.Now;
this.length = length;
}
public DateTime ExpirationDate
{
get { return created.AddDays(length); }
}
}
class Program
{
static void Main(string[] args)
{
string testpath = System.IO.Path.Combine(
Environment.GetFolderPath(Environment.SpecialFolder.Desktop), "TestFile");
// Method 1: Automatic XML serialization
// Requires that the type being serialized and all its serializable members are public
System.Xml.Serialization.XmlSerializer xs =
new System.Xml.Serialization.XmlSerializer(typeof(MyPublicData));
MyPublicData o1 = new MyPublicData() {id = 3141, value = "a test object"};
MyEncapsulatedData o2 = new MyEncapsulatedData(7);
using (System.IO.StreamWriter w = new System.IO.StreamWriter(testpath + ".xml"))
{
xs.Serialize(w, o1);
}
// Method 2: Manual XML serialization
System.Xml.XmlWriter xw = System.Xml.XmlWriter.Create(testpath + "1.xml");
xw.WriteStartElement("MyPublicData");
xw.WriteStartAttribute("id");
xw.WriteValue(o1.id);
xw.WriteEndAttribute();
xw.WriteAttributeString("value", o1.value);
xw.WriteEndElement();
xw.Close();
// Method 3: Automatic binary serialization
// Requires that the type being serialized be marked with the "Serializable" attribute
using (System.IO.FileStream f = new System.IO.FileStream(testpath + ".bin", System.IO.FileMode.Create))
{
System.Runtime.Serialization.Formatters.Binary.BinaryFormatter bf =
new System.Runtime.Serialization.Formatters.Binary.BinaryFormatter();
bf.Serialize(f, o2);
}
// Demonstrate how automatic binary deserialization works
// and prove that it handles objects with private members
using (System.IO.FileStream f = new System.IO.FileStream(testpath + ".bin", System.IO.FileMode.Open))
{
System.Runtime.Serialization.Formatters.Binary.BinaryFormatter bf =
new System.Runtime.Serialization.Formatters.Binary.BinaryFormatter();
MyEncapsulatedData o3 = (MyEncapsulatedData)bf.Deserialize(f);
Console.WriteLine(o3.ExpirationDate.ToString());
}
// Method 4: Manual binary serialization
using (System.IO.FileStream f = new System.IO.FileStream(testpath + "1.bin", System.IO.FileMode.Create))
{
using (System.IO.BinaryWriter w = new System.IO.BinaryWriter(f))
{
w.Write(o1.id);
w.Write(o1.value);
}
}
// Demonstrate how manual binary deserialization works
using (System.IO.FileStream f = new System.IO.FileStream(testpath + "1.bin", System.IO.FileMode.Open))
{
using (System.IO.BinaryReader r = new System.IO.BinaryReader(f))
{
MyPublicData o4 = new MyPublicData() { id = r.ReadInt32(), value = r.ReadString() };
Console.WriteLine("{0}: {1}", o4.id, o4.value);
}
}
}
}
As you are writing the employee objects with WriteLine, the underlying ToString() is being invoked. What you have to do first is to customize that ToString() methods to fit your needs, in this way:
public class Employee
{
public string FirstName;
public string LastName;
public string JobTitle;
// all other declarations here
...........
// Override ToString()
public override string ToString()
{
return string.Format("'{0}', '{1}', '{2}'", this.FirstName, this.LastName, this.JobTitle);
}
}
This way, your writing code still keeps clean and readable.
By the way, there is not a reverse equivalent of ToSTring, but to follow .Net standards, I suggest you to implement an Employee's method like:
public static Employee Parse(string)
{
// your code here, return a new Employee object
}
You have to determine a way of saving that suits your needs. A simple way to store this info could be CSV:
"Firstname1","Lastname 1", "Jobtitle1"
" Firstname2", "Lastname2","Jobtitle2 "
As you can see, data won't be truncated, since the delimiter " is used to determine string boundaries.
As shown in this question, using CsvHelper might be an option. But given this is homework and the constraints therein, you might have to create this method yourself. You could put this in Employee (or make it override ToString()) that does something along those lines:
public String GetAsCSV(String firstName, String lastName, String jobTitle)
{
return String.Format("\"{0}\",\"{1}\",\"{2}\"", firstName, lastName, jobTitle);
}
I'll leave the way how to read the data back in as an exercise to you. ;-)

Binary serialization of mutable F# record

I have used binary serialization to save an F# record from a C# class. All works fine:
F#:
type GameState =
{
LevelStatus : LevelStatus
Grid : Variable<Option<Ball> [,]>
...
}
let game_state : GameState = aGameState()
C#:
public void OnSaveGame() {
using (var stream = File.Open("game_status.sav", FileMode.Create))
{
var binary_formatter = new BinaryFormatter();
binary_formatter.Serialize(stream, PuzzleBobble.game_state);
}
}
Now, I'm refactoring my F# module and I would like to have a mutable record to serialize:
let mutable game_state = aGameState()
game_state <- myGameState()
This way the file is created but when I try to deserialize it I get a null object.
I've changed nothing of my previous implementation, except the added mutable keyword.
My question is: is there anything wrong with serialization of a mutable F# record? Or serialization it self is right, and I have to look for a different error somewhere else in my code?
EDIT:
Even accessing the record through methods like suggested #Brian, it seems not to work.
Here's some more details. When I deserialize the previous saved object this way (that works without game_state declared mutable):
public void OnLoadGame() {
using (var stream = File.Open("game_status.sav", FileMode.Open))
{
var binary_formatter = new BinaryFormatter();
try
{
GameLogic.GameState state = binary_formatter.Deserialize(stream) as GameLogic.GameState;
GameLogic.load_game_state(state);
}
catch (ArgumentNullException e) {
Console.WriteLine(e.Message);
}
}
}
I get the following exception:
'System.ArgumentNullException' in FSharp.Core.dll
I have once seen some weird bug where module-scoped mutable variables in F# libraries were not properly initialized, is it possible you're hitting that?
If you change the code to define
let getGameState() = game_state
let setGameState(x) = game_state <- x
and then use the get/set functions, instead of referring to the mutable variable directly, does the problem go away? If so, this might be an exotic compiler bug that we know about.

Categories

Resources