So, I want to save object and then load it and take data from it. I made an class called SaveData, in there I have field isVibrationOn.
Working code below:
public class SaveData
{
public bool isVibratonOn;
}
Here is the code for serialization:
public void SaveGame()
{
SaveData saveData = new SaveData();
BinaryFormatter bf = new BinaryFormatter();
FileStream file = File.Create(Path.Combine(Application.persistentDataPath, FILE_NAME));
SaveData(saveData);
bf.Serialize(file, saveData);
file.Close();
}
private void SaveData(SaveData saveData)
{
saveData.isVibrationOn = VibrationController.controller.isVibrationOn;
}
And here is code for load data:
public void LoadGame()
{
if (File.Exists(Path.Combine(Application.persistentDataPath, FILE_NAME)))
{
FileStream file = File.Open(Path.Combine(Application.persistentDataPath, FILE_NAME), FileMode.Open);
if (file.Length > 0)
{
SaveData saveData = new SaveData();
BinaryFormatter bf = new BinaryFormatter();
saveData = (SaveData)bf.Deserialize(file);
LoadData(saveData);
file.Close();
}
}
}
public LoadData(SaveData saveData)
{
VibrationController.controller.isVibrationOn = saveData.isVibrationOn;
}
My question here is, when I decide to add or remove some fields (lists etc) to SaveData object, my LoadData would look different, but object that is saved on device whould have different fields as well. Simple expample
public LoadData(SaveData saveData)
{
VibrationController.controller.isVibrationOn = saveData.isVibrationOn;
//old save data doesn't have isMusicOn field
//LoadData method is different because I added new field on SaveData object after I saved file.
//This is simple example, but also it could be any changes like list of objects with an object that has different fields added/changed.
MusicController.controller.isMusicOn = saveData.isMusicOn;
}
How would I check if old instance has that field?
As a general recommendation, do not use binaryFormatter. It is slow, inefficient, unsafe and has poor backwardscompatibility.
So if you change the class I would not expect it to be possible to de serialize older data at all, let alone tell you what fields where missing. Switching .net versions can also be an issue with binaryformatter.
There are much better serialization libraries out there. Json.net is the standard for text-based serialization, and I have used protobuf.net for binary serialization. But there are many other libraries that can be used.
To handle missing or optional fields you would typically have some default value, like null, that you can check. It should also be possible to initialize the fields to some other default value if desired.
I would recommend separating your serialization objects from your domain objects, since serialization frameworks may require parameter less constructors or public setters. And separate serialization objects provide a chance to manage differences in object structures between versions.
If your goal is to make sure your code doesn't break because of "missing" field:
If you already have the "old" version rolled out, that's tough - you will have to implement some kind of "migration" from old data to new, probably by keeping the old class as-is, implementing your changes in a new class (possibly derived from old, to keep code duplication to a minimum), and then checking if the data you are deserializing is old (I'll refer to it as "MyClass_Old") or new ("MyClass_New"). If you are able to determine that from some metadata, file attributes or such - great. If not, you could just deserialize it as MyClass_New on purpose and wrap in in try-catch. If you caught a SerializationException, then it's probably MyClass_Old, and then you deserialize it as MyClass_Old and then use it to construct a new MyClass_New instance.
However, if you didn't yet roll out those changes, you can make use of version-tolerant serialization. You can use attributes like [OptionalFieldAttribute] to mark fields that might be missing in different version, [OnDeserializingAttribute] (that goes on method that will be called before deserializing - to maybe set some values in those missing fields), and [OnDeserializedAttribute] to "fix" or validate your deserialized object.
So for your example, knowing that you added the isMusicOn field, you'd mark it as optional (because it might be missing in deserialized data):
[OptionalField(VersionAdded = 2)]
bool isMusicOn;
and then set it to some kind of default value if it's missing. Let's say you want it to be on by default:
[OnDeserializing]
internal void OnDeserializingMethod(StreamingContext context)
{
isMusicOn = true;
}
Note that here you use the "deserializing" attribute, because you do want to keep the value if it is not missing. Since OnDeserializingMethod is called before deserializing, it will be overwritten by deserialized value if it is present. This specific case (you have a field that might be missing, and you want it to have some specific value) is also covered here.
If your goal is to check if this specific field was missing in deserialized object:
With some preparation this same principle (version-tolerant serialization) can also be used to specifically determine if the field was present in deserialized object. You can set it to some predetermined value (one that would not be allowed when serializing) before deserializing (using [OnDeserializing]). Then after deserialization check if that value is still there or it was replaced by something reasonable.
So in your case let's say that allowed values for isMusicOn are true and false, then you can make your isMusicOn a nullable bool:
[OptionalField(VersionAdded = 2)]
bool? isMusicOn;
Then you'll make sure that serialized object cannot have isMusicOn set to null, by using [OnSerializing]:
[OnSerializing()]
internal void OnSerializingMethod(StreamingContext context)
{
if (isMusicOn == null)
isMusicOn = false;
}
Then assign null in a method marked with [OnDeserializing]:
[OnDeserializing]
internal void OnDeserializingMethod(StreamingContext context)
{
isMusicOn = null;
}
and then in [OnDeserialized] check if it is still null (if it was deserialized, it will have changed to proper value like true or false, if not it'll stay null):
[OnDeserialized]
internal void OnDeserializedMethod(StreamingContext context)
{
Console.WriteLine($"isMusicOn {isMusicOn == null ? "wasn't" : "was" } present in deserialized object!");
if (isMusicOn == null)
isMusicOn = false;//set it to some "proper" default value;
}
In this example, actually, nullable value wouldn't even require the setup (it'll be null by default even without the OnDeserializing part), but i'll leave it in as an example.
All this last part is probably more trouble then it's worth. Besides all the hassle with attributes, you have to change your field type to allow for that "super special value", and that might require you to change a lot of other code that depends on it. I suspect the real question is not "how to check if the field was there", but rather "how to make sure my app doesn't break because it wasn't there", and for that you don't need to know if your field was deserialized - just to make sure that it has some reasonable value in it after deserialization.
Related
I am trying to write a MongoDb serializer in c# that will allow me to decorate properties via a [Encrypt()] attribute and then at runtime it would allow me to generate an additional property called PropertyName_Encrypted which would contain the encrypted value.
On deserialization, the encrypted property value would be set in the parent property so that the default GET for the property always returns the encrypted value. Users will then call an optional Decrypt() method on the object to get decrypted values.
In doing so, I'm running into some interesting challenges:
How do I add Additional properties to the document when I am serializing current Element? How do I get the current element's name?
Is there a way I can read a specific property from the document/object? For e.g. say I want to pass a symmetric encryption key and read that to encrypt the data while serializing the current element? Is there any way I can do that?
Here are things I have done so far:
I've built an Encrypt Attribute as follows:
[AttributeUsage(AttributeTargets.Property)]
public class EncryptAttribute : Attribute
{
private readonly EncryptedFieldType _fieldType;
private readonly bool _tokenizeDisplay;
private readonly string _encryptedFieldName;
/// <summary>
///
/// </summary>
/// <param name="fieldType">The field type to encrypt. Useful if display needs to show some formatting. If no formatting is necessary, simply set to "Other".</param>
/// <param name="tokenizeDisplay">If set to true, will persist the tokenized value in the original field for display purposes.</param>
/// <param name="encryptedFieldName">Optional. If set, will save the encrypted value in the field name specified. By default all encrypted field values are stored in the corresponding _Encrypted field name. So EmailAddress field if encrypted, would have value under EmailAddress_Encrypted.</param>
public EncryptAttribute(EncryptedFieldType fieldType, bool tokenizeDisplay, string encryptedFieldName = "")
{
_fieldType = fieldType;
_tokenizeDisplay = tokenizeDisplay;
_encryptedFieldName = encryptedFieldName;
}
}
I read this Attribute on Startup and add an Encryption Serializer to the properties that are decorated using this attribute. The code that does that is like so:
var assemblies = AppDomain.CurrentDomain.GetAssemblies()
.Where(x => x.FullName.StartsWith("MongoCustomSerializer"))
.ToList();
var mapper = new Mapper();
foreach (var assembly in assemblies)
{
mapper.Map(assembly);
}
The mapper simply checks which properties in the document have the Encrypt attribute to add the serializer:
public sealed class Mapper
{
public void Map(Assembly assembly)
{
var encryptableTypes = assembly.GetTypes().Where(p =>
typeof(IEncryptable).IsAssignableFrom(p) && p.IsClass && !p.IsInterface && !p.IsValueType &&
!p.IsAbstract).ToList();
if (encryptableTypes.Any())
{
foreach (var encryptableType in encryptableTypes)
{
Map(encryptableType);
}
}
}
private void Map(Type documentType)
{
var properties =
documentType.GetProperties(BindingFlags.Public | BindingFlags.Instance);
if (properties.Length <= 0)
{
return;
}
foreach (var property in properties)
{
RegisterEncrpytionSerializer(property, typeof(EncryptAttribute), documentType);
}
}
private void RegisterEncrpytionSerializer(PropertyInfo property, Type encryptAttributeType, Type documentType)
{
var encryptAttributes = property.GetCustomAttributes(encryptAttributeType, false).ToList();
if (!encryptAttributes.Any()) return;
var memberMap = BsonClassMap.LookupClassMap(documentType).GetMemberMap(property.Name);
memberMap?.SetSerializer(new EncryptionSerializer());
}
}
In my unit tests, I'm getting an error stating that the Bson Class Map is already frozen. Even if I were to figure out a way to bypass that, how would this EncryptionSerializer class work to where I could write an additional property?
Would love to see if someone can assist!
UPDATE 1 - I was able to get the FREEZE error taken care of. It would appear that the LookupClassMap freezes the Member and Class Map info.
This change from the link allows me to take care of that issue:
private void RegisterEncrpytionSerializer(PropertyInfo property, Type encryptAttributeType, Type documentType)
{
var encryptAttributes = property.GetCustomAttributes(encryptAttributeType, false).ToList();
if (!encryptAttributes.Any()) return;
var classMapDefinition = typeof(BsonClassMap<>);
var classMapType = classMapDefinition.MakeGenericType(documentType);
var classMap = (BsonClassMap)Activator.CreateInstance(classMapType);
classMap.AutoMap();
var memberMap = classMap.GetMemberMap(property.Name);
memberMap?.SetSerializer(new KeyVaultEncryptionSerializer(memberMap.ElementName));
}
Are you using a service for saving/retrieving your items that actually call the DB?
I believe you should move the responsibility for writing/reading encrypted values to the calling service (i.e a repository implementation) instead of the BsonSerializer.
It would make sense to me that encryption/decryption is part of the persistence layer and something not handled in the application when needed.
Your implementation targets only the specified property you want to serialize. It doesn't make sense that it creates another property.
A second thought is that your suggested approach with properties that change value based on Decrypt() probably isn't a good idea since it makes your code unpredictable and hard to read. Make your properties dead simple.
What extra security in your code does it really give you if you can decrypt properties by just calling a method anyway?
If you still need to have a Decrypt() would suggest that you create methods for decrypting that return the decrypted value like GetUnencryptedCode() etc, it could just as well be an extension method but still not a readable property.
You should also be looking into using SecureString depending on your use case.
I am building a C#/.NET 4.5 client for a REST API using JSON.NET. The API supports partial updates; therefore the presence or lack of an attribute in the json on an update has meaning. If the attribute is in the json, the server will set the value accordingly; the the attribute is not passed the server will not update it. This also applies to null values. I have .NET classes for each model; with properties for each JSON attribute (pretty standard).
As an example lets say I have this account object (name, notes) that already exists on the server:
{
'name':'craig',
'notes:'these are notes'
}
If I pass in this json for an update it will update the name, but will leave the notes set to 'these are notes':
var account = api.GetAccount();
account.Name = "bob";
api.UpdateAccount(account);
{
'name':'bob'
}
If I pass this json in for an update, it will set the name and the notes to null on the server:
var account = api.GetAccount();
account.Name = "bob";
account.Notes = null;
api.UpdateAccount(account);
{
'name':'bob',
'notes':null
}
All good up to this point.
My question is how to you get JSON.NET to play along nicely with this. JSON.NET allows control the NullValueHandling which basically says if null values should be serialized or not. However that is not enough in this case. I need to be able to determine if the calling code explicitly set the value to null. Is there a recommended way to handle this?
Ive tried using a Dictionary internal to my models to store the attributes to be serialized via JSON. This allows me to tell if the attribute has been set to anything (including null) via the presence of the key in the dictionary. I found that this approach has some difficulties and I end up rewriting a lot of code that comes standard to JSON.NET (type serialization, generics, nullables, enums...).
Note: I do realize the above example is a bit contrived. In reality the account object returned back from the server would have both name and notes populated, and that when the update happened it would send both back.
The other case where this applies is during creating objects and handling server generated default. For example, lets say the server defaults the account's notes to 'put notes here' when the account is created. If I pass in the Notes attribute with a null value, the server will think the client wants to set it to null. The reality though is the client is not trying to set the Notes to null, and in this case would want the default to be set.
var account = new Account();
account.Name = "bob";
api.CreateAccount(account);
{
'name':'bob',
'notes':null
}
Im always impressed by JSON.NET...
Here is what I ended up with. I used a combination of a ContractResolver, the ShouldSerialize predicate and the NullValueHandling property. This link was very useful. The properties are stored in a Dictionary in a base class ApiModel; that code is straightforward.
Account Model
[JsonProperty("name")]
public string Name
{
get { return this.GetAttributeValue<string>("name"); }
set { this.SetAttributeValue<string>("name", value); }
}
Json Serialization
ApiModel.JsonSerializerSettings = new Newtonsoft.Json.JsonSerializerSettings();
ApiModel.JsonSerializerSettings.NullValueHandling = Newtonsoft.Json.NullValueHandling.Include;
ApiModel.JsonSerializerSettings.ContractResolver = ApiModel.JsonContractResolver;
internal class ApiModelContractResolver : Newtonsoft.Json.Serialization.DefaultContractResolver
{
protected override Newtonsoft.Json.Serialization.JsonProperty CreateProperty(System.Reflection.MemberInfo member, Newtonsoft.Json.MemberSerialization memberSerialization)
{
var property = base.CreateProperty(member, memberSerialization);
property.ShouldSerialize =
instance =>
{
var apiModel = instance as ApiModel;
var hasAttribute = apiModel.HasAttribute(property.PropertyName);
property.NullValueHandling = hasAttribute ? Newtonsoft.Json.NullValueHandling.Include : Newtonsoft.Json.NullValueHandling.Ignore;
return hasAttribute;
};
return property;
}
}
I was looking at the question Use 'dynamic' throw a RuntimeBinderException. I face a similar problem:
Basically, I want to create a "HTML helper" in ASP.NET MVC that uses dynamic arguments, akin to the htmlArguments parameter for many of the existing helpers (more code below):
public BootstrapCell(Action<string> emitContentAction, dynamic args)
View:
#using (grid.Cell(ViewContext.Writer.Write, new {Position = 4}))
{
<p>zomg!</p>
}
However in the naive approach, i get RuntimeBinderException thrown at me, declaring that 'object' does not contain a definition for 'Position', even though when debugging and hovering over the _args variable, it clearly does have a Position property.
The caller and the callee are in separate assemblies. Why is that problem happening?
(The solution to that has been shown in the same question: Manually create an ExpandoObject to hold the args.)
Implementation:
public class Cell
{
private readonly string _tagName;
private dynamic _args;
private Action<string> EmitContentAction;
public BootstrapCell(Action<string> emitContentAction, dynamic args) : DisposableBaseClass
{
_args = args;
EmitContentAction = emitContentAction;
OnContextEnter();
}
protected void OnContextEnter()
{
var sb = new StringBuilder("<");
sb.Append(_tagName);
if (_args.Position > 0)
{
sb.Append(" class=\"offset");
sb.Append(args.Position);
sb.Append("\"");
}
sb.Append(">");
EmitContentAction(sb.ToString());
}
}
[Edited to make clearer that my problem arises when "obviously" the Position property is set. I am aware that if the property never was defined in the first place, an exception must be raised.]
That code is fatally flawed.
It does work, as long as you specify that property:
void Bar()
{
Foo(new {Position = 0});
}
void Foo(dynamic args)
{
Console.WriteLine(args.Position);
}
That will output 0, it will not throw a RuntimeBinderException.
But the purpose of such code is the possibility for the caller to specify only the properties needed and omit the rest.
You are trying to check for this omission via if(args.Position != null). But that doesn't work, it already requires Position to exist.
When you have a look at the routing API of ASP.NET that also supports those anonymous configuration objects you will notice that the type of the parameter is object and not dynamic.
Using object instead of dynamic will enable your API to be used across assembly boundaries.
So how does it work?
Just like in the linked answer, you need to manually create a dictionary of the properties. Whether you use a plain old Dictionary<string, object> or an ExpandoObject is a matter of preference.
Using ExpandoObject will make your code a bit simpler to read and write, but it is not required.
About the actual exception you are getting:
Please note that it tells you it can't find the Position property on object. If it would be an anonymous type that was missing the Position property the exception message wouldn't refer to object but to an anonymous type. Something like this:
'<>f__AnonymousType0' does not contain a definition for 'Position'
I have used binary serialization to save an F# record from a C# class. All works fine:
F#:
type GameState =
{
LevelStatus : LevelStatus
Grid : Variable<Option<Ball> [,]>
...
}
let game_state : GameState = aGameState()
C#:
public void OnSaveGame() {
using (var stream = File.Open("game_status.sav", FileMode.Create))
{
var binary_formatter = new BinaryFormatter();
binary_formatter.Serialize(stream, PuzzleBobble.game_state);
}
}
Now, I'm refactoring my F# module and I would like to have a mutable record to serialize:
let mutable game_state = aGameState()
game_state <- myGameState()
This way the file is created but when I try to deserialize it I get a null object.
I've changed nothing of my previous implementation, except the added mutable keyword.
My question is: is there anything wrong with serialization of a mutable F# record? Or serialization it self is right, and I have to look for a different error somewhere else in my code?
EDIT:
Even accessing the record through methods like suggested #Brian, it seems not to work.
Here's some more details. When I deserialize the previous saved object this way (that works without game_state declared mutable):
public void OnLoadGame() {
using (var stream = File.Open("game_status.sav", FileMode.Open))
{
var binary_formatter = new BinaryFormatter();
try
{
GameLogic.GameState state = binary_formatter.Deserialize(stream) as GameLogic.GameState;
GameLogic.load_game_state(state);
}
catch (ArgumentNullException e) {
Console.WriteLine(e.Message);
}
}
}
I get the following exception:
'System.ArgumentNullException' in FSharp.Core.dll
I have once seen some weird bug where module-scoped mutable variables in F# libraries were not properly initialized, is it possible you're hitting that?
If you change the code to define
let getGameState() = game_state
let setGameState(x) = game_state <- x
and then use the get/set functions, instead of referring to the mutable variable directly, does the problem go away? If so, this might be an exotic compiler bug that we know about.
I have an application that serializes data using BinaryFormatter. A member was added to the class that was serialized from one version to the next without changing the class name. Code was added to handle the possible absence of the added member in old serialized files:
private void readData(FileStream fs, SymmetricAlgorithm dataKey)
{
CryptoStream cs = null;
try
{
cs = new CryptoStream(fs, dataKey.CreateDecryptor(),
CryptoStreamMode.Read);
BinaryFormatter bf = new BinaryFormatter();
string string1 = (string)bf.Deserialize(cs);
// do stuff with string1
bool bool1 = (bool)bf.Deserialize(cs);
// do stuff with bool1
ushort ushort1 = (ushort)bf.Deserialize(cs);
// do stuff with ushort1
// etc. etc. ...
// this field was added later, so it may not be present
// in the serialized binary data. Check for it, and if
// it's not there, do some default behavior
NewStuffIncludedRecently newStuff = null;
try
{
newStuff = (NewStuffIncludedRecently)bf.Deserialize(cs);
}
catch
{
newStuff = null;
}
_newStuff = newStuff != null ?
new NewStuffIncludedRecently(newStuff) :
new NewStuffIncludedRecently();
}
catch (Exception e)
{
// ...
}
finally
{
// ...
}
}
The point I'm at now is that I'd really like to just rinse and repeat with another member I'd like to add, which would mean I'd add another field and try-catch block similar to that for NewStuffIncludedRecently.
I had thought of just making the entire class [Serializable] but wouldn't that break compatibility with the old serialized data?
My main concern is that I'm not clear how the deserialization works. If I add in handling for another optional field similarly to above, will it work? What are other options I have for handling these changes better?
Thanks in advance as always.
If you mark the new fields with [OptionalField] it should work, but I have heard reports of flakiness in some cases. I can't say for sure, since I avoid BinaryFormatter, because it has so many issues when versioning :) (plus, it isn't as "tight" as some alternatives, and has severe issues if you want to go cross-platform, or to CF/SL etc)
If you are implementing ISerializable, you might try:
foreach(SerializationEntry entry in info) {
switch(entry.Name) {
case "Name": Name = (string)info.Value;
case "Id": Id = (int)info.Value;
...
}
}
But again, must stress - this is doing things the hard way :p
With this approach, you only process the data that is actually there.