I was trying to add a new enum value for a certain protobuf-serialized class in a new app version, and while testing, noticed that the previous version will throw an exception, given this new file format:
An unhandled exception of type 'ProtoBuf.ProtoException' occurred in protobuf-net.dll
Additional information: No {enum-type-name} enum is mapped to the wire-value 3
It is fairly obvious that it's telling me that there is no enum value for the int value of 3, but I always had the idea that Protocol Buffers defaulted to the zero-valued ("default") enum value (if such exists), in case that an actual enum value couldn't be mapped to.
To clarify, this can be reproduced using the following example (I am intentionally doing the deserialization step into a different class to mimic old app trying to load the new format):
// --- version 1 ---
public enum EnumV1
{
Default = 0,
One = 1,
Two = 2
}
[ProtoContract]
public class ClassV1
{
[ProtoMember(1)]
public EnumV1 Value { get; set; }
}
// --- version 2 ---
public enum EnumV2
{
Default = 0,
One = 1,
Two = 2,
Three = 3 // <- newly added
}
[ProtoContract]
public class ClassV2
{
[ProtoMember(1)]
public EnumV2 Value { get; set; }
}
And the following code will fail:
// serialize v2 using the new app
var v2 = new ClassV2() { Value = EnumV2.Three };
var v2data = Serialize(v2);
// try to deserialize this inside the old app to v1
var v1roundtrip = Deserialize<ClassV1>(v2data);
Since v1 is out in the open, is there some metadata I can use when serializing in v2 to avoid this issue? I can, of course, get myself out of this trouble by rewriting v2 to use a separate property and leave the enum values unmodified, but I'd like to make enums backwards compatible if possible.
Adding [ProtoContract(EnumPassthru=true)] to your enums will allow protobuf-net to deserialize unknown values.
Unfortunately, there is no way to retroactively fix your v1. You'll have to use a different property.
Since v1 is out in the open, is there some metadata I can use when serializing in v2 to avoid this issue? I can, of course, get myself out of this trouble by rewriting v2 to use a separate property and leave the enum values unmodified, but I'd like to make enums backwards compatible if possible.
What you are experiencing is a protobuf-net bug described here protobuf-net - issue #422: Invalid behaviour while deserializing unknown enum value.
It seems that it's not fixed yet according to here protobuf-net faulty enum exception (issue 422) need a good workaround (and of course your post).
Unfortunately you need to either fix the protobuf-net source code or use the workarounds mentioned.
UPDATE: I've checked the code in the GitHub repository and confirming that the issue is still not fixed. Here is the problematic code inside the EnumSerializer.cs (the ISSUE #422 comment is mine):
public object Read(object value, ProtoReader source)
{
Helpers.DebugAssert(value == null); // since replaces
int wireValue = source.ReadInt32();
if(map == null) {
return WireToEnum(wireValue);
}
for(int i = 0 ; i < map.Length ; i++) {
if(map[i].WireValue == wireValue) {
return map[i].TypedValue;
}
}
// ISSUE #422
source.ThrowEnumException(ExpectedType, wireValue);
return null; // to make compiler happy
}
Your ClassV1 lacks forward compatiblity.
I would have implemented the Proto contract in such a way that it serializes/deserializes the string representation of the enum value. This way you can handle the fallback to the default value by yourself. The Value property would not be serialized/deserialized.
public enum EnumV1
{
Default = 0,
One = 1,
Two = 2
}
public enum EnumV2
{
Default = 0,
One = 1,
Two = 2,
Three = 3 // <- newly added
}
[ProtoContract]
public class ClassV1
{
[ProtoMember(1)]
public string ValueAsString
{
get { return Value.ToString(); }
set
{
try
{
Value = (EnumV1) Enum.Parse(typeof (EnumV1), value);
}
catch (Exception)
{
Value = EnumV1.Default;
}
}
}
public EnumV1 Value { get; set; }
}
[ProtoContract]
public class ClassV2
{
[ProtoMember(1)]
public string ValueAsString
{
get { return Value.ToString(); }
set
{
try
{
Value = (EnumV2)Enum.Parse(typeof(EnumV2), value);
}
catch (Exception)
{
Value = EnumV2.Default;
}
}
}
public EnumV2 Value { get; set; }
}
Still it does not solve the problem of having a non-forward-compabtible class in production.
You could add the DefaultValue attribute to your proto member property.
[ProtoContract]
public class ClassV1
{
[ProtoMember(1), DefaultValue(EnumV1.Default)]
public EnumV1 Value { get; set; }
}
To make clear how the property should be initialized for the default case.
Related
Initializing a class like this:
var x = new Item()
{
ID = (int)...,
Name = (string)...,
..
};
I am getting an InvalidCastException on one of the assignments. There are quite a lot of them and the exception occurs on the whole expression even if I run the debugger line-by-line. The exception doesn't give any clue either what it's trying to cast to what.
Is there a way to debug each assignment individually? I've seen the debugger separately stops 3 times on expressions like foreach(x in y) so it seems a little strange it isn't doing that here, and detracts from the attraction of using this handy initialization syntax. Maybe there is a more fine-grained debug step I can use?
Your question is "Is it possible to debug a struct/class initialization member by member?".
So, up front, I'm not directly answering that question as worded because when I carefully read the body of your post it sounds like the essential question is how to identify the 'smoking line' root cause of this InvalidCastException right when it happens.
What I've found in similar situations is that if Visual Studio can be made to break at the moment the InvalidCastException occurs i.e. on that specific line then the Call Stack and local variables are much more immediate and useful.
Unfortunately, 'break when thrown' is suppressed by Visual Studio default settings for many exception types. But it's very easy to turn on 'break when thrown' for ALL exceptions. Just change this default setting in the Exceptions window in Visual Studio from this:
to this:
This doesn't "always" help but it's a good start. It's so easy why not try that first to see if rapid resolution is possible. Hope this turns out to be useful in your case.
Not sure if this is an option in VS 2017, I only have 2019 at hand. In your settings under Options -> Debugging -> General, uncheck Step over properties and operators. Then set a break point at your initializer and step through it with F11 (Step-Into). You will hit each property setter until the exception is thrown.
Please excuse me if I am missing something (will delete if thats the case) but using an invalid cast like this:
struct Item
{
public int ID { get; set; }
public Derived Derived { get; set; }
}
public class Base
{
public string Name { get; set; }
}
public class Derived : Base
{
public string AdditionalProperty { get; set; }
}
var baseClass = new Base()
{
Name = "foo",
};
try
{
var x = new Item()
{
ID = (int)20,
Derived = (Derived)baseClass,
};
}
catch(Exception e)
{
Console.WriteLine(e.Message);
}
neatly catches the error like this
Unable to cast object of type 'Base' to type 'Derived'.
in vs2017
However this breaks on the first error, we might wanna initalize an object this way and log all cast errors withouth exiting the initialisation. We can do this by implementing our own casting for this sample:
public static T TryCast<T>(Object _object)
{
try
{
return (T)_object;
}
catch (Exception e)
{
Console.WriteLine($"Cant cast object of type {_object.GetType().ToString()} to object of type {typeof(T)}");
}
return default(T);
}
public static T TryCast<T>(IConvertible _object)
{
try
{
return (T)Convert.ChangeType(_object, typeof(T));
}
catch (Exception e)
{
Console.WriteLine($"Cant convert object of type {_object.ToString()} to object of type {typeof(T)}");
}
return default(T);
}
New types for demonstrating purposes
struct Item
{
public int ID { get; set; }
public double FooDouble { get; set; }
public Base Base { get; set; }
public Derived Derived { get; set; }
public string Bar { get; set; }
}
public class Base
{
public string Name { get; set; }
}
public class Derived : Base
{
public string AdditionalProperty { get; set; }
}
We can then initalize our object like this:
var derived = new Derived()
{
Name = "DerivedFoo",
AdditionalProperty = "Bar"
};
var _base = new Base()
{
Name = "BaseFoo"
};
var x = new Item()
{
ID = Utils.TryCast<int>("please no"),
FooDouble = Utils.TryCast<double>(2),
Base = Utils.TryCast<Base>(derived),
Derived = Utils.TryCast<Derived>(_base),
Bar = "Foo"
};
And we neatly log any errors that might occur when casting:
Is it possible to obtain the value from (lets say a string property) of a class from a custom attribute?
For example:
public class test
{
[EncodeHTML]
public string body { get; set; }
public int id { get; set; }
}
I would want the custom attribute EncodeHTML to be able to obtain the value of the setting value of the "body" property.
I know this can be achieved via the following:
public string body
{
get;
set {
value = HttpUtility.HTMLEncode(this);
}
But was wondering if this could be isolated for re-use across many class properties.
Here is a plain example of the custom attribute:
[AttributeUsage(AttributeTargets.Property, Inherited = true, AllowMultiple = false)]
sealed class EncodeHTMLAttribute : Attribute
{
public EncodeHTMLAttribute()
{
}
}
It is
var attr = typeof(test).GetProperty("body").GetCustomAttribute<EncodeHTMLAttribute>()
and you can access whatever you want.
To answer the question directly - no, the attributes are just metadata. They have no idea that a runtime instance of the decorated type even exist.
The closest you could do with an attribute is, at runtime, right when the app starts, find all types that have properties marked with that attribute and rewrite the setters code to what you want. It can be done in theory, but is hard, crazy, irresponsible and completely not recommended. Refer to this SO question to jump into the rabbit hole.
Having that said, to solve the underlying problem, you can just make a custom wrapper on a string.
public class HtmlEncodedString
{
public string Value { get; }
public HtmlEncodedString(string value) =>
Value = HttpUtility.HtmlEncode(value);
public static implicit operator string(HtmlEncodedString htmlEncodedString) =>
htmlEncodedString.Value;
public static implicit operator HtmlEncodedString(string value) =>
new HtmlEncodedString(value);
}
That's of course only a sketch. If you're using ASP.NET Core, consider implementing IHtmlContent. If you're allocating a lot of these, maybe making it a value type will decrease the pressure on the GC. Neverminding these details, you now can get reusability by just using this type instead of an attribute on a string.
public class Test
{
public HtmlEncodedString Body { get; set; }
public int Id { get; set; }
}
Because of the implicit operators, transition is seemless:
var test = new Test();
test.Body = "2 < 4";
string s = test.Body;
Console.WriteLine(s);
Console.WriteLine(test.Body);
> 2 < 4
> 2 < 4
Some sample code below. The interesting/problem case is the Data property in
Mad. This code blows up (null value in the enumerable). Also, it works if i don't use the static attributes but instead the runtime type model, where i put in member.SupportNull = true for the fields (which is the behaviour i want), so what am i missing in the attributes / settings? Google search seems to indicate this is an open issue with probuf-net? That the same functionality is not available via attributes?
As as aside, if someone could suggest a way - i really love the runtime type model, i want to use that everywhere with a nice compiled model... but with it i lose the object versioning that protocol buffers solves! (via explicit tags). Is there any good way to maintain object version compatibility (simply adding fields) without doing all the static notation with fixed tags?
Basically the key thing with the runtime model is the assignment of tag indices and i can't think of a way of handling versions without explicitly specifying the tag indices via attributes...
[ProtoContract]
[ProtoInclude(1, typeof(ing))]
public class Eff
{
[ProtoMember(2)]
public string gg { get; set; }
}
[ProtoContract]
public class ing : Eff
{
[ProtoMember(1)]
public int zz { get; set; }
}
[ProtoContract]
public class Mad
{
[ProtoMember(1)]
public string Name { get; set; }
[ProtoMember(2)]
public IEnumerable<ing> Data { get; set; }
[ProtoMember(3)]
public ing Single { get; set; }
}
private static void Main(string[] args)
{
var obj = new Mad
{
Name = "test"
,Data = new[] { new ing {gg = "ooga", zz = -101},null,new ing()}
,Single = new ing {gg = "abc", zz = -999}
};
var m = new MemoryStream();
Serializer.Serialize(m, obj);
m.Seek(0, SeekOrigin.Begin);
var copy = Serializer.Deserialize<Mad>(m);
}
Short answer, it seems unavailable via attributes.
Workaround i'm doing for now - for every single type of interest(including the whole inheritance hierarchy) - add it to the type model yourself (with default handling so that it processes attributes), then call .GetFields() and set .SupportNull = true for each field (or only the relevant one)
In my project I need to build a generic deserializer that should be backward compatible.
Example: The XML looks like
<PolicyDef name = "sample" type="type1">
<Options ......>
</PolicyDef>
The "type" is enum - PolicyTypes
e.g
public Enum PolicyTypes
{
type1 = 0,
type2 = 1
}
The PolicyDef class is defined as
[XmlRoot("PolicyDef")]
public class PolicyDef
{
private string policyName;
private PolicyTypes policyType;
public PolicyDefinition()
{
}
[XmlAttribute]
public string Name
{
get
{
return this.policyName;
}
set
{
this.policyName = value;
}
}
[XmlAttribute]
public PolicyTypes Type
{
get
{
return this.policyType;
}
set
{
this.policyType = value;
}
}
}
The Problem with this approach is that if later on I put any type other than type 1 or type 2, the XMLDeserializer will throw exception.
so if i have the xml like
<PolicyDef name = "sample" type="type_new">
<Options ......>
</PolicyDef>
The deserializer will throw error as type_new not valid.
I was wondering if there is a way to hook into the deserializer process to catch that and set a default value rather than throw error. Say if there is any invalid value, then I would set that to "type1"
Or am open to suggestions regarding how to handle this problem
Thanks and Regards
This is possibly a duplicate of C# XML Deserialization W/ Default Values
Unfortunately it seems there is no way to fall back on default enum values during deserialisation. It will require slightly more work, but if you follow the linked example and implement IXmlSerializable in your PolicyDef class, you'll be able to implement the ReadXml method in a similar way (reflecting each of the properties using a try/catch block in order to check for a default value).
Hope that helps!
Thanks Chris for the suggestion, but I don't want end up writing the complete parsing code which could be messy if the XML and corresponding class is huge and complex. I anyway used a different approach.
I changed all the enum fields to string. In this case there would be no parsing error and then expose another property that would return the parsed value as enum and if the parsing fails, then return default enum value. E.g
private string policyName;
[XmlAttribute("Type")]
public string Type
{
private get
{
return this.policyType;
}
set
{
this.policyType = value;
try
{
this.PolicyType = (PolicyTypes)Enum.Parse(typeof(PolicyTypes), this.policyType);
}
catch(Exception)
{
this.PolicyType = PolicyTypes.DefaultPolicy;
}
}
}
public PolicyTypes PolicyType
{
get;
private set;
}
And use the class property to access the value rather than the xml attribute field.
In my code, I have a class that maintains a number of lists. We'll focus on one of them for the moment, since it's the one that highlighted the problem.
internal List<Badge> Badges { get; private set; }
In the code, I add Badge instances to this list when an XML document is parsed. Later, I want to update the individual instances in the list so I can have the data written back out to XML. Because of the way the data's XML structure differs from the original file structure, there's some hocus-pocus involved, but that's largely mapped out. The surprise came when I attempted to update an item in the List<Badge>.
Specifically, the problematic code is here:
// Get the current badge from the loaded XML data, so we can update it.
var currentBadge = this.GameData.GetCurrentBadge();
I always get a valid badge back. The surprise, as I've come to find out, is that this simple test always fails:
var result = this.GameData.Badges.IndexOf(currentBadge);
result always evaluates to -1, indicating that the object doesn't exist in the collection. (EDIT: Updating the properties on currentBadge has no effect whatsoever on the contents of the matching item in this.GameData.Badges.) Which leads me to conclude that I'm getting a copy of my object back, and not a reference, as I would have expected.
For the inquisitive, the code to retrieve badges from the GameData class is included below. I have a sneaking suspicion that this is a documented behavior of generic lists, and that this is the first time I've stumbled across it. If so, I'm in for a very rude awakening. If it's not, I'd really like to know why my objects are coming back "disconnected" from their originals.
private Badge GetCurrentBadge()
{
var badgeItem = GetCurrentBadgeItem();
if (badgeItem != null)
{
return this.GameData.GetBadgeByText(badgeItem.Text);
}
return null;
}
private MenuOption GetCurrentBadgeItem()
{
if (!(this.currentItem is MenuOption &&
(this.currentItem as MenuOption).IsLocked))
{
return null;
}
MenuOption result = null;
var children = this.currentMenu.Children;
for (var n = children.Count - 1; n >= 0; n--)
{
var child = children[n] as MenuOption;
if (child == null || !child.IsLocked)
{
break;
}
if (!child.Text.StartsWith(" "))
{
result = child;
break;
}
}
return result;
}
UPDATE: Per request, GetBadgeByText, which comes from the GameData class.
internal Badge GetBadgeByText(string badgeText)
{
foreach (var badge in Badges)
{
if (badge.Text.ToLower() == badgeText.ToLower())
{
return badge;
}
}
return null;
// var b = (from l in Badges
// where l.Text.ToLower().StartsWith(badgeText.ToLower())
// select l).FirstOrDefault();
//return b;
}
As you can see, I've tried it both with and without Linq, just to eliminate that as the culprit. Changing the implementation had no noticable effect.
And for the record, all the objects in this application are CLASSES. No structs anywhere.
UPDATE #2: The Badge class.
internal class Badge
: GameDataItem
{
public Badge()
: base()
{
}
public string AuthId { get; set; }
public string Category { get; set; }
public string Description { get; set; }
public bool IsAccoladePower { get; set; }
public string RequiredBadges { get; set; }
public override string ToString()
{
return Text;
}
internal string ToXml()
{
var template = "<Badge value=\"{0}\" title=\"{1}\" category=\"{2}\" authid=\"{3}\" requires=\"{4}\" accolade=\"{5}\" description=\"{6}\" />";
return string.Format(template,
this.Value,
this.Text,
this.Category,
this.AuthId,
this.RequiredBadges,
this.IsAccoladePower,
this.Description);
}
}
And just in case someone asks for it, the base class:
internal class GameDataItem
{
private string _text;
public string Text
{
get
{
return this._text;
}
set
{
this._text = value.Replace("<", "<")
.Replace(">", ">")
.Replace("&", "&");
}
}
public string Value { get; set; }
public override string ToString()
{
return Text + "=\"" + Value + "\"";
}
}
Looks to me like this has something to do with MenuOption's implementation of Equals(object). The IndexOf() method of the List<> will use Equals(object) when deciding what to return.
Either:
You are putting a copy of the object in the list. (List<T> does not clone objects or do any other sort of trickery.)
Badge is a struct, not a class, which means that you don't actually hold references to it since it would be a value type.
There's some copying going on elsewhere in code you haven't pasted.
A generic List<T> does not copy objects. You add references to it, and the same references comes out - so there must be another problem in the code.
How is GetBadgeFromText implemented ? Does it read directly from the Badges List ?
Is this a web app ? If yes, does your List live between requests, or is it deserialized and serialized on each request (this could also be the problem).