In the ProtoBuf-Net implementation, what does the ProtoInclude attribute mean, and what does it do?
An example would be appreciated.
I saw it in this post and I'm not sure what it does. The example was:
[Serializable,
ProtoContract,
ProtoInclude(50, typeof(BeginRequest))]
abstract internal class BaseMessage
{
[ProtoMember(1)]
abstract public UInt16 messageType { get; }
}
[Serializable,
ProtoContract]
internal class BeginRequest : BaseMessage
{
[ProtoMember(1)]
public override UInt16 messageType
{
get { return 1; }
}
}
Also, is there a way to generate such inheritance using the protogen tool?
Sorry, I didn't mean to miss this one - alas, I don't see everything.
Given the specifics in the question, I'm going to assume that you are at least passingly familiar with .proto; correct me if I am wrong.
[ProtoInclude] works a lot like [XmlInclude] for XmlSerializer - or [KnownType] for DataContractSerializer - it allows it to recognise subclasses of a type during (de)serialization. The only additional thing is that it needs a tag (number) to identify each sub-type (that must be unique, and not clash with any of the fields from the parent type).
Re protogen: nope; the underlying spec (by google) makes no provision for inheritance at all, so protogen (via .proto) has no mechanism to express this. protobuf-net provides inheritance support as an extension, but does it in a way that still leaves the messages wire-compatible with the other implementations. At a push, maybe I could add protogen support via the new extension properties in the google spec, but I haven't done this yet.
So; to look at the example; that expresses an inheritance relationship between BaseMessage and BeginRequest; regardless of whether you do:
Serialize<BaseMessage>(...)
Serialize<BeginRequest>(...)
either way, it will start at the base (BaseMessage) and work upwards; which isn't exactly true - it writes the data starting with BeginRequest (so that it knows we have a BeginRequest as early as possible during deserialization). The important thing is that the fields from any parent contract types is included, and the serializer looks at the actual object passed in - not just the type you say it is.
Likewise, during deserilaization, regardless of whether you use:
Deserialize<BaseMessage>(...)
Deserialize<BeginRequest>(...)
you will get the type you actually serialized (presumably a BeginRequest).
Under the bonnet, for compatibility purposes (with the wide protocol buffers specification), this is similar to writing something like (forgive any errors, my .proto is rusty):
message BaseMessage {
optional BeginRequest beginRequest = 50;
optional uint32 messageType = 1;
}
message BeginRequest {
}
(the override probably shouldn't specify [ProtoMember], btw.
Normally, it would write fields in ascending tag order, but to make for efficient deserialization the engine cheekily chooses to write the subclass data first (which is explicitly allowed by the spec) - i.e. it writes something like (you'll have to imagine the binary...):
[tag 50, string][length of sub-message][body of sub-message][tag 1, int][value]
(in this case, the body of the sub-message is empty)
Does that cover it?
Related
I'm developing a framework where a class inheriting from an abstract class of the framework needs to be able to specify the schema for the options it can accept when it is called to DoStuff().
I started out with an abstract GetOptionsSchema() method like this:
public abstract class Widget
{
public abstract OptionsSchema GetOptionsSchema();
public abstract void DoStuff(Options options);
}
Other developers would then extend on my framework by creating custom Widget types:
public abstract class FooWidget: Widget
{
public overide DoStuff(Options options)
{
//Do some FooWidget stuff
}
public overide OptionsSchema GetOptionsSchema()
{
//Return options for FooWidget
}
}
This works but requires the framework to create an instance of every Widget type to determine options schema they accept, even if it has no need to actually DoStuff() with any of these types.
Ultimately, I'd like to be able to determine the options schema for a specific Widget type directly from a System.Type. I would create a custom OptionsSchema attribute, but constructing these schemas is more complicated then would make sense to do in the constructor of an attribute. It needs to happen in a method.
I've seen other frameworks solve similar problems by creating a custom attribute that identifies a static method or property by name. For example the TestCaseSource attribute in NUnit.
Here's what this option might look like:
public abstract class Widget
{
public abstract void DoStuff(Options options);
}
[OptionsSchemaSource(nameof(GetOptionsSchema))]
public abstract class FooWidget: Widget
{
public overide DoStuff(Options options)
{
//Do some FooWidget stuff
}
public static OptionSchema GetOptionsSchema()
{
//Return options for FooWidget
}
}
I like how the OptionsSchemaSource attribute makes it possible to get the options schema directly from a System.Type, but this also seem much less discoverable to other developers creating custom Widget types.
With the abstract method another Widget developer knows they must override GetOptionSchema() because their code would not compile otherwise. With the OptionsSchemaSource attribute the best I could do would be to hope people read my documentation and have the framework throw an exception at run-time if it encounters a Widget with out an OptionsSchemaSource attribute.
Is there an alternative/better/recommended approach to this?
You pretty much already know everything of interest to judge what's the best approach.
As already mentioned, you cannot have static interfaces defined on your type, so there is no way you can ensure a new developer is enforced to add the attribute.
So, the two alternatives you identified are the only two I can think of.
Now, let's do a pros and cons and try to sharpen them.
Attribute
You can lessen the pain of ensuring devs put attributes on the classes with meaningful error messages. I would say that you should manage the discovery of the classes based exclusively on Attributes, not in inheritance.
If you manage everything with Attributes, you don't need to inherit from Widget.
This is a pro, because now everyone can inherit if it's desirable, and re-implement if it's preferred.
The con is that the implementation of discoverability will be more complex: you will need to use reflection at start up, get a MethodInfo, check that the method has the correct signature, give proper errors in case and invoke the method unboxing the result as needed.
Think about it: you would like a static method because you don't need to instantiate a single typed Widget instance, but actually instantiating a new Widget could very well be not a big deal.
Abstract class
Well, you enforce an inheritance chain over you developers, which could be ok, necessary or entirely optional (you judge), but you get a self documenting experience.
The apparent con is that at startup you need to instantiate a Widget for every derived type you discover, but that could very well be peanuts compared to assembly scanning and type checking and methodinfo discovery and method calls through reflection.
Ugly? Kind of. Inefficient? Not so much. And it's code that is invisible to your end user.
IMHO
I find quite a good tradeoff, when designing a framework, to put some "ugly" code inside the framework, if it means that every single implementation using the library is going to be even a little bit better.
All in all, if you're designing a library that you want to be flexible and discoverable, you should expect a developer to read at least a quick start guide. If they can read in 5 minutes a single bit of information (either "extend a base class" or "add a single or a couple attributes") and that single bit gives them an direction into discovering every aspect of widget registration, I would be ok: you can't really get much better than this.
My call: I would go the abstract class route with a smallish caveat. I really don't like having an enforced base class. So I would organize discovery at startup based on interface, IWidget, containing the GetOptionsSchema method and everything is needed to use the widget (which could be the DoStuff method, but could very well be something else). At startup you search for implementations of the interface which are not abstract, and you're good to go.
If, and only if, the only bit you really need in advance is a string or other similarly simple type, I would require an additional attribute.
[OptionsSchemaName("http://something")]
public class MyWidget : WidgetBase
{
public overide DoStuff(Options options)
{
//Do some FooWidget stuff
}
public static OptionSchema GetOptionsSchema()
{
//Return options for FooWidget
}
}
Then, your type discovery infrastructure can search for non-abstract IWidgets and throw a meaningful error right at startup like the type MyWidget is lacking an OptionsSchemaName attribute. Every implementation of IWidget must define one. See http://mydocs for information.
Bang! Nailed it!
It's not currently possible to enforce the attribute at compile time; that would've been ideal for your use case. It's also not possible to have an abstract static method, or have a static method specified in an interface; so there is no way to ensure the method is actually there at compile time, except by enforcing an instance method via abstract class or interface (which will require an instance of the type to access).
I'd go with the attribute idea - it's not unreasonable to expect developers to read documentation; even with overriding an abstract method, the developer would need to know how to construct an OptionSchema in the overridden method - back to documentation!
Consider the case of StructLayout(LayoutKind.Explicit):
using System.Runtime.InteropServices;
[StructLayout(LayoutKind.Explicit)]
public struct UnionThingy
{
[FieldOffset(0x00)]
public short word;
[FieldOffset(0x00)]
public byte hiByte;
[FieldOffset(0x01)]
public byte lowByte;
public bool additionalField; // compile error!
}
Attempting to compile the above code results in the error CS0625
'UnionThingy.additionalField': instance field types marked with StructLayout(LayoutKind.Explicit) must have a FieldOffset attribute.
This seems an incredibly useful feature and I'm very curious how it is implemented.
I've read the MSDN tutorials on custom attributes, googled every related term I could think of, taken a look at the documentation for StructLayout and FieldOffset and even examined the metadata for those two attributes with "Go to definition..." in the VS code editor. I've been unable to find any clues on how the relationship between those two attributes is enforced.
If I want to do something similar:
[OptionalCustomAttribute]
public class DecoratedClass
{
[DetailsRequiredByOptionalCustomAttribute(2)] // compiler error if ommitted
public int SomeProperty {get; set;}
}
How would I go about it?
Edit: The question Force usage of custom attribute, proposed as a duplicate, asks how to force classes that inherit from a base class to include an attribute, not how to enforce members of a decorated class to include a certain attribute. Also, the answer provided is "it can't be done", whereas in this case is clearly has been done by the authors of the .net libraries. The question is simply how.
Some attributes contain information for the compiler. So naturally, there is explicit code in the compiler to interpret it and also error messages if this fails.
In our project, we have unit tests which load all types of our assemblies and check some attributes for consistency.
You can also make you application throw exceptions as early as possible (e.g. when starting up) in such cases. This is a simple solution if your application detects such errors anyway. It is important to throw early, otherwise you miss it until it went live...
I have an inheritance chain such as this:
[ProtoContract]
public abstract class Message
{
[ProtoMember(1, OverwriteList = true)]
public List<Header> Headers {get; set;}
}
[ProtoContract]
public class EventMessage<T> : Message
{
[ProtoMember(2)]
public T Event {get; set;}
}
The inheritance chain is very straight forward (my . In order to get the Headers to be included in the serialization, I need to do:
RuntimeTypeModel.Default[typeof(Packet)].AddSubType(3, typeof(Message<PayloadType>));
I know this sort of answer (the line above) has been documented on quite a few StackOverflow posts. However, I don't like this design because then I need to declare all my subtypes ahead of time and it's also implying a limited, small number of subtypes.
I'm attempting to code a message bus in my application, using protobuf-net for the serialization/deserialization. The message bus needs to send out 'events' and respond to request/replies. Since I have many (easily > 100) events in my system, I don't want to declare a subtype for every closed generic type in the RuntimeTypeModel.
Does protobuf-net have the ability to infer subtypes/classes? Or ideally, I'd like something like:
RuntimeTypeModel.Default[typeof(Packet)].AddSubType(3, typeof(Message<>));
(Which I tried and it doesn't work).
In the protobuf wire format, the only identifiers sent are the numeric keys (like the 1,2,3 in your example). After that data has been serialized, you will presumably want it to deserialize in the future - and doing that reliably if the keys are not explicitly specified is hugely problematic. Especially since those sub-types could be declared in different assemblies, so it can't even infer them by reflection.
At the moment, the short answer is "they must be specified". Note that I didn't say "in attributes" - the subtypes can also be specified via the RuntimeTypeModel API at runtime, if that is more convenient.
I'm implementing claims based security in my .net 4.5 application. Lots of hoops to jump through, but it is basically working.
The only part I don't like is that I can't create my own attributes. ClaimsPrincipalPermissionAttribute is sealed. Why?
I'm always marking throughout my application such as:
[ClaimsPrincipalPermission(SecurityAction.Demand, Resource = "Foo", Operation = "Bar")]
And since I want my resource and operation strings to not get misspelled and be easily refactorable, I have created classes so I can do this:
[ClaimsPrincipalPermission(SecurityAction.Demand, Resource = Resources.Foo, Operation = Operations.Foo.Bar)]
(Note that since different resources may have different operations, the operations themselves are subclassed by resource.)
This all works fine and dandy, but it's a hell of a lot to type or copy/paste every time. I'd rather do something like:
[DemandPermission(Resources.Foo, Operations.Foo.Bar)]
I could create this attribute, but I would need to inherit from ClaimsPrincipalPermissionAttribute, which I can't because it's sealed. :(
Is there some other way to approach this? Perhaps I don't need to inherit, but can I register my own attribute type somehow so it works in all the same places?
ClaimsPrincipalPermissionAttribute derives from CodeAccessSecurityAttribute. It does almost nothing except implement CreatePermission() returning a new ClaimsPrincipalPermission based on the value of Resource and Operation that you pass in.
You could implement a new class deriving from CodeAccessSecurityAttribute (this is not sealed) that does what you want.
Using JustDecompile, you can see that the code in ClaimsPrincipalPermissionAttribute is simple. You could make your own attribute like this:
[AttributeUsage(AttributeTargets.Class | AttributeTargets.Method | AttributeTargets.Property, AllowMultiple = true)]
public sealed class DemandPermissionAttribute : CodeAccessSecurityAttribute
{
public Operations Operation { get; set; }
public Resources Resource { get; set; }
public DemandPermissionAttribute(SecurityAction action = SecurityAction.Demand)
: base(action)
{
}
public override IPermission CreatePermission()
{
return new ClaimsPrincipalPermission(this.Resource.ToString(), this.Operation.ToString());
}
}
One important thing to note on this is that you must define your custom attribute in a separate assembly from the one that is referencing it, otherwise the framework will throw a TypeLoadException as described here
http://msdn.microsoft.com/en-us/library/vstudio/yaah0wb2.aspx
Also, note the use of the default value for the constructor parameter. You need to have a constructor that takes a SecurityAction parameter for the attribute to get instantiated by the framework. Maybe DemandPermission is a bad name in this case, because you can override the SecurityAction to be something other than SecurityAction.Demand.
ClaimsPrincipalPermissionAttribute is sealed. Why?
Eric Lippert talked about the commonness of sealed in Framework types, and since we are talking about code security, this bit is very important:
Every time you implement a method which takes an instance of an unsealed type, you MUST write that method to be robust in the face of potentially hostile instances of that type. You cannot rely upon any invariants which you know to be true of YOUR implementations, because some hostile web page might subclass your implementation, override the virtual methods to do stuff that messes up your logic, and passes it in. Every time I seal a class, I can write methods that use that class with the confidence that I know what that class does.
This is even more important in this case, ClaimsPrincipalPermissionAttribute is checked via IClaimsPrincipal an interface. So by making ClaimsPrincipalPermissionAttribute sealed, they allow any implementer of IClaimsPrincipal to not have to worry about hostile implementations. This is quite a savings, given this is all security related.
My immediate reaction is that that isn't a whole lot to write - and how often do you need to write it? If it is general to actions in your controller, place it on the controller - if is applicable to many controllers, create a ControllerBase with that attribute.
If your case is more special than that, I suppose you are forced to implement your own variety of that attribute.
I have a situation where I would like to have objects of a certain type be able to be used as two different types. If one of the "base" types was an interface this wouldn't be an issue, but in my case it is preferable that they both be concrete types.
I am considering adding copies of the methods and properties of one of the base types to the derived type, and adding an implicit conversion from the derived type to that base type. Then users will be able treat the derived type as the base type by using the duplicated methods directly, by assigning it to a variable of the base type, or by passing it to a method that takes the base type.
It seems like this solution will fit my needs well, but am I missing anything? Is there a situation where this won't work, or where it is likely to add confusion instead of simplicity when using the API?
EDIT: More details about my specific scenario:
This is for a potential future redesign of the way indicators are written in RightEdge, which is an automated trading system development environment. Price data is represented as a series of bars, which have values for the open, low, high, and close prices for a given period (1 minute, 1 day, etc). Indicators perform calculations on series of data. An example of a simple indicator is the moving average indicator, which gives the moving average of the most recent n values of its input, where n is user-specified. The moving average might be applied to the bar close, or it could be applied to the output of another indicator to smooth it out.
Each time a new bar comes in, the indicators compute the new value for their output for that bar.
Most indicators have only one output series, but sometimes it is convenient to have more than one output (see MACD), and I want to support this.
So, indicators need to derive from a "Component" class which has the methods that are called when new data comes in. However, for indicators which have only one output series (and this is most of them), it would be good for them to act as a series themselves. That way, users can use SMA.Current for the current value of an SMA, instead of having to use SMA.Output.Current. Likewise, Indicator2.Input = Indicator1; is preferable to Indicator2.Input = Indicator1.Output;. This may not seem like much of a difference, but a lot of our target customers are not professional .NET developers so I want to make this as easy as possible.
My idea is to have an implicit conversion from the indicator to its output series for indicators that have only one output series.
You don't provide too many details, so here is an attempt to answering from what you provide.
Take a look at the basic differences:
When you have a base type B and a derived type D, an assignment like this:
B my_B_object = my_D_object;
assigns a reference to the same object. On the other hand, when B and D are independent types with an implicit conversion between them, the above assignment would create a copy of my_D_object and store it (or a reference to it if B is a class) on my_B_object.
In summary, with "real" inheritance works by reference (changes to a reference affect the object shared by many references), while custom type conversions generally work by value (that depends on how you implement it, but implementing something close to "by reference" behavior for converters would be nearly insane): each reference will point to its own object.
You say you don't want to use interfaces, but why? Using the combo interface + helper class + extension methods (C# 3.0 and .Net 3.5 or newer required) can get quite close to real multiple inheritance. Look at this:
interface MyType { ... }
static class MyTypeHelper {
public static void MyMethod(this MyType value) {...}
}
Doing that for each "base" type would allow you to provide default implementations for the methods you want to.
These won't behave as virtual methods out-of-the-box; but you may use reflection to achieve that; you would need to do the following from within the implementation on the Helper class:
retrieve a System.Type with value.GetType()
find if that type has a method matching the signature
if you find a matching method, invoke it and return (so the rest of the Helper's method is not run).
Finally, if you found no specific implementation, let the rest of the method run and work as a "base class implementation".
There you go: multiple inheritance in C#, with the only caveat of requiring some ugly code in the base classes that will support this, and some overhead due to reflection; but unless your application is working under heavy pressure this should do the trick.
So, once again, why you don't want to use interfaces? If the only reason is their inability to provide method implementations, the trick above solves it. If you have any other issue with interfaces, I might try to sort them out, but I'd have to know about them first ;)
Hope this helps.
[EDIT: Addition based on the comments]
I've added a bunch of details to the original question. I don't want to use interfaces because I want to prevent users from shooting themselves in the foot by implementing them incorrectly, or accidentally calling a method (ie NewBar) which they need to override if they want to implement an indicator, but which they should never need to call directly.
I've looked at your updated question, but the comment quite summarizes it. Maybe I'm missing something, but interfaces + extensions + reflection can solve everything multiple inheritance could, and fares far better than implicit conversions at the task:
Virtual method behavior (an implementation is provided, inheritors can override): include method on the helper (wrapped in the reflection "virtualization" described above), don't declare on the interface.
Abstract method behavior (no implementation provided, inheritors must implement): declare method on the interface, don't include it on the helper.
Non-virtual method behavior (an implementation is provided, inheritors may hide but can't override): Just implement it as normal on the helper.
Bonus: weird method (an implementation is provided, but inheritors must implement anyway; they may explicitly invoke the base implementation): that's not doable with normal or multiple inheritance, but I'm including it for completeness: that's what you'd get if you provide an implementation on the helper and also declare it on the interface. I'm not sure of how would that work (on the aspect of virtual vs. non-virtual) or what use it'd have, but hey, my solution has already beaten multiple inheritance :P
Note: On the case of the non-virtual method, you'd need to have the interface type as the "declared" type to ensure that the base implementation is used. That's exactly the same as when an inheritor hides a method.
I want to prevent users from shooting themselves in the foot by implementing them incorrectly
Seems that non-virtual (implemented only on the helper) will work best here.
or accidentally calling a method (ie NewBar) which they need to override if they want to implement an indicator
That's where abstract methods (or interfaces, which are a kind of super-abstract thing) shine most. The inheritor must implement the method, or the code won't even compile. On some cases virtual methods may do (if you have a generic base implementation but more specific implementations are reasonable).
but which they should never need to call directly
If a method (or any other member) is exposed to client code but shouldn't be called from client code, there is no programmatic solution to enforce that (actually, there is, bear with me). The right place to address that is on the documentation. Because you are documenting you API, aren't you? ;) Neither conversions nor multiple inheritance could help you here. However, reflection may help:
if(System.Reflection.Assembly.GetCallingAssembly()!=System.Reflection.Assembly.GetExecutingAssembly())
throw new Exception("Don't call me. Don't call me!. DON'T CALL ME!!!");
Of course, you may shorten that if you have a using System.Reflection; statement on your file. And, BTW, feel free to change the Exception's type and message to something more descriptive ;).
I see two issues:
User-defined type conversion operators are generally not very discoverable -- they don't show up in IntelliSense.
With an implicit user-defined type conversion operator, it's often not obvious when the operator is applied.
This doesn't been you shouldn't be defining type conversion operators at all, but you have to keep this in mind when designing your solution.
An easily discoverable, easily recognizable solution would be to define explicit conversion methods:
class Person { }
abstract class Student : Person
{
public abstract decimal Wage { get; }
}
abstract class Musician : Person
{
public abstract decimal Wage { get; }
}
class StudentMusician : Person
{
public decimal MusicianWage { get { return 10; } }
public decimal StudentWage { get { return 8; } }
public Musician AsMusician() { return new MusicianFacade(this); }
public Student AsStudent() { return new StudentFacade(this); }
}
Usage:
void PayMusician(Musician musician) { GiveMoney(musician, musician.Wage); }
void PayStudent(Student student) { GiveMoney(student, student.Wage); }
StudentMusician alice;
PayStudent(alice.AsStudent());
It doesn't sound as if your method would support a cross-cast. True multiple inheritance would.
An example from C++, which has multiple inheritance:
class A {};
class B {};
class C : public A, public B {};
C o;
B* pB = &o;
A* pA = dynamic_cast<A*>(pB); // with true MI, this succeeds
Then users will be able treat the derived type as the base type by using the duplicated methods directly, by assigning it to a variable of the base type, or by passing it to a method that takes the base type.
This will behave differently, however. In the case of inheritance, you're just passing your object. However, by implementing an implicit converter, you'll always be constructing a new object when the conversion takes place. This could be very unexpected, since it will behave quite differently in the two cases.
Personally, I'd make this a method that returns the new type, since it would make the actual implementation obvious to the end user.
Maybe I'm going too far off with this, but your use case sounds suspiciously as if it could heavily benefit from building on Rx (Rx in 15 Minutes).
Rx is a framework for working with objects that produce values. It allows such objects to be composed in a very expressive way and to transform, filter and aggregate such streams of produced values.
You say you have a bar:
class Bar
{
double Open { get; }
double Low { get; }
double High { get; }
double Close { get; }
}
A series is an object that produces bars:
class Series : IObservable<Bar>
{
// ...
}
A moving average indicator is an object that produces the average of the last count bars whenever a new bar is produced:
static class IndicatorExtensions
{
public static IObservable<double> MovingAverage(
this IObservable<Bar> source,
int count)
{
// ...
}
}
The usage would be as follows:
Series series = GetSeries();
series.MovingAverage(20).Subscribe(average =>
{
txtCurrentAverage.Text = average.ToString();
});
An indicator with multiple outputs is similar to GroupBy.
This might be a stupid idea, but: if your design requires multiple inheritance, then why don't you simply use a language with MI? There are several .NET languages which support multiple inheritance. Off the top of my head: Eiffel, Python, Ioke. There's probable more.