I am working on an application that loads plugins at startup from a subdirectory, and currently i am doing this by using reflection to iterate over the types of each assembly and to find public classes implementing the IPluginModule interface.
Since Reflection involves a performance hit, and i expect that there will be several plugins after a while, i wondered if it would be useful to define a custom attribute applied at the assembly level, that could be checked before iterating over the types (possibly about a dozen types in an assembly, including 1 implementor of IPluginModule).
The attribute, if present, could then provide a method to return the needed types or instances, and iterating over the types would then only be a fallback mechanism. Storing the type info in a configuration file is not an option.
Would this improve performance, or does it just not matter compared to the time to actually takes to load the assembly from storage? Also, would this usage be appropriate for an attribute at all?
I will answer your question with a question: Why are you worried about this?
You're worrying about a potential performance hit in a one time operation because there might be several plugins at a later date.
Unless your application startup time is excessively long to a user, I wouldn't waste time thinking about it - there are probably much better things that you can work on to improve your application.
You could also have the plugable types in a configuration, so you know the exact classes instead of looping through all classes. Would have to have some configuration utility for this option...but could possibly get a good increase in performance depending on the number of classes you are looping through.
I believe both of Microsoft's two .net plugin frameworks, the Managed AddIn Framework (MAF) and the Managed Extensibility Framework (MEF) can use either attributes or reflection to discover plugins. So Microsoft seems to feel attributes are appropriate.
I'm not sure what the performance differences are, though.
A good solution is to cache all information about plugins. The first time the application is started it does a full scan of the plugin dlls, and saves the list of types found in a file. The next time the application starts, it loads the information from the file, which will be much faster than scanning all the dlls again. The application can also store a timestamp of each dll, so if it detects a change in a dll it can re-scan it and update the cache.
That's basically the approach followed by the Mono.Addins framework.
I'd have thought that asking an assembly for all the classes that are tagged with an attribute would also use reflection. It would then come down to which is a faster look up in the metadata, interface implementation or attribute marking?
Related
I'm developing a system which needs to support customization via a plugins module. I'm coding against interfaces so that plugin code only needs to implement these interfaces in order to be able to plug into the system.
// for illustration purposes; not actual code
public interface IPluggable
{
void Setup(PluginConfig c);
bool Process(IProcessable p);
}
I read from configuration which plugins need to be loaded, where the assembly name and fully-qualified type name are specified.
<plugin assembly="Foo.Bar.PluginAssembly" type="Foo.Bar.Plugins.AwesomePlugin" />
Where the type Foo.Bar.Plugins.AwesomePlugin implements IPluggable and is contained in the assembly Foo.Bar.PluginAssembly.dll. With this information I proceed to create instances of the required plugins.
IPluggable plugin = (IPluggable)Activator.CreateInstance(assemblyName, typeName).Unwrap();
So my question is threefold:
What would be a recommended pattern for a plugin system? Does the approach I'm taking make sense or are there any obvious flaws/caveats I'm missing?
Is Activator.CreateInstance() a good choice for dynamically instantiating the plugin objects?
How can I be more specific about the assembly to load and its location? Say, if I want to load plugins only from assemblies located in a .\plugins subfolder.
Answers to your questions, in order:
I like this and I use patterns like this when I need to write plug in components. Other people recommend using various frameworks - I know that MEF is very popular. But I find that using the .NET framework is easy enough for me, and learning the MEF framework is just another thing I need to learn and remember. It's probably worth a try but up to you.
I've always used Assembly.CreateInstance, but the difference is probably not going affect you (Difference between Assembly.CreateInstance and Activator.CreateInstance?)
You simply use the System.IO namespace. The DirectoryInfo class has a method that enumerates all the files matching a given pattern (presumably *.dll). For each match I'd use System.Reflection namespace to interrogate and find any types that implement your interface, and then CreateInstance.
Just on MEF, my opinion is this: if I were going to be using a large, manageable and flexible plug-in system on a number of systems or projects then I'd be very interested in it, leveraging the work that other people have done to save time and avoid common pitfalls.
If I were writing a very simple, one-off plug-in system and I know the basics of how to do so using the .NET framework, I'd skip the overhead of learning MEF and write the code. I could write a reasonable plug-in process in far less than an hour, but after downloading, referencing, attempting to configure MEF - I doubt I'd have anything to show for it.
I'm writing an app which plays host to a series of plug-ins. Those plug-ins generally use two libraries .Common and .UI which contain the interfaces that the plug-ins need to implement etc.
I am now at the point where I'm adding the capability for plug-ins to be subject to licensing. I have modified my host application such that it will only load plug-ins which define an interface instance (ILicenseInfoProvider) and export it through MEF. That bit is fine.
We have a selected provider of licensing code, and their licensing system involves use of a library. Now, I don't want to force each plug-in to be licensed through that system, and, by extension, require a reference to that system's assembly. So, I am planning on putting the code that references the third-party library in it's own assembly (something like .Licensing.Vendor). This way plug-ins can simply add a reference to that assembly, and include a class that looks somewhat like this:
[Export(typeof(ILicenseInfoProvider))]
class MyAssemblyLicenseInfoProvider : BaseVendorLicenseInfoProvider
{
public MyAssemblyLicenseInfoProvider() : base("My Assembly's Product Name")
}
I'm reasonably happy with that set-up, apart from one niggling thing - which is that the .Licensing.Vendor assembly will only contain a single class, which is the BaseVendorLicenseInfoProvider relating to the specific licensing system in use.
So, after all that, my question is pretty simple:
Does it seem overkill to put that class in it's own assembly, or is the benefit of not forcing all plug-ins to hold a reference to the third party library worth it?
At the moment there's a suitable purpose for the assembly - a publicly visible assembly for third parties to provide a means to interact via licensing. Seems perfectly reasonable to me:
even if there is only the one class currently, there may be more in the future
it's publicly visible, so you only want to provide only that which is necessary
it encapsulates a reasonable level of responsibility, namely licensing, without forcing specific implementations
I vote no, its not overkill, some plugins may not need a license, some may do..
It depends on what you are trying to achieve. Assemblies are a way of physically separating code whereas namespaces are a way of logically separating code.
Given that there can be a slight performance hit of loading too many assemblies (by which I mean a significant number, not just a few) then I suppose you could consider if it is possible to group as much as you can into one assembly but separate them by namespaces. But if you feel that it really does make sense to keep BaseVendorLicenseInfoProvider completely separate from everything else then I also do not see that as an issue.
At the end of the day it is all about what you feel is right, everyone has their own opinion of course but as long as what you have works for you then I don't see a problem.
I want to "hot" load some pre-packaged assembli(es) into a separate AppDomain, the thing however is I do not know the name of the entry point class or even the assembly file. I need to find this entry point so I can run some initialization routine.
So what I intend to do is to run ReflectionOnlyLoad on all the files and find the one that follows a certain convention ie. annotated/implements a certain interface etc.
Question is, will I start leaking memory if I were to run ReflectionOnlyLoad from the main AppDomain over and over? If this can't be run from the main app domain, what are my options, because again I do not know where the entry point is.
Also any additional information about the subtleties in using ReflectionOnlyLoad is appreciated.
I recommend Mono.Cecil. It's a simple assembly you can use on .net (it doesn't require the Mono runtime). It offers an API to load assemblies as data, and works pretty well. I found the API easy to work with, and it suffered from none of the problems I experienced when using reflection-only-load.
You can also use CCI, which is an open source project by MS that offers an assembly reader.
See also: CCI vs. Mono.Cecil -- advantages and disadvantages
ReflectionOnlyLoad won't solve your problem, see docs
Why don't you execute the code for finding the entry point etc. in the new AppDomain?
Cannot reflect through the dlls. Even with reflection only load, the type sticks to the main AppDomain.
2 Solutions:
Put the entry point in an xml somewhere and parse that.
Use a
2 stage AppDomain, one for the reflector, and then another for the
actual object.
I picked (1) since it's the most sensible.
(2) I have to pass through 2 separate proxies in order to issue command to the actual remote object, that or I need to couple the interfaces much more closely than I like. Not to mention being a pain to code.
Short Version
I have an application which utilizes a plug-in infrastructure. The plug-ins have configurable properties that help them know how to do their job. The plug-ins are grouped into profiles to define how to complete a task, and the profiles are stored in XML files serialized by the DataContractSerializer. The problem is when reading the configuration files, the application deserializing has to have knowledge of all of the plug-ins defined in the configuration file. I'm looking for a way to handle the resolution of unknown plug-ins. See the proposed solution section below for a couple of the ideas I've looked into implementing, but I am open to just about anything (though I'd rather not have to reinvent the application).
Detail
Background
I've developed a sort of Business Process Automation System for internal use for the company I'm currently working for in C# 4. It makes exhaustive use of 'plug-ins' to define everything (from the tasks that are to be performed to the definition of units of work) and relies heavily on a dynamic configuration model which in turn relies on C# 4/DLR dynamic objects to fulfill jobs. It's a little heavy while executing because of its dynamic nature but it works consistently and performs well enough for our needs.
It includes a WinForms configuration UI that uses Reflection extensively to determine the configurable properties/fields of the plug-ins, as well as, the properties/fields that define each unit of work to be processed. The UI is also built on top of the BPA engine so it has a thorough understanding of the (loose) object model put in place that allows the engine to do its job, which, coincidentally, has led to several user experience improvements, such as, ad-hoc job execution and configure-time validation of user input. Again there is room for improvement, however, it seems to do its job.
The configuration UI utilizes the DataContractSerializer to serialize/deserialize the settings specified, so any plug-ins referenced by the configuration must be loaded before (or at the time of) configuration load.
Structure
The BPA engine is implemented as a shared assembly (DLL) which is referenced by the BPA service (a Windows Service), the Configuration UI (WinForms app), and a plug-in tester (Console application version of the Windows Service). Each of the three applications that reference the shared assembly only include the minimum amount of code necessary to perform their specific purpose. Additionally, all plug-ins must reference a very thin assembly which basically just defines the interface(s) that the plugin must implement.
Problem
Because of the extensibility model used in the application, there has always been a requirement that the config UI is run from the same directory (on the same PC) as the Service application. That way the UI always knows about all of the assemblies that the Service knows about so they can be deserialized without running into missing assemblies. Now that we are getting close to roll out of the system, a demand to allow the Configuration UI remotely on any PC in our network has come about from our network admins for security purposes. Typically this wouldn't be a problem if there was always a known set of assemblies to deploy, however, with the ability to extend the application using user built assemblies, there has to be a way to resolve the assemblies from which the plug-ins can be instantiated/used.
Proposed (potentially obvious) Solution
Add a WCF service to the Service application to allow the typical CRUD operations against the configurations which that instance of the service is aware of and rework the configuration UI to act more like SSMS with a Connect/Disconnect model. This doesn't really solve the problem so we would also need to expose some sort of ServiceContract from the Service application to allow querying of the assemblies it knows about/has access to. That's fine and fairly straight forward however the question arises, "When should the UI find out about the assemblies that the Service is aware of?" On connect we could send all of the assemblies from the Service to the UI to ensure that it always knows about all of the assemblies the service does but that gets messy with AppDomain management (potentially unnecessarily) and assembly version conflicts. So I suggested hooking into the AppDomain.AssemblyResolve/AppDomain.TypeResolve events to only download the assemblies that the client isn't aware of yet and only as needed. This doesn't necessarily cleanup the AppDomain management issues but it definitely helps address the version conflicts and related issues.
Question
If you've stuck with me this long I applaud and thank you, but now I'm finally getting to the actual question here. After months of research and finally coming to a conclusion I am wondering if anyone here has had to deal with a similar issue and how you dealt with the pitfalls and shortcomings? Is there a standard way of handling this that I have missed completely, or do you have any recommendations based on how you have seen this successfully handled in the past? Do you see any problems with the proposed approaches or can you offer an alternative?
I'm aware that not everyone lives in my head so please let me know if you need further clarification/explanation. Thanks!
Update
I've given MEF a fair shake and feel that it is too simplistic for my purposes. It's not that it couldn't be bent to handle the plug-in requirements of my application, the problem is doing so would be too cumbersome and dirty to make it feasible. It is a nice suggestion and it has a lot of potential, but in its current state it just isn't there yet.
Any other ideas or feedback on my proposed solutions?
Update
I don't know if the issue I'm encountering is just too localized, if I failed to properly describe what I am trying to achieve, or if this question is just too unreasonably long to be read in its entirety; but the few answers I've received have been subtly helpful enough to help me think through the problem differently and identify some shortcomings in what I am after.
In short, what I'm trying to do is take three applications which in their current state share information (configuration/assemblies) using a common directory structure, and try to make those applications work across a network with minimal impact on usability and architecture.
File shares seem like the obvious answer to this problem (as #SimonMourier proposed in the comments), but using them translates into lack of control and debugability when something goes wrong. I can see them as a viable short term solution, but long term they just don't seem feasible.
tl;dr, but I'm 90% sure you should take a look into MEF.
When I first saw it I was like "aah, another acronym", but you'll see it's very simple, and it's built in into .NET 4. Best of all, it even runs seamlessly on mono and it's a matter of less than an hour (including coffee break) between hearing about it and compiling hello worlds to get used with the features. It's really that simple.
Basically, you "export" something in an assembly and "import" it into another (all via simple attribute decorations), and you choose where to search for it (example, on the applications directory, plug-ins folder, etc).
Edit: what if you try to download and load (and possibly cache) plugins on-the-fly on configuration load?
I think that you could be overlooking a relatively simple solution that derives somewhat from the Microsoft web.config approach:
Have two sections in the config file:
Section 1 contains enough information about the plugin (i.e. name, version) to allow you to load it into an app domain.
Section 2 contains the information serialized by the plugin.
On loading the plugin, pass the information in section 2 and let the plugin deserialize it according to its needs.
Maybe you can divide this problem into two
administrator allow users to download one of predefined configuration (set of libraries) and MEF helps to inject required dependencies
each activity from user should pass through security proxy, plugin modules not allowed call BL directly. Proxy could match custom security attribute and allowed activities.
i.e.
[MyRole(Name = new[] { "Security.Action" })]
void BlockAccount(string accountId){}
[MyRole(Name = new[] { "Manager.Action" })]
void CreateAccount(string userName){}
[MyRole(Name = new[] { "Security.View", "Manager.View" })]
List<> AcountList(Predicate p){}
and allow for AD groups (some abstract description)
corp\securityOperators = "Security.*" //allow calls to all security manipulation
corp\HQmanager = "Manager.View" //allow only view access
corp\Operator = "Manager.*"
I'm not sure I completely understand the problem but I think this situation calls for "type-preserving serialization" - that is, the serialized file contains enough type information to deserialize back to the original object graph without any hints from the calling application as to what types are involved.
I've used Json.NET to do this and I can highly recommend the library for type-preserving serialization of object graphs. It looks like the NetDataContractSerializer can also do this, from the MSDN Remarks
The NetDataContractSerializer differs from the DataContractSerializer in one important way: the NetDataContractSerializer includes CLR type information in the serialized XML, whereas the DataContractSerializer does not. Therefore, the NetDataContractSerializer can be used only if both the serializing and deserializing ends share the same CLR types.
I chose Json.NET because it can serialize POCOs without any special attributes or interfaces. Both Json.NET and the NetDataContractSerializer allow you to use a custom SerializationBinder - in here you could put any logic regarding loading assemblies that may not yet be loaded.
Unfortunately, changing serialization schemes might be the "breaking-est" change to suggest because all your existing files will become incompatible. You might be able to write a conversion utility that deserializes a file using the old method and serializes the resulting object graph using the new method.
I know how to branch the code based on Mono (Type.GetType("Mono.Runtime") != null) but even when the Mono code path is taken, Mono is attempting to load assemblies that would be required by the non-Mono code path. This is not all that surprising, but how do I get around the problem? I have tried putting the call to the non-Mono assembly in a different class, but that didn't help.
The only option to do it directly is Reflection all the way, so far as I can see.
I'd suggest a more roundabout approach: refactor all your code that is dependent on Mono or .NET into separate assemblies, one for each platform - let's call them MA and NA. Make sure that the entire API surface of your classes there is covered by common interfaces, which should be in the 3rd assembly, IA. After that, your main application references IA for interfaces, and uses Reflection just once to load either MA or NA depending on whether it's running on Mono or .NET, and obtain the instance of "top-level factory class". Once there, it just uses normal calls via IA interfaces to instantiate all other objects via that factory and work with them.
Expanding on Pavel's answer you can use a plugin framework to help with the conditionality of loading bits of code that are specific to a platform. You can use Mono.Addins or MS' own open sourced Managed Extensibility Framework known as MEF (http://www.codeplex.com/MEF)
Don't add the reference in the command-line compiler options. If you are using a high level IDE tool then you might have to play with its project settings to effect the same thing.
There are other files that come into play too like AssemblyInfo.cs and might contain instructions about assemblies that you are considering. Also the program might be using types from App.Config (Configuration file) or Web.Config (ASP.NET) / dynamic type loading.
Don't rely for your dependencies on the fact that your code is JITted and that only called code is JITted.
Best is always to assume, that whatever is referenced will be loaded and has to be available.
You user might choose to use AOT, which is Mono's counterpart of NGEN.
Or subtle differences in how newer runtime versions handle things like serialization, remoting, security, reflection, etc. can lead to your references being loaded even your code does not use anything directly. (But the serializer might have pulled all types, which then loaded other assemblies)
Use interfaces or classic inheritance, or maybe events or other means of indirection to load the .Net parts only when they are appropriate. And by hat I mean an assembly that you don't reference but load dynamically.