I'm currently working on a large real-time OLAP application. All data are hold in RAM (a few gigabytes) and the common tasks involve brute scanning over the large quantity of that data (which is fine). The results of processing are exposed via a Web service (singleton/multithreaded) and presented using Silverlight-based client.
The problem is that various customers need different functionality/algorithms and I don't know how to provide extensibility on the server-side. For the client side (Silverlight) I can use MEF/Prism, but I'm not sure what would be a good approach to tackle this problem on the server.
Please note that ideally other web-services should have a direct access (i.e. without marshaling) to the data of the main/current service which holds the large data model.
Are there any:
a) frameworks/libraries
b) patterns
c) good pracitces
which would help me to modularize the application and make the selection of desired modules and their deployment relatively easy?
Sounds to me like Dependency Inversion is required: isolate logical parts of the system (algorithms, etc) by defining interfaces, then use a DI / IoC framework to load the desired implementation at runtime (or on application start, etc).
I haven't used Ninject, but plenty of people love it, so you could try that; there's also Spring.Net.
Good Practices:
Ensure you have clear precise logging so you know what's being used and when.
Think about whether you want a 'default' implementation to load if the desired one fails, or whether you deliberately want to fail so that the wrong data isn't returned by mistake (such as the use of a different algorythm).
I've found that using attributes to decorate injectable modules is really helpful (especially in a web-based system that you don't have immeadiate access to) one reason for this is that you can build pages or controls that list all the known / available implementations at runtime.
You can also use the attribute approach to build a UI that lets users select which one they want; I use it for an open source web-application framework I built: http://www.morphological.geek.nz/Morphfolia/Capabilities/AttributeDriven.aspx
Related
I think elements of this question have been answered elsewhere, but I couldn't find an answer to my specific circumstance.
I work with an enterprise application. This application interfaces with various 3rd party APIs & services via what is currently a single class and a plethora of proxy dlls. This means that each of these dlls is referenced in the main project. In addition to this, over time as we've added new service calls, a lot of code has been duplicated, with only very small amendments. Most of the service calls do roughly the same thing and take largely the same objects as parameters.
As you can imagine, this presents us with a number of problems, not least of which is the time it takes to add a new service.
We have a task now to refactor and streamline this process to make it more manageable and resilient - I have an idea in my head and I've done a lot of research but I just wanted to see if anyone had any better ideas or similar experiences before I dive in.
What I want to do is add a facade layer so that the base code gets all the data commonly used and bundles it off to the facade along with a parameter stipulating which service it wants to call. The facade would then pass the data to either the proxies or a bridge, which would transform it into the right format for the target service, make the required calls in order, and return any responses back to the facade to pass on.
Although I have an idea of the architecture I want, I'm not 100% sure which way to go in terms of concrete C# code - whether to add the facade and bridge/adapter code in a new project which also has the proxy dlls referenced, or whether to go down the base/interface route and add the required transformation classes directly into the proxy dlls.
Edited to add: I am unable to consider consolidating this functionality into a service or microservice due to wider infrastructure considerations that I am unable to discuss here.
Any suggestions appreciated!
I have a winforms app. I give it to three clients and each one wants a small tweak or customization specific just to them. To accomplish this, I'd have to keep a separate version just for each client. I may wind up having many versions doing it this way. I thought dependency injection would be how to handle this but I hear you have to register your dependencies in the main method and you'd still have to add a reference to each clients DLL so I'd still need different versions. What is the preferred object oriented way to handle this? Any better ways to handle this?
You can use a Plug-in patten to load assembly at runtime: (from link)
Separated Interface (476) is often used when application code runs in multiple runtime environments, each requiring different implementations of particular behavior.
Most DI frameworks provide this functionality. You can search at get lots of Examples for framework you choose - if you don't want to roll your own.
Ninject
MEF
You can use a configuration file to configure your DI container, so that you can reuse the same binaries with different configuration files to implement the different customizations. But you need to be sure that you thoroughly test all of your different configurations. Slightly different versions of the same application are not trivial to maintain without causing unanticipated breaks.
Depending on the nature of the customixations, you might be able to capture all relevant modifications into a distinct part of the project (as opposed to keeping them spread all-over the project). If you can (e.g. a filering functionality is provided by the client), you can then load a DLL dynamically (e.g. based on a config file) and allow the functions in the DLL to perform the necessary functionality that accomplishes the customization (based on parameters provided by the main code).
This way you provide pre-defined hooks to your code that can be changed dynamically (even if only to load the dlls at startup time) as per the need of the client. You can separate these DLLs into multiple ones if there are distinct features that the clients want to change, but not necessarily all of the clinets all of the features. Then you can provide a "default" version of the DLLs.
Who develops the hooks is dependent on your setup with the clients.
Make sure you provide adequate documentation on how these hooks supposed to work -- even if you end up developing them.
Short Version
I have an application which utilizes a plug-in infrastructure. The plug-ins have configurable properties that help them know how to do their job. The plug-ins are grouped into profiles to define how to complete a task, and the profiles are stored in XML files serialized by the DataContractSerializer. The problem is when reading the configuration files, the application deserializing has to have knowledge of all of the plug-ins defined in the configuration file. I'm looking for a way to handle the resolution of unknown plug-ins. See the proposed solution section below for a couple of the ideas I've looked into implementing, but I am open to just about anything (though I'd rather not have to reinvent the application).
Detail
Background
I've developed a sort of Business Process Automation System for internal use for the company I'm currently working for in C# 4. It makes exhaustive use of 'plug-ins' to define everything (from the tasks that are to be performed to the definition of units of work) and relies heavily on a dynamic configuration model which in turn relies on C# 4/DLR dynamic objects to fulfill jobs. It's a little heavy while executing because of its dynamic nature but it works consistently and performs well enough for our needs.
It includes a WinForms configuration UI that uses Reflection extensively to determine the configurable properties/fields of the plug-ins, as well as, the properties/fields that define each unit of work to be processed. The UI is also built on top of the BPA engine so it has a thorough understanding of the (loose) object model put in place that allows the engine to do its job, which, coincidentally, has led to several user experience improvements, such as, ad-hoc job execution and configure-time validation of user input. Again there is room for improvement, however, it seems to do its job.
The configuration UI utilizes the DataContractSerializer to serialize/deserialize the settings specified, so any plug-ins referenced by the configuration must be loaded before (or at the time of) configuration load.
Structure
The BPA engine is implemented as a shared assembly (DLL) which is referenced by the BPA service (a Windows Service), the Configuration UI (WinForms app), and a plug-in tester (Console application version of the Windows Service). Each of the three applications that reference the shared assembly only include the minimum amount of code necessary to perform their specific purpose. Additionally, all plug-ins must reference a very thin assembly which basically just defines the interface(s) that the plugin must implement.
Problem
Because of the extensibility model used in the application, there has always been a requirement that the config UI is run from the same directory (on the same PC) as the Service application. That way the UI always knows about all of the assemblies that the Service knows about so they can be deserialized without running into missing assemblies. Now that we are getting close to roll out of the system, a demand to allow the Configuration UI remotely on any PC in our network has come about from our network admins for security purposes. Typically this wouldn't be a problem if there was always a known set of assemblies to deploy, however, with the ability to extend the application using user built assemblies, there has to be a way to resolve the assemblies from which the plug-ins can be instantiated/used.
Proposed (potentially obvious) Solution
Add a WCF service to the Service application to allow the typical CRUD operations against the configurations which that instance of the service is aware of and rework the configuration UI to act more like SSMS with a Connect/Disconnect model. This doesn't really solve the problem so we would also need to expose some sort of ServiceContract from the Service application to allow querying of the assemblies it knows about/has access to. That's fine and fairly straight forward however the question arises, "When should the UI find out about the assemblies that the Service is aware of?" On connect we could send all of the assemblies from the Service to the UI to ensure that it always knows about all of the assemblies the service does but that gets messy with AppDomain management (potentially unnecessarily) and assembly version conflicts. So I suggested hooking into the AppDomain.AssemblyResolve/AppDomain.TypeResolve events to only download the assemblies that the client isn't aware of yet and only as needed. This doesn't necessarily cleanup the AppDomain management issues but it definitely helps address the version conflicts and related issues.
Question
If you've stuck with me this long I applaud and thank you, but now I'm finally getting to the actual question here. After months of research and finally coming to a conclusion I am wondering if anyone here has had to deal with a similar issue and how you dealt with the pitfalls and shortcomings? Is there a standard way of handling this that I have missed completely, or do you have any recommendations based on how you have seen this successfully handled in the past? Do you see any problems with the proposed approaches or can you offer an alternative?
I'm aware that not everyone lives in my head so please let me know if you need further clarification/explanation. Thanks!
Update
I've given MEF a fair shake and feel that it is too simplistic for my purposes. It's not that it couldn't be bent to handle the plug-in requirements of my application, the problem is doing so would be too cumbersome and dirty to make it feasible. It is a nice suggestion and it has a lot of potential, but in its current state it just isn't there yet.
Any other ideas or feedback on my proposed solutions?
Update
I don't know if the issue I'm encountering is just too localized, if I failed to properly describe what I am trying to achieve, or if this question is just too unreasonably long to be read in its entirety; but the few answers I've received have been subtly helpful enough to help me think through the problem differently and identify some shortcomings in what I am after.
In short, what I'm trying to do is take three applications which in their current state share information (configuration/assemblies) using a common directory structure, and try to make those applications work across a network with minimal impact on usability and architecture.
File shares seem like the obvious answer to this problem (as #SimonMourier proposed in the comments), but using them translates into lack of control and debugability when something goes wrong. I can see them as a viable short term solution, but long term they just don't seem feasible.
tl;dr, but I'm 90% sure you should take a look into MEF.
When I first saw it I was like "aah, another acronym", but you'll see it's very simple, and it's built in into .NET 4. Best of all, it even runs seamlessly on mono and it's a matter of less than an hour (including coffee break) between hearing about it and compiling hello worlds to get used with the features. It's really that simple.
Basically, you "export" something in an assembly and "import" it into another (all via simple attribute decorations), and you choose where to search for it (example, on the applications directory, plug-ins folder, etc).
Edit: what if you try to download and load (and possibly cache) plugins on-the-fly on configuration load?
I think that you could be overlooking a relatively simple solution that derives somewhat from the Microsoft web.config approach:
Have two sections in the config file:
Section 1 contains enough information about the plugin (i.e. name, version) to allow you to load it into an app domain.
Section 2 contains the information serialized by the plugin.
On loading the plugin, pass the information in section 2 and let the plugin deserialize it according to its needs.
Maybe you can divide this problem into two
administrator allow users to download one of predefined configuration (set of libraries) and MEF helps to inject required dependencies
each activity from user should pass through security proxy, plugin modules not allowed call BL directly. Proxy could match custom security attribute and allowed activities.
i.e.
[MyRole(Name = new[] { "Security.Action" })]
void BlockAccount(string accountId){}
[MyRole(Name = new[] { "Manager.Action" })]
void CreateAccount(string userName){}
[MyRole(Name = new[] { "Security.View", "Manager.View" })]
List<> AcountList(Predicate p){}
and allow for AD groups (some abstract description)
corp\securityOperators = "Security.*" //allow calls to all security manipulation
corp\HQmanager = "Manager.View" //allow only view access
corp\Operator = "Manager.*"
I'm not sure I completely understand the problem but I think this situation calls for "type-preserving serialization" - that is, the serialized file contains enough type information to deserialize back to the original object graph without any hints from the calling application as to what types are involved.
I've used Json.NET to do this and I can highly recommend the library for type-preserving serialization of object graphs. It looks like the NetDataContractSerializer can also do this, from the MSDN Remarks
The NetDataContractSerializer differs from the DataContractSerializer in one important way: the NetDataContractSerializer includes CLR type information in the serialized XML, whereas the DataContractSerializer does not. Therefore, the NetDataContractSerializer can be used only if both the serializing and deserializing ends share the same CLR types.
I chose Json.NET because it can serialize POCOs without any special attributes or interfaces. Both Json.NET and the NetDataContractSerializer allow you to use a custom SerializationBinder - in here you could put any logic regarding loading assemblies that may not yet be loaded.
Unfortunately, changing serialization schemes might be the "breaking-est" change to suggest because all your existing files will become incompatible. You might be able to write a conversion utility that deserializes a file using the old method and serializes the resulting object graph using the new method.
I'm working on a system right now that is in a single process space; we are breaking this up into several processes, initially to run on the same box but ultimately to distribute across several separate machines. I'm leaning towards using an ESB (NServiceBus, Rhino ESB) or possibly rolling my own with WCF + queues to handle the pub/sub and request/response scenarios our app has.
However, I'm struggling with the abstraction: I don't want the various components to know they are talking over the bus. The current APIs connecting the various services translate pretty well to this kind of model, but I want to hide that from the client and server sides. Short of writing a lot of custom proxy code for the client and server, is there a better way to approach this? I realize WCF can auto-generate proxies based on the service definition, but I really like some of the other stuff I get with (say) rhino servicebus.
Ideally, I'd like to be able to swap out different implementations (with and without an ESB/messaging layer) just using IoC (knowing there would have to be limits enforced by convention on what can be passed across the interfaces), but I'm not sure where to go with that. I'd really prefer to not have to change every method call on the current interfaces into its own discrete message class, either.
Any resources/patterns/tools to help me do this? Please ask questions if I'm not clear. Thanks.
There may not be one solution/off-the-shelf component that might help you.
Problem 1:
The basic problem can be solved via an ESB, as it provides location transparency and service aggregation. A regular ESB mediates/brokers requests between service consumer and service provider.
Take a simple example:
Service_A depends on Service_B
Service_C depends on Service_B
Service_B depends on Service_D
In this scenario, the best way to progress is this:
Define contracts exposed by Service_B and Service_D as external dependencies (possibly as a web service, though an ESB supports multiple protocols) in services Service_A, Service_C and Service_B, and consume via an ESB.
In ESB, to start with, route thes services Service_B and Service_D on the same instance.
If you migrate Service_D and Service_B as Service_Dx and Service_Bx on a different location, the ESB can be reconfigured to route to the new location. Also, an ESB can be configured to route to Service_B or Service_Bx based on some set of parameters (eg., test data to Service_B and production data to Service_Bx)
Problem 2:
The problem of IOC could probably be hard to achieve; there may not be a need.
I presume the clients, instead of consuming from a known location, are injected with the whereabouts of service location. This in reality transfers the configuration to client side. With this, for every new client added to the system there needs to be a separate configuration control. This might lead to logistical issues.
Please post your final solution, very interested to know your approach.
We have a server written in C# (Framework 3.5 SP1). Customers write client applications using our server API. Recently, we created several levels of license schemes like Basic, Intermediate and All. If you have Basic license then you can call few methods on our API. Similarly if you have Intermediate you get some extra methods to call and if you have All then you can call all the methods.
When server starts it gets the license type. Now in each method I have to check the type of license and decide whether to proceed further with the function or return.
For example, a method InterMediateMethod() can only be used by Intermediate License and All license. So I have to something like this.
public void InterMediateMethod()
{
if(licenseType == "Basic")
{
throw new Exception("Access denied");
}
}
It looks like to me that it is very lame approach. Is there any better way to do this? Is there any declarative way to do this by defining some custom attributes? I looked at creating a custom CodeAccessSecurityAttribute but did not get a good success.
Since you are adding the "if" logic in every method (and god knows what else), you might find it easier to use PostSharp (AOP framework) to achieve the same, but personally, I don't like either of the approaches...
I think it would be much cleaner if you'd maintained three different branches (source code) for each license, which may add a little bit of overhead in terms of maintenance (maybe not), but at least keep it clean and simple.
I'm also interested what others have to say about it.
Good post, I like it...
Possibly one easy and clean approach would be to add a proxy API that duplicates all your API methods and exposes them to the client. When called, the proxy would either forward the call to your real method, or return a "not licensed" error. The proxies could be built into three separate (basic, intermediate, all) classes, and your server would create instances of the approprate proxy for your client's licence. This has the advantage of having minimal performance overhead (because you only check the licence once). You may not even need to use a proxy for the "all" level, so it'll get maximum performance. It may be hard to slip this in depending on your existing design though.
Another possibility may be to redesign and break up your APIs into basic/intermediate/all "sections", and put them in separate assemblies, so the entire assembly can be enabled/disabled by the licence, and attempting to call an unlicensed method can just return a "method not found" error (e.g. a TypeLoadException will occur automatically if you simply haven't loaded the needed assembly). This will make it much easier to test and maintain, and again avoids checking at the per-method level.
If you can't do this, at least try to use a more centralised system than an "if" statement hand-written into every method.
Examples (which may or may not be compatible with your existing design) would include:
Add a custom attribute to each method and have the server dispatch code check this attribute using reflection before it passes the call into the method.
Add a custom attribute to mark the method, and use PostSharp to inject a standard bit of code into the method that will read and test the attribute against the licence.
Use PostSharp to add code to test the licence, but put the licence details for each method in a more data driven system (e.g. use an XML file rather than attributes to describe the method permissions). This will allow you to easily change the licensing across the entire server by editing a single file, and allow you to easily add whole new levels or types of licences in future.
Hope that gives you some ideas.
You might really want to consider buying a licensing solution rather than rolling your own. We use Desaware and are pretty happy with it.
Doing licensing at the method level is going to take you into a world of hurt. Maintenance on that would be a nightmare, and it won't scale at all.
You should really look at componentizing your product. Your code should roughly fall into "features", which can be bundled into "components". The trick is to make each component do a license check, and have a licensing solution that knows if a license includes a component.
Components for our products are generally on the assembly level, but for our web products they can get down to the ASP.Net server control level.
I wonder how the people are licensing the SOA services. They can be licensed per service or per end point.
That can be very hard to maintain.
You can try with using strategy pattern.
This can be your starting point.
I agree with the answer from #Ostati that you should keep 3 branches of your code.
What I would further expand on that is then I would expose 3 different services (preferably WCF services) and issue certificates that grant access to the specific service. That way if anyone tried to access the higher level functionality they would just not be able to access the service period.