Are MEF exports cached or discovering every time on request? - c#

If I have one type MyClass, register with
[Export(typeof(Myclass))]
attribute, and
[PartCreationPolicy(CreationPolicy.Shared)]
or
[PartCreationPolicy(CreationPolicy.NonShared)]
and later trying to call
compositionContainer.GetExportedValue<Myclass>() multiple times.
Question: with the first call, I will get my registered class via MEF - llokup all registered assemblies, then trying to find one registered contract. Question is about second time and so on - will MEF do global lookup again or it caches somewhere internally?

will MEF do global lookup again or it caches somewhere internally
Yes, MEF perfoms some caching and widely uses lazy initialization, if you question is about MEF performance:
1) metadata (composable parts, export definitions and import definitions) is cached. Example:
public override IEnumerable<ExportDefinition> ExportDefinitions
{
get
{
if (this._exports == null)
{
ExportDefinition[] exports = this._creationInfo.GetExports().ToArray<ExportDefinition>();
lock (this._lock)
{
if (this._exports == null)
{
this._exports = exports;
}
}
}
return this._exports;
}
}
2) exported values are cached too:
public object Value
{
get
{
if (this._exportedValue == Export._EmptyValue)
{
object exportedValueCore = this.GetExportedValueCore();
Interlocked.CompareExchange(ref this._exportedValue, exportedValueCore, Export._EmptyValue);
}
return this._exportedValue;
}
}
Of course, when using CreationPolicy.NonShared, exported value becomes created again and again, when you requesting it. But even in this case "global lookup" isn't performed, because metadata is cached anyway.

It does a lookup every time, when you use [PartCreationPolicy(CreationPolicy.NonShared)]. You then have to implement the caching yourself.
The default implementation is using a Singleton pattern. This equals the attribute [PartCreationPolicy(CreationPolicy.Shared)]. This is the best practice.
For more information, read http://blogs.microsoft.co.il/blogs/bnaya/archive/2010/01/09/mef-for-beginner-part-creation-policy-part-6.aspx

Although the values/metadata might be partially cached, doing some performance testing shows that some lookup is performed every time a call to GetExportedValue is made. So if you have many calls where you need to get the value, you should do the caching yourself.
namespace MEFCachingTest
{
using System;
using System.ComponentModel.Composition;
using System.ComponentModel.Composition.Hosting;
using System.ComponentModel.Composition.Primitives;
using System.Diagnostics;
using System.Reflection;
public static class Program
{
public static CompositionContainer Container { get; set; }
public static ComposablePartCatalog Catalog { get; set; }
public static ExportedClass NonCachedClass
{
get
{
return Container.GetExportedValue<ExportedClass>();
}
}
private static ExportedClass cachedClass;
public static ExportedClass CachedClass
{
get
{
return cachedClass ?? (cachedClass = Container.GetExportedValue<ExportedClass>());
}
}
public static void Main()
{
Catalog = new AssemblyCatalog(Assembly.GetExecutingAssembly());
Container = new CompositionContainer(Catalog);
const int Runs = 1000000;
var stopwatch = new Stopwatch();
// Non-Cached.
stopwatch.Start();
for (int i = 0; i < Runs; i++)
{
var ncc = NonCachedClass;
}
stopwatch.Stop();
Console.WriteLine("Non-Cached: Time: {0}", stopwatch.Elapsed);
// Cached.
stopwatch.Restart();
for (int i = 0; i < Runs; i++)
{
var cc = CachedClass;
}
stopwatch.Stop();
Console.WriteLine(" Cached: Time: {0}", stopwatch.Elapsed);
}
}
[Export]
[PartCreationPolicy(CreationPolicy.Shared)]
public class ExportedClass
{
}
}
For more variations, look at this gist: https://gist.github.com/DanielRose/d79f0da2ef61591176ce
On my computer, Windows 7 x64, .NET 4.5.2:
Non-Cached: Time: 00:00:02.1217811
Cached: Time: 00:00:00.0063479
Using MEF 2 from NuGet:
Non-Cached: Time: 00:00:00.2037812
Cached: Time: 00:00:00.0023358
In the actual application where I work, this made the application 6x slower.

Related

Is there a design pattern for one input passed to n methods that each return the input for the next method

I'd like to know if there is a design pattern for this problem:
One input is used to construct an object (via constructor or via method return I don't care), that object is feed to the next method or constructor. This is repeated on a user specified set of processors, obviously throwing exceptions if there is a break in the chain of required inputs for the processors.
The output of all, some or none of the implemented processors is the same object.
I've got about 6 processors planned, possibly more in future.
Composition:
I'm not sure I like the composition design pattern because not every object is intended to be an output of this process and I can't think how to not output null values without the user knowing it's going to be null.
Chain of responsibility:
Chain of responsibility is the way to go according to what I've heard however I'm not sure i understand it. Is this design pattern suggesting to pass n function pointers to a function that runs through each? If so, I'm not sure how to setup the function that gets passed n function pointers.
my attempt:
I've got two interface that are inherited by n classes ie (FirstProcessor, FirstInput, FirstOutput, SecondProcessor, SecondOutput, ThirdProcessor,.., NProcessor, NOutput)
IChainedOutput
{
IChainedOutput Input {get;}
FinalOutputOBj GetFinalOutput()
}
IChainedProcessor
{
IChainedOutput Run(IChainedOutput previousOutput)
}
used like this:
IChainedProcessor previous = FirstProcessor(originalInput)
foreach(IChainedProcessor processor in processorList.Skip(1)
{
IChainedOutput current = processor.Run(previous)
previous = current;
}
FinalOutputObj output = previous.GetFinalOutput();
Problems:
FinalOutputObj is coupled with all the processor implementations which is bad. It's not comprised of all the IChainedOutput child class members but uses a good subset to calculate other values.
FinalOutputObj is being composed in a bad way and I don't see how I can escape from outputting null values if the list of processors does not contain every processor implemented.
There is a lot of downcasting in the constructors of the child classes which is a red flag for oop. However, the inputs for each block of processing logic are completely different. First input is a couple of vectors, second input is the output of the first which includes a handful of custom types and more vectors, etc
Each IChainedOutput contains the reference to the inputs used to create it. currently there is one to one mapping input to processor but i'm not sure in future. And this is more bad downcasting.
I'd like to not have to perfectly organise the list of processors, makes it too easy for other developers to make mistakes here. so the next one selected should be the one that has the correct constructor.
You could try a decorator approach like this:
public interface IChainProcessor
{
IChainOutput Run(IChainOutput previousOutput);
}
public interface IChainOutput
{
string Value { get; }
}
public class OutputExample : IChainOutput
{
public string Value { get; }
public OutputExample(string value)
{
this.Value = value;
}
}
public abstract class Processor : IChainProcessor
{
protected IChainProcessor nextProcessor;
public IChainOutput Run(IChainOutput previousOutput)
{
var myOutput = this.MyLogic(previousOutput);
return this.nextProcessor == null ? myOutput : this.nextProcessor.Run(myOutput);
}
protected abstract IChainOutput MyLogic(IChainOutput input);
}
public class ProcessorA : Processor
{
public ProcessorA() { }
public ProcessorA(ProcessorB nextProcessor)
{
this.nextProcessor = nextProcessor;
}
protected override IChainOutput MyLogic(IChainOutput input)
{
return new OutputExample($"{input.Value} + Processor_A_Output");
}
}
public class ProcessorB : ProcessorA
{
public ProcessorB() { }
public ProcessorB(ProcessorC nextProcessor)
{
this.nextProcessor = nextProcessor;
}
protected override IChainOutput MyLogic(IChainOutput input)
{
return new OutputExample($"{input.Value} + Processor_B_Output");
}
}
public class ProcessorC : ProcessorB
{
protected override IChainOutput MyLogic(IChainOutput input)
{
return new OutputExample($"{input.Value} + Processor_C_Output");
}
}
The usage would be something like the below:
private static int Main(string[] args)
{
var chain = new ProcessorA(new ProcessorB(new ProcessorC()));
var simpleChain = new ProcessorA(new ProcessorC());
var verySimpleChain = new ProcessorA();
var initialInput = new OutputExample("Start");
Console.WriteLine(chain.Run(initialInput).Value);
Console.WriteLine(simpleChain.Run(initialInput).Value);
Console.WriteLine(verySimpleChain.Run(initialInput).Value);
return 0;
}
The output of this example is:
Start + Processor_A_Output + Processor_B_Output + Processor_C_Output
Start + Processor_A_Output + Processor_C_Output
Start + Processor_A_Output
The abstract Processor class provides a template method that you can implement in subclasses. So every ProcessorX class only defines MyLogic(IChainOutput input)
The Processors extend each other to enforce compile time preservation of processor order. So it is impossible to build a chain where ProcessorB comes before ProcessorA. It is possible though to build a chain that omits some processors as in the above example.
The example I provide here does not cater for the final output, which I know is one of your main concerns. To deal with issue I would rather build a mapping class to convert IChainOutput into the final format (I don't know the real structure of your data so maybe this is not possible).
in some of my cases it would make sense to have the output of one processor be the input for multiple other processors
Using this pattern it would also be possible to construct a processor 'tree' rather than a chain, by allowing the Processor class to have a list of next steps. Your usage would then become something like this:
var chain = new ProcessorA(new ProcessorB(new ProcessorC()), new ProcessorB(new ProcessorD()));
I hope this can help you.
If I understood your explanation correctly you can use delegates to overcome your problem. One of the important point about delegates is that they can be chained together so that you can call any number of methods in a single event.
Each processor transforming specific input into a specific output. Therefore the processor implementation should know only two types.
public interface IStepProcessor<TInput, TOutput>
{
TOutput Process(TInput input);
}
The client code ideally should know only two type of data that is input data and final product. The client code don't care if there were some intermediary steps in the middle. Client make use the conveyor as a black box
public delegate TOutput Conveyor<TInput, TOutput>(TInput input);
Yet some external code should understand how the whole transformation is done. This code should know all the intermediate data types and have access to all intermediate processors. It is done best with dependency injection.
public class Factory
{
private readonly IStepProcessor<IDataInput, IDataStep1> m_Step1;
private readonly IStepProcessor<IDataStep1, IDataStep2> m_Task2;
private readonly IStepProcessor<IDataStep2, IDataStep3> m_Task3;
private readonly IStepProcessor<IDataStep3, IDataStepN> m_TaskN;
private readonly IStepProcessor<IDataStepN, IDataOutput> m_FinalTask;
public Factory(
IStepProcessor<IDataInput, IDataStep1> task1,
IStepProcessor<IDataStep1, IDataStep2> task2,
IStepProcessor<IDataStep2, IDataStep3> task3,
IStepProcessor<IDataStep3, IDataStepN> taskN,
IStepProcessor<IDataStepN, IDataOutput> finalTask
)
{
m_Step1 = task1;
m_Task2 = task2;
m_Task3 = task3;
m_TaskN = taskN;
m_FinalTask = finalTask;
}
public Conveyor<IDataInput, IDataOutput> BuildConveyor()
{
return (input) =>
{
return m_FinalTask.Process(
m_TaskN.Process(
m_Task3.Process(
m_Task2.Process(
m_Step1.Process(input)))));
};
}
}
Here goes my offer
public interface IDataInput { }
public interface IDataStep1 { }
public interface IDataStep2 { }
public interface IDataStep3 { }
public interface IDataStepN { }
public interface IDataOutput { }
public interface IStepProcessor<TInput, TOutput>
{
TOutput Process(TInput input);
}
public delegate TOutput Conveyor<TInput, TOutput>(TInput input);
public class Factory
{
private readonly IStepProcessor<IDataInput, IDataStep1> m_Step1;
private readonly IStepProcessor<IDataStep1, IDataStep2> m_Task2;
private readonly IStepProcessor<IDataStep2, IDataStep3> m_Task3;
private readonly IStepProcessor<IDataStep3, IDataStepN> m_TaskN;
private readonly IStepProcessor<IDataStepN, IDataOutput> m_FinalTask;
public Factory(
IStepProcessor<IDataInput, IDataStep1> task1,
IStepProcessor<IDataStep1, IDataStep2> task2,
IStepProcessor<IDataStep2, IDataStep3> task3,
IStepProcessor<IDataStep3, IDataStepN> taskN,
IStepProcessor<IDataStepN, IDataOutput> finalTask
)
{
m_Step1 = task1;
m_Task2 = task2;
m_Task3 = task3;
m_TaskN = taskN;
m_FinalTask = finalTask;
}
public Conveyor<IDataInput, IDataOutput> BuildConveyor()
{
return (input) =>
{
return m_FinalTask.Process(
m_TaskN.Process(
m_Task3.Process(
m_Task2.Process(
m_Step1.Process(input)))));
};
}
}
public class Client
{
private readonly Conveyor<IDataInput, IDataOutput> m_Conveyor;
public Client(Conveyor<IDataInput, IDataOutput> conveyor)
{
m_Conveyor = conveyor;
}
public void DealWithInputAfterTransformingIt(IDataInput input)
{
var output = m_Conveyor(input);
Console.Write($"Mind your business here {typeof(IDataOutput).IsAssignableFrom(output.GetType())}");
}
}
public class Program {
public void StartingPoint(IServiceProvider serviceProvider)
{
ISomeDIContainer container = CreateDI();
container.Register<IStepProcessor<IDataInput, IDataStep1>, Step1Imp>();
container.Register<IStepProcessor<IDataStep1, IDataStep2>, Step2Imp>();
container.Register<IStepProcessor<IDataStep2, IDataStep3>, Step3Imp>();
container.Register<IStepProcessor<IDataStep3, IDataStepN>, StepNImp>();
container.Register<IStepProcessor<IDataStepN, IDataOutput>, StepOImp>();
container.Register<Factory>();
Factory factory = container.Resolve<Factory>();
var conveyor = factory.BuildConveyor();
var client = new Client(conveyor);
}
}

Thread Safety with less locking [duplicate]

This question already has answers here:
Closed 10 years ago.
Possible Duplicate:
C# Threading & Blocking
I am trying to effectively determine which approach is better:
Currently, I have a singleton instance that exposes entities that are loaded in lazy load fashion. I have listed three approaches which each of which has some advantages. The first approach relies solely on double lock pattern to ensure thread safety. The second approach doesn't use locking but it has the potential of double Load in case of a race. The third approach really uses a solution that I am becoming very fond of. (System.Lazy).
For some reason, I feel there is something wrong with the second approach (System.Thread.InterLocked), yet i can't pin point it. Is there a reason to favor one approach over the other? I did cover this in a previous post where I felt the third option is the way to go from now on.
I stripped the code to the barebones to be able explain the design.
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
namespace TPLDemo
{
public class SomeEntity
{
}
public class MultiThreadedManager
{
private static readonly System.Lazy<MultiThreadedManager> instance = new Lazy<MultiThreadedManager>(() => { return new MultiThreadedManager(); });
private readonly object _syncRoot = new object();
private List<SomeEntity> _inMemoryEntities = null;
private List<SomeEntity> _inMemoryEntitiesUsingLockFreeApproach = null;
private System.Lazy<List<SomeEntity>> _inMemoryUsingLazy = new Lazy<List<SomeEntity>>(() => { return MultiThreadedManager.Instance.LoadFromSomewhere(); });
public static MultiThreadedManager Instance
{
get { return instance.Value; }
}
public IEnumerable<SomeEntity> LazyEntities
{
get
{
return _inMemoryUsingLazy.Value;
}
}
public IEnumerable<SomeEntity> LocklessEntities
{
get
{
if (_inMemoryEntitiesUsingLockFreeApproach == null)
{
do
{
// Is it possible multiple threads hit this at the same time?
} while (System.Threading.Interlocked.CompareExchange<List<SomeEntity>>(ref _inMemoryEntitiesUsingLockFreeApproach, this.LoadFromSomewhere(), null) != null);
}
return _inMemoryEntitiesUsingLockFreeApproach;
}
}
/// <summary>
/// This is thread safe but it involved some locking.
/// </summary>
public IEnumerable<SomeEntity> Entities
{
get
{
if (_inMemoryEntities == null)
{
lock (_syncRoot)
{
if (_inMemoryEntities == null)
{
List<SomeEntity> list = this.LoadFromSomewhere();
_inMemoryEntities = list;
}
}
}
return _inMemoryEntities;
}
}
private List<SomeEntity> LoadFromSomewhere()
{
return new List<SomeEntity>();
}
public void ReloadEntities()
{
// This is sufficient becasue any subsequent call will reload them safely.
_inMemoryEntities = null;
// This is sufficient becasue any subsequent call will reload them safely.
_inMemoryEntitiesUsingLockFreeApproach = null;
// This is necessary becasue _inMemoryUsingLazy.Value is readonly.
_inMemoryUsingLazy = new Lazy<List<SomeEntity>>(() => { return MultiThreadedManager.Instance.LoadFromSomewhere(); });
}
}
}
The third option (Lazy) allows you to configure how it should behave. You can make it behave like (1) or like (2).
In any case, once it is loaded it does not need to lock or interlock internally to hand you back the loaded Value.
So by all means go for System.Lazy.
There is one nasty thing though: If the factory function fails, the exception is stored and thrown everytime the Value property is accessed. This means that this Lazy instance is not ruined. You cannot ever retry. This means that a transient failure (network error, ...) might permanently take down your application until it is manually restarted.
If have complained about this on MS Connect but it is by design.
My solution was to write my own Lazy. It's not hard.

C# making app support plugins, run multiple instances, possible?

Basically what I need is being able to add new functionality to my application, without updating the application itself.
Lets say I have an application with two plugins.
Application has a function called generateNumber();
Plugin_1 would have the following code:
void init(){ }
int exec(){
int number = HOST_APPLICATION.generateNumber();
number = number * 2;
return number;
}
Plugin_2 would have the same structure, only different functionality:
void init(){ }
int exec(){
int number = HOST_APPLICATION.generateNumber();
number = (number * 10) + 13;
return number;
}
I need multiple instances of each 'plugin' to run simultaneously (each in its own thread), so 20 threads total at the same time. The code would be something like this:
Main Application:
void init_plugins() { }
void execute_plugins(){
for(int i=0; i<plugins.count; i++){
for(int z=0; z<10; z++){
Thread t = new Thread(plugin_research);
t.start(plugins[i]);
}
}
}
void plugin_research(PluginStructure Plugin){
Plugin.init();
int return_val = Plugin.exec();
// do something with return_val
}
I also need host application to call plugin's functions (the structure will be the same for all plugins) and plugin to be able to call host application's functions.
Each plugin would have different controls for configuration. I would need to show configuration in one place, but call plugin's functions multiple times simultaneously (using threads)
Is this possible? What would be the best way to accomplish this? can I do this with iPlugin?
Look into MEF, the managed extensibility framework at http://mef.codeplex.com/. It has a lot of built in support for runtime discovery of plugin components.
Basicly each plugin should be derived from a base class exposing methods you want.
public abstract class BasePlugin
{
protected int number;
public abstract void init_plugin();
public abstract int exec(int number);
public BasePlugin(int number)
{
this.number = number;
}
}
Then you have
public class Plugin1: BasePlugin
{
public override void init_plugin()
{
}
public override int exec(int number)
{
}
}
In your app you can create plugins and keep them in a list
List<BasePlugin> list = new List<BasePlugin>();
list.Add(new Plugin1(generateNumber()));
BasePlugin p2 = new Plugin2(generateNumber());
p2.init_plugin();
list.Add(p2);
and do with loaded plugins whatever you please.
Or (as I see from your edited question) you can create threads for every plugin...
To load plugins you could use functions like these:
public static List<T> GetFilePlugins<T>(string filename)
{
List<T> ret = new List<T>();
if (File.Exists(filename))
{
Type typeT = typeof(T);
Assembly ass = Assembly.LoadFrom(filename);
foreach (Type type in ass.GetTypes())
{
if (!type.IsClass || type.IsNotPublic) continue;
if (typeT.IsAssignableFrom(type))
{
T plugin = (T)Activator.CreateInstance(type);
ret.Add(plugin);
}
}
}
return ret;
}
public static List<T> GetDirectoryPlugins<T>(string dirname)
{
List<T> ret = new List<T>();
string[] dlls = Directory.GetFiles(dirname, "*.dll");
foreach (string dll in dlls)
{
List<T> dll_plugins = GetFilePlugins<T>(Path.GetFullPath(dll));
ret.AddRange(dll_plugins);
}
return ret;
}
Check out MEF - http://msdn.microsoft.com/en-us/library/dd460648.aspx. Even if you decide not to use it you can see how such architecture can be implemented and used.
Look into MEF, the Managed Extensibility Framework.

ObjectPool<T> or similar for .NET already in a library?

I don't want to write my own because i'm afraid i might miss something and/or rip off other people's work, so is there an ObjectPool (or similar) class existing in a library for .NET?
By object pool, i mean a class that assists caching of objects that take a long time to create, generally used to improve performance.
In the upcoming version of .NET (4.0), there's a ConcurrentBag<T> class which can easily be utilized in an ObjectPool<T> implementation; in fact the there's an article on MSDN that shows you how to do precisely this.
If you don't have access to the latest .NET framework, you can get the System.Collections.Concurrent namespace (which has ConcurrentBag<T>) in .NET 3.5 from Microsoft's Reactive Extensions (Rx) library (in System.Threading.dll).
UPDATE:
I'd also put forward BufferBlock<T> from TPL DataFlow. IIRC it's part of .net now. The great thing about BufferBlock<T> is that you can wait asynchronously for items to become available using the Post<T> and ReceiveAsync<T> extension methods. Pretty handy in an async/await world.
ORIGINAL ANSWER
A while back I faced this problem and came up with a lightweight (rough'n'ready) threadsafe (I hope) pool that has proved very useful, reusable and robust:
public class Pool<T> where T : class
{
private readonly Queue<AsyncResult<T>> asyncQueue = new Queue<AsyncResult<T>>();
private readonly Func<T> createFunction;
private readonly HashSet<T> pool;
private readonly Action<T> resetFunction;
public Pool(Func<T> createFunction, Action<T> resetFunction, int poolCapacity)
{
this.createFunction = createFunction;
this.resetFunction = resetFunction;
pool = new HashSet<T>();
CreatePoolItems(poolCapacity);
}
public Pool(Func<T> createFunction, int poolCapacity) : this(createFunction, null, poolCapacity)
{
}
public int Count
{
get
{
return pool.Count;
}
}
private void CreatePoolItems(int numItems)
{
for (var i = 0; i < numItems; i++)
{
var item = createFunction();
pool.Add(item);
}
}
public void Push(T item)
{
if (item == null)
{
Console.WriteLine("Push-ing null item. ERROR");
throw new ArgumentNullException();
}
if (resetFunction != null)
{
resetFunction(item);
}
lock (asyncQueue)
{
if (asyncQueue.Count > 0)
{
var result = asyncQueue.Dequeue();
result.SetAsCompletedAsync(item);
return;
}
}
lock (pool)
{
pool.Add(item);
}
}
public T Pop()
{
T item;
lock (pool)
{
if (pool.Count == 0)
{
return null;
}
item = pool.First();
pool.Remove(item);
}
return item;
}
public IAsyncResult BeginPop(AsyncCallback callback)
{
var result = new AsyncResult<T>();
result.AsyncCallback = callback;
lock (pool)
{
if (pool.Count == 0)
{
lock (asyncQueue)
{
asyncQueue.Enqueue(result);
return result;
}
}
var poppedItem = pool.First();
pool.Remove(poppedItem);
result.SetAsCompleted(poppedItem);
return result;
}
}
public T EndPop(IAsyncResult asyncResult)
{
var result = (AsyncResult<T>) asyncResult;
return result.EndInvoke();
}
}
In order to avoid any interface requirements of the pooled objects, both the creation and resetting of the objects is performed by user supplied delegates: i.e.
Pool<MemoryStream> msPool = new Pool<MemoryStream>(() => new MemoryStream(2048), pms => {
pms.Position = 0;
pms.SetLength(0);
}, 500);
In the case that the pool is empty, the BeginPop/EndPop pair provide an APM (ish) means of retrieving the object asynchronously when one becomes available (using Jeff Richter's excellent AsyncResult<TResult> implementation).
I can't quite remember why it is constained to T : class... there's probably none.
CodeProject has a sample ObjectPool implementation. Have a look here. Alternatively, there are some implementations here, here, and here.
How about System.Collections.Generic.Dictionary?
Sounds like you need a Factory pattern with caching.
You can try use .net reflector to look at the ThreadPool implementation.

Publishing to multiple subscribes in RX

I am investigating how to develop a plugin framework for a project and Rx seems like a good fit for what i am trying to achieve. Ultimately, the project will be a set of plugins (modular functionality) that can be configured via xml to do different things. The requirements are as follows
Enforce a modular architecture even within a plugin. This encourages loose coupling and potentially minimizes complexity. This hopefully should make individual plugin functionality easier to model and test
Enforce immutability with respect to data to reduce complexity and ensure that state management within modules is kept to a minimum
Discourage manual thread creation by providing thread pool threads to do work within modules wherever possible
In my mind, a plugin is essentially a data transformation entity. This means a plugin either
Takes in some data and transforms it in some way to produce new data (Not shown here)
Generates data in itself and pushes it out to observers
Takes in some data and does some work on the data without notifying outsiders
If you take the concept further, a plugin can consist of a number of all three types above.For example within a plugin you can have an IntGenerator module that generates some data to a ConsoleWorkUnit module etc. So what I am trying to model in the main function is the wiring that a plugin would have to do its work.
To that end, I have the following base classes using the Immutable nuget from Microsoft. What I am trying to achieve is to abstract away the Rx calls so they can be used in modules so the ultimate aim would be to wrap up calls to buffer etc in abstract classes that can be used to compose complex queries and modules. This way the code is a bit more self documenting than having to actually read all the code within a module to find out it subscribes to a buffer or window of type x etc.
public abstract class OutputBase<TOutput> : SendOutputBase<TOutput>
{
public abstract void Work();
}
public interface IBufferedBase<TOutput>
{
void Work(IList<ImmutableList<Data<TOutput>>> list);
}
public abstract class BufferedWorkBase<TInput> : IBufferedBase<TInput>
{
public abstract void Work(IList<ImmutableList<Data<TInput>>> input);
}
public abstract class SendOutputBase<TOutput>
{
private readonly ReplaySubject<ImmutableList<Data<TOutput>>> _outputNotifier;
private readonly IObservable<ImmutableList<Data<TOutput>>> _observable;
protected SendOutputBase()
{
_outputNotifier = new ReplaySubject<ImmutableList<Data<TOutput>>>(10);
_observable = _outputNotifier.SubscribeOn(ThreadPoolScheduler.Instance);
_observable = _outputNotifier.ObserveOn(ThreadPoolScheduler.Instance);
}
protected void SetOutputTo(ImmutableList<Data<TOutput>> output)
{
_outputNotifier.OnNext(output);
}
public void ConnectOutputTo(IWorkBase<TOutput> unit)
{
_observable.Subscribe(unit.Work);
}
public void BufferOutputTo(int count, IBufferedBase<TOutput> unit)
{
_observable.Buffer(count).Subscribe(unit.Work);
}
}
public abstract class WorkBase<TInput> : IWorkBase<TInput>
{
public abstract void Work(ImmutableList<Data<TInput>> input);
}
public interface IWorkBase<TInput>
{
void Work(ImmutableList<Data<TInput>> input);
}
public class Data<T>
{
private readonly T _value;
private Data(T value)
{
_value = value;
}
public static Data<TData> Create<TData>(TData value)
{
return new Data<TData>(value);
}
public T Value { get { return _value; } }
}
These base classes are used to create three classes; one for generating some int data, one to print out the data when they occur and the last to buffer the data as it comes in and sum the values in threes.
public class IntGenerator : OutputBase<int>
{
public override void Work()
{
var list = ImmutableList<Data<int>>.Empty;
var builder = list.ToBuilder();
for (var i = 0; i < 1000; i++)
{
builder.Add(Data<int>.Create(i));
}
SetOutputTo(builder.ToImmutable());
}
}
public class ConsoleWorkUnit : WorkBase<int>
{
public override void Work(ImmutableList<Data<int>> input)
{
foreach (var data in input)
{
Console.WriteLine("ConsoleWorkUnit printing {0}", data.Value);
}
}
}
public class SumPrinter : WorkBase<int>
{
public override void Work(ImmutableList<Data<int>> input)
{
input.ToObservable().Buffer(2).Subscribe(PrintSum);
}
private void PrintSum(IList<Data<int>> obj)
{
Console.WriteLine("Sum of {0}, {1} is {2} ", obj.First().Value,obj.Last().Value ,obj.Sum(x=>x.Value) );
}
}
These are run in a main like this
var intgen = new IntGenerator();
var cons = new ConsoleWorkUnit();
var sumPrinter = new SumPrinter();
intgen.ConnectOutputTo(cons);
intgen.BufferOutputTo(3,sumPrinter);
Task.Factory.StartNew(intgen.Work);
Console.ReadLine();
Is this architecture sound?
You are buffering your observable (.Buffer(count)) so that it only signals after count notifications arrive.
However, your IntGenerator.DoWork only ever produces a single value. Thus you never "fill" the buffer and trigger downstream notifications.
Either change DoWork so that it eventually produces more values, or have it complete the observable stream when it finishes its work. Buffer will release the remaining buffered values when the stream completes. To do this, it means somewhere IntGenerator.DoWork needs to cause a call to _outputNotifier.OnCompleted()

Categories

Resources