One application I work on does only one thing, looking from outside world. Takes a file as input and after ~5 minutes spits out another file.
What happens inside is actually a sequential series of action. The application is, in our opinion, structured well because each action is like a small box, without too many dependencies.
Usually some later actions use some information from previous one and just a few can be executed in parallel - for the sake of simplicity we prefer to the execution sequential.
Now the problem is that the function that executes all this actions is like a batch file: a long list of calls to different functions with different arguments. So, looking in the code it looks like:
main
{
try
{
result1 = Action1(inputFile);
result2 = Action2(inputFile);
result3 = Action3(result2.value);
result4 = Action4(result1.value, inputFile);
... //You get the idea. There is no pattern passed paramteres
resultN = ActionN(parameters);
write output
}
catch
{
something went wrong, display the error
}
}
How would you model the main function of this application so is not just a long list of commands?
Not everything needs to fit to a clever pattern. There are few more elegant ways to express a long series of imperative statements than as, well, a long series of imperative statements.
If there are certain kinds of flexibility you feel you are currently lacking, express them, and we can try to propose solutions.
If there are certain clusters of actions and results that are re-used often, you could pull them out into new functions and build "aggregate" actions from them.
You could look in to dataflow languages and libraries, but I expect the gain to be small.
Not sure if it's the best approach, but you could have an object that would store all the results and you would give it to each method in turn. Every method would read the parameters it needs and write its result there. You could then have a collection of actions (either as delegates or objects implementing an interface) and call them in a loop.
class Results
{
public int Result1 { get; set; }
public string Result2 { get; set; }
…
}
var actions = new Action<Results>[] { Action1, Action2, … };
Results results = new Results();
foreach (var action in actions)
action(results);
You can think of implementing a Sequential Workflow from Windows Workflow
First of all, this solution is far not bad. If the actions are disjunct, I mean there are no global parameters or other hidden dependencies between different actions or between actions and the environment, it's a good solution. Easy to maintain or read, and when you need to expand the functionality, you have just to add new actions, when the "quantity" changes, you have just to add or remove lines from the macro sequence. If there's no need for change frequently the process chain: don't move!
If it's a system, where the implementation of actions don't often changes, but their order and parameters yes, you may design a simple script language, and transform the macro class into that script. This script should be maintained by someone else than you, someone who is familiar with the problem domain in the level of your "actions". So, he/she can assembly the application using script language without your assistance.
One nice approach for that kind of problem splitting is dataflow programming (a.k.a. Flow-based programming). In dataflow programming, there are pre-written components. Components are black boxes (from the view of the application developer), they have consumer (input) and producer (output) ports, which can be connected to form a processing network, which is then the application. If there're a good set of components for a domain, many applications can created without programming new components. Also, components can be built of other components (they called composite components).
Wikipedia (good starting point):
http://en.wikipedia.org/wiki/Dataflow_programming
http://en.wikipedia.org/wiki/Flow-based_programming
JPM's site (book, wiki, everything):
http://jpaulmorrison.com/fbp/
I think, bigger systems must have that split point you describe as "macro". Even games have that point, e.g. FPS games have a 3D engine and a game logic script, or there's SCUMM VM, which is the same.
Related
Suppose we have a CommandRunner class that runs Commands, when a Command is created it's kept in the processingQueue for proccessing, if the execution of the Command finishes with errors the Command is moved to the faultedQueue for later processing but when everything is OK the Command is moved to the archiveQueue, the archiveQueue is not going to be processed in any way
the CommandRunner is something like this
class CommandRunner
{
public CommandRunner(IQueue<Command> processingQueue,
IQueue<Command> faultedQueue,
IQueue<Command> archiveQueue)
{
this.processingQueue = processingQueue;
this.faultedQueue= faultedQueue;
this.archiveQueue= archiveQueue;
}
public void RunCommands()
{
while(processingQueue.HasItems)
{
var current = processingQueue.Dequeue();
var result = current.Run();
if(result.HasError)
curent.MoveTo(faultedQueue);
else
curent.MoveTo(archiveQueue);
...
}
}
}
The CommandeRunner recives the three dependecies as a PersistentQueue the PersistentQueue is responsible for the long term storage of the Commands and so we free the CommandRunner from handling this
And the only purpose of the archiveQueue is to keep the design homogenous, to keep the CommandRunner persistence ignorant and with few dependencies
for example we can imagine a Property like this
IEnumerable<Command> AllCommands
{
get
{
return Enumerate(archiveQueue).Union(processingQueue).Union(faultedQueue);
}
}
many portions of the class need to do so(handle the Archive as a Queue to make the code simpler as shown above)
Does it make sense to use a Queue even if it's not the best abstraction, or do I have to use another abstraction for the archive concept.
what are other alternatives to meet these requirement?
Keep in mind that code, especially running code usually gets tangled and messy as time pass. To combat this, good names, good design, and meaningful comments come into play.
If you don't going to process the archiveQueue, and it's just a storage for messages that has been successfully processed, you can always store it as a different type (list, collection, set, whatever suits your needs), and then choose one of the following two:
Keep the name archiveQueue and change the underlying type. I would leave a comment where it's defined (or injected) saying : Notice that this might not be an actual queue. Name is for consistency reasons only.
Change the name to archiveRepository or something similar, while keeping the queue type. Obviously, since it's still a queue, you'll leave a comment saying: Notice, this is actually a queue.
Another thing to keep in mind, is that if you have n people working on your code base, you'll probably get n+1 different perferences about which way it shoud be done :)
Queue is a useful structure when you need to take care about the order of items inside it. If you need in your command post process, take care about the orders commands ran, then the queue can be a good choice.
If you don't need info about the order or commands, maybe you can use a List (on System.Collections namespace).
I think your choice are good, in the same case, I'll use a queues, we have a good example with OS design principles, inside OS (on Kernel) the process are queued for execution, clearly the OS queues are more complicated because they have other variables in mind like priority, and CPU utilization, but we can learn about the use of queues like data structures in process management.
Hi I am using the Simple Injector DI library and have been following some really interesting material about an architectural model designed around the command pattern:
Meanwhile... on the command side of my architecture
Meanwhile... on the query side of my architecture
The container will manage the lifetime of the UnitOfWork, and I am using commands to perform specific functions to the database.
My question is if I have a command, for example an AddNewCustomerCommand, which in turn performs another call to another service (i.e. sends a text message), from a design standpoint is this acceptable or should this be done at a higher level and if so how best to do this?
Example code is below:
public class AddNewBusinessUnitHandler
: ICommandHandler<AddBusinessUnitCommand>
{
private IUnitOfWork uow;
private ICommandHandler<OtherServiceCommand> otherHandler;
AddNewBusinessUnitHandler(IUnitOfWork uow,
ICommandHandler<OtherServiceCommand> otherHandler)
{
this.uow = uow;
this.otherHandler = otherHandler;
}
public void Handle(AddBusinessUnitCommand command)
{
var businessUnit = new BusinessUnit()
{
Name = command.BusinessUnitName,
Address = command.BusinessUnitAddress
};
var otherCommand = new OtherServiceCommand()
{
welcomePostTo = command.BusinessUnitName
};
uow.BusinessUnitRepository.Add(businessUnit);
this.otherHandler.Handle(otherCommand);
}
}
It depends on your architectural view of (business) commands, but it is quite natural to have a one to one mapping between a Use Case and a command. In that case, the presentation layer should (during a single user action, such as a button click) do nothing more than create the command and execute it. Furthermore, it should do nothing more than execute that single command, never more. Everything needed to perform that use case, should be done by that command.
That said, sending text messages, writing to the database, doing complex calculations, communicating with web services, and everything else you need to operate the business' needs should be done during the context of that command (or perhaps queued to happen later). Not before, not after, since it is that command that represents the requirements, in a presentation agnostic way.
This doesn't mean that the command handler itself should do all this. It will be quite naturally to move much logic to other services where the handler depends on. So I can imagine your handler depending on a ITextMessageSender interface, for instance.
Another discussion is if command handlers should depend on other depend command handlers. When you look at use cases, it is not unlikely that big use cases consist of multiple smaller sub use cases, so in that sense it isn't strange. Again, there will be a one to one mapping between commands and use cases.
However, note that having a deep dependency graph of nested command handlers depending on each other, can complicate navigating through the code, so take a good look at this. It might be better to inject an ITextSessageSender instead of using an ICommandHandler<SendTextMessageCommand>, for instance.
Another downside of allowing handlers to nest, is that it makes doing infrastructural stuff a bit more complex. For instance, when wrapping command handlers with a decorator that add transactional behavior, you need to make sure that the nested handlers run in the same transaction as the outer most handler. I happened to help a client of me with this today. It's not incredibly hard, but takes a little time to figure out. The same holds for things like deadlock detection, since this also runs at the boundary of the transaction.
Besides, deadlock detection is an great example to show case the power of this command/handler pattern, since almost every other architectural style will make it impossible to plug-in this behavior. Take a look at the DeadlockRetryCommandHandlerDecorator class in this article) to see an example.
I come from a PHP background and have used Wordpress quite a lot, I love how their plugin architecture works and the ability to hook events to event names. One of the best parts I like about it is being able to *add_filter()* to any database value just before it gets shown to the end user. My question is multi-part on how to replicate the whole plugin architecture in a C#.NET environment?
Part 1:
To create plug-ins I have researched the MEF framework would probably be the best (Managed Extensibility Framework -http://mef.codeplex.com/). This is designed specifically to take the grunt work out by giving you the ability to monitor directories for new plug-ins, tracking dependencies and other normal things. MEF ships with .NET 3.5+
Part 2
Hooking events? I can't seem to find much information about replicating a global channel based event system. From what I have upto yet I need a publish/subscribe pattern (which isn't that hard to make as you just create some concrete objects and give them events). The hard part is giving each event a 'channel' name and for all the events in the whole system to be part of a global collection (Mediator pattern).
To replicate: (http://codex.wordpress.org/Function_Reference/add_filter)
Example 1
// Add's my button to the end of the content
add_filter('the_content', 'my_plugin_button');
function my_plugin_button( $content ) {
// Add's my button to the end of the content
return $content . "<a href='#'>My button</a>";
}
OR
Example 2
// Add a new admin menu item by hooking in
add_action('admin_menu', 'my_plugin_menu');
function my_plugin_menu() {
add_options_page('My Plugin Options', 'My Plugin', 'manage_options', 'my-unique-identifier', 'my_plugin_options');
}
I hope your all with me upto yet? I have managed to replicate the functionality I need in Javascript and even jQuery has their .on() event function... same thing but channel or list based...
My 2 examples:
http://jsfiddle.net/AaronLayton/U3ucS/53/
http://jsfiddle.net/AaronLayton/eyNre/33/
Can anyone point me in the correct direction or is this the totaly wrong approach for c#?
I think NServiceBus can help you a lot with these issues. Udi Dahan which is the author of NServiceBus has also written a lot of articles about Domain Event pattern, which is a publish/subscribe mechanism.
Know it's been a long time since you posted this and you probably built something already. However I have been thinking about something like this myself. There are 2 options - really forget WordPress and try and build something much cleaner - it's a mess at the bottom of WordPress' code :D
Or this:
function the_content()
{
var result = get_the_content();
// do other stuff...if you want to.
execute_filters(ref result);
execute_actions(ref result);
return result;
}
function execute_filters(ref string result, string action_name)
{
var filters = get_pre_filters(action_name);
filters.ForEach(filter =>
{
/// somehow call the method name in filter. PHP is generally global. C# namespaced,
/// so you would need to think about that.
}
}
function execute_actions(ref string result, string action_name)
{
/// and so on....
}
When building something to mimic WordPress, you need to remember many of the issues of WordPress' plugin architecture (in my personal opinion)... It seems to want to run every plugin near enough on every page even if that page has nothing to do with that plugin. I onced installed a plugin that added 60 database queries to each page call, and it wasn't used.
Try and think smart about it when you are building it. Try and add a way to only have the plugins that are going to get used on the page/post of your new setup to be run e.g. in your database have a "Plugins" field on the post/page object with a list of plugins allowed to run on that page. That way you won't need to check all the plugins each time to see if it wants to run.
Anyways. Hope you got something working.
Recently I have been attempting to implement a robust behavior tree using the treesharp library posted by apoc. I have been going over iterators and interfaces in my books, but I still can't even figure out how to test let alone use this library. How the interfaces connect with eachother and how to actually perform a test/build a tree with them is confusing the heck out of me.
Usually in this situation, I would look for code examples and derive enlightenment from looking at other people's work, however, for this library, there does not seem to be any example code.
Could anyone help me figure out how I could start to build a behavior tree using this library? I am sorry if the question is very noobish (and I think it may be) but Enumerators and progressive interfaces within interfaces are extremely difficult for me to understand right now.
I'm the author of TreeSharp, if you guys have any questions, feel free to shoot me an email (its contained in every source file in the header).
You'll first need to understand the concepts of behavior trees (the differences between selectors, sequences, decorators, actions, and the like). I also provide a few "vanity" composites to make things slightly easier (such as Wait).
The constructor-based API lets you define trees entirely via ctors (with the use of delegates which are evaluated at runtime to provide decisions, etc)
Unfortunately I never got around to implementing the "TreeExecutor" class, which handles executing an arbitrary behavior branch from something like a Tick() method. The easiest way (using a PrioritySelector in this example, but you can use any composite) is as follows;
static void Start()
{
// Start MUST be called before you can tick the tree.
Logic.Start(null);
// do work to spool up a thread, or whatever to call Tick();
}
private static void Tick()
{
try
{
Logic.Tick(null);
// If the last status wasn't running, stop the tree, and restart it.
if (Logic.LastStatus != RunStatus.Running)
{
Logic.Stop(null);
Logic.Start(null);
}
}
catch (Exception e)
{
// Restart on any exception.
Logging.WriteException(e);
Logic.Stop(null);
Logic.Start(null);
throw;
}
}
Unfortunately, giving "examples" of its usage really depends on what you're using it for. (Since its so generic, its difficult to give examples that will make sense for any given project. I've been using it from things to AI logic, to workflows, down to scheduling processes)
A small example which may help a bit;
static Composite CreateFireMissile()
{
return new PrioritySelector(
new Decorator(ret => CurrentShip.CurrentTarget != null,
new Action(ret => CurrentShip.CurrentTarget.FireMissile())),
new Decorator(ret => CurrentShip.CurrentTarget == null,
new Decorator(ret => CurrentShip.NearbyHostiles.Count > 0,
new Sequence(
new Action(ret => CurrentShip.SetTarget(CurrentShip.NearbyHostiles[0])),
new Action(ret => CurrentShip.RotateTo(CurrentShip.CurrentTarget.Location))
)
)
)
);
}
Again, this really depends on your requirements. The library will let you subclass any composites for easier to re-use composites. (Eg; you can create a SetTargetAndRotate action, which eliminates the two actions within the Sequence)
Again, if you guys have questions, don't hesitate to ask.
Jason,
The library seems to implement all the ideas I've seen demonstrated in other examples of BT libraries. I'm looking into leveraging such a library for one of my own projects, so until I do, I'm only guessing on how to use this one based on a brief inspection.
Nevertheless, I think you would supply your own callback functions to instances of the Action class, and then piece together various actions into group combinations based on BT ideas (wait until something is true before acting, act until one of these actions is successful, perform all these actions unless one fails, etc).
HTH
I have these container objects (let's call them Container) in a list. Each of these Container objects in turn has a DataItem (or a derivate) in a list. In a typical scenario a user will have 15-20 Container objects with 1000-5000 DataItems each. Then there are some DataMatcher objects that can be used for different types of searches. These work mostly fine (since I have several hundred unit tests on them), but in order to make my WPF application feel snappy and responsive, I decided that I should use the ThreadPool for this task. Thus I have a DataItemCommandRunner which runs on a Container object, and basically performs each delegate in a list it takes as a parameter on each DataItem in turn; I use the ThreadPool to queue up one thread for each Container, so that the search in theory should be as efficient as possible on multi-core computers etc.
This is basically done in a DataItemUpdater class that looks something like this:
public class DataItemUpdater
{
private Container ch;
private IEnumerable<DataItemCommand> cmds;
public DataItemUpdater(Container container, IEnumerable<DataItemCommand> commandList)
{
ch = container;
cmds = commandList;
}
public void RunCommandsOnContainer(object useless)
{
Thread.CurrentThread.Priority = ThreadPriority.AboveNormal;
foreach (DataItem di in ch.ItemList)
{
foreach (var cmd in cmds)
{
cmd(sh);
}
}
//Console.WriteLine("Done running for {0}", ch.DisplayName);
}
}
(The useless object parameter for RunCommandsOnContainer is because I am experimenting with this with and without using threads, and one of them requires some parameter. Also, setting the priority to AboveNormal is just an experiment as well.)
This works fine for all but one scenario - when I use the AllWordsMatcher object type that will look for DataItem objects containing all words being searched for (as opposed to any words, exact phrase or regular expression for instance).
This is a pretty simple somestring.Contains(eachWord) based object, backed by unit tests. But herein lies some hairy strangeness.
When the RunCommandsOnContainer runs using ThreadPool threads, it will return insane results. Say I have a string like this:
var someString = "123123123 - just some numbers";
And I run this:
var res = someString.Contains("data");
When it runs, this will actually return true quite a lot - I have debugging information that shows it returning true for empty strings and other strings that simply do not contain the data. Also, it will some times return false even when the string actually contains the data being looked for.
The kicker in all this? Why do I suspect the ThreadPool and not my own code?
When I run the RunCommandsOnContainer() command for each Container in my main thread (i.e. locking the UI and everything), it works 100% correctly - every time! It never finds anything it shouldn't, and it never skips anything it should have found.
However, as soon as I use the ThreadPool, it starts finding a lot of items it shouldn't, while some times not finding items it should.
I realize this is a complex problem (it is painful trying to debug, that's for sure!), but any insight into why and how to fix this would be greatly appreciated!
Thanks!
Rune
It's a bit hard to see from the fragment you're posting, but judging by the symptoms I would look at the AllWordsMatcher (look for static state). If AllWordsMatcher is stateful you should also check that you're creating a new instance for each thread.
More generally I'd look at all the instances involved in the matching/searching process, specifically at the working objects being used when multithreaded. From past experience, the problem usually lies there. (It's easy to look too much at the object graph representing your business data Container/DataItem in this case)