I can’t wrap my head about the 2 concepts being different, when I need a distributed transaction or event business logic my mind always goes to using a state machine. I know routing slips are useful but I can’t really know when to use one.
My question is, when would I need to use one over the other? Or why use a state machine to track the state of a routing slip and not just manage state and use the state machine? I can’t really tell when to use one or the other
A state machine only tracks one state at a time, and that state could have any number of potential exits. If you think of the Mario Brothers games, big Mario could get a fire flower, or a leaf/tail, or a star/invincibility, or get shrunk, or fall in a pit and die. Those are all new states that could transition from the big Mario state.
Routing slip requires a linear set of processes or actions that are fixed from the start. Outgoing mail goes to the origin mailbox (where the flag gets raised), to the origin post office (to be aggregated with all other outgoing mail), to the sorting facility (to choose the destination post office), to the destination post office (to select the route to the destination mailbox), and then to the destination mailbox.
You can't skip any of those steps. You can't do them out of order. There aren't multiple potential exits at each step along the way.
Related
I'm making an app as a learning experience, and I've run into two major issues:
1) I have an implementation that may or may not be a good idea for Orleans...I just don't know.
2) I'm looking for advice on hosting and deployment, basically: where and how? This is just a learning experience, so small and cheap/free is important.
So, to start... A little info on the app I'm making:
It's a mobile gps based app with MongoDB storage.
The real world is divided up into distinct and persistent regions, blocks of gps locations.
When a user is active in a given region, he becomes visible to every other user in the region, and he begins receiving updates about the region's properties and the locations of other users active in the region.
The user can manipulate the properties of the region.
Users need to be able to explicitly join and leave regions, but also leave after some time-out.
I have two grains for this: UserGrain and RegionGrain.
For issue 1) Is it even a good idea to implement the Region as a grain? A few different samples support this implementation. But, the best practices manual suggests it wouldn't be a good idea, since it is a long-running job (the Region persists indefinitely), and this type of grain could be a bottleneck. But... I need it to act like the "GameGrain" from the "Presence" sample, that players join and leave (explicitly or timing out).
For issue 2)
I have a silo implemented as a console app. I've used amazon elastic beanstalk to deploy a .net web app, but I don't know where to start with a console app. Is amazon a good choice at all? I just defaulted to it from recent usage.
If the RegionGrain is a bad idea... what might a good idea look like?
If the RegionGrain is a good idea... is there a way to tell Orleans to just never deactivate it? If so, I should use Timers for the time-out requirement, and the presence and heartbeat for updates...right? If not, should I just host another console app somewhere that maintains and runs RegionGrain(s)?
If this is still an actual question:
On Issue 1) - Your region grain seems fine to me - it's not a constantly working grain but it's your region registry: when user enters the region - he registers in the region grain (or being registered by an external region monitor if you want to remove user responsibility around this). Such regions can persists indefinitely, but this doesn't mean they are long running jobs.
Long running job in orleans terminology is the one that is doing a lot of active work in a single method (or worse - sleeps for seconds or minutes) - this will hold the grain and thread from orleans thread pool - so such practice is considered bad but it's not really applicable here.
On hosting - Amazon is as good as any other hosting as long as you know how to host and run your work there. We are using Azure for our hosting - Cloud Service is very easy to start with and VM cost is usually on par with Amazon (except Azure free trial is shorter than Amazon 3.months vs 1.year but with more resources).
On Issue 2), timers and other - timers are not persistent so if the grain is deactivated - they are lost and have to be restarted. Reminders are better option to revive grain, as well as there is a method on a grain which allows you to prevent deactivation - DelayDeactivation(), the opposite one for explicit deactivation is DeactivateOnIdle().
Regarding the updates - we found that Orleans Streams are very nice concept to broadcast mass updates (e.g. from RegionGrain to all subscribed\registered users) so take a look on it and it may solve many questions with subscription and broadcasting.
In general, I'd build this a bit differently - make your system reactive instead of pulsing\proactive. e.g.:
User enters the region - so he just registering his presence on the region grain and subscribes for region events.
Any changes to the region are recorded and pushed to Orleans stream - so all subscribers will receive region event.
Any changes to other players - messaged to region grain (so it'll be our source of truth) and then re-broadcasted to other region subscribers.
In case of issues with very noisy regions (when too many changes are happening in the system and too big re-broadcasts - you can make your region just an another subscriber to the region stream - so every player will broadcast to every other via Orleans Stream and to region itself. OR make your region into a broadcast controller which would broadcast all changes in one batch to the stream only after a certain period of time - every second or every 5 seconds...
Hope this helps.
I am designing a system that simulates the flow of a document in an organization. For the sake of simplicity let's assume that the flow has the following states:
Opened
Evaluated
Rejected
Accepted
There are times that some external resources are required to be available to proceed. Thus if a resource isn't available the whole flow should be put on hold. I imagine that there's another (somehow) parallel state machine that has two states:
In Progress
On Hold
The way that I thought I could solve this problem was to check the state of the second state machine as a guard condition in every transition of the first state machine.
But I wonder if there's a common way or pattern for solving this kind of problem?
BTW, I want to implement this state machines using the Stateless or bbv Common(Appccelerate) libraries.
With a UML state machine you could use hierarchical states together with a history states.
In Progress
Opened
Evaluated
Rejected
Accepted
(H) History state
On Hold
Events for the substates of 'In Progress' will only be handled, if 'In Progress' and one of it's substates is active.
The (H) history state can be used to reactivate the most recently active substate when 'In Progress' becomes active.
Based on my experience with state machines I think you would be better off with just the one state machine. Make the On Hold a state and have the other states check in their check conditions whether the desired external resource is available, if it isn't then move the document to the On Hold state.
As for the in progress state, I think this is implied by the other states and not really necessary.
States made up of:
CheckConditions, this is where you put your guard conditions
ExitConditions
EntryConditions
I have an NHibernate MVC application that is using ReadCommitted Isolation.
On the site, there is a certain process that the user could initiate, and depending on the input, may take several minutes. This is because the session is per request and is open that entire time.
But while that runs, no other user can access the site (they can try, but their request won't go through unless the long-running thing is finished)
What's more, I also have a need to have a console app that also performs this long running function while connecting to the same database. It is causing the same issue.
I'm not sure what part of my setup is wrong, any feedback would be appreciated.
NHibernate is set up with fluent configuration and StructureMap.
Isolation level is set as ReadCommitted.
The session factory lifecycle is HybridLifeCycle (which on the web should be Session per request, but on the win console app would be ThreadLocal)
It sounds like your requests are waiting on database locks. Your options are really:
Break the long running process into a series of smaller transactions.
Use ReadUncommitted isolation level most of the time (this is appropriate in a lot of use cases).
Judicious use of Snapshot isolation level (Assuming you're using MS-SQL 2005 or later).
(N.B. I'm assuming the long-running function does a lot of reads/writes and the requests being blocked are primarily doing reads.)
As has been suggested, breaking your process down into multiple smaller transactions will probably be the solution.
I would suggest looking at something like Rhino Service Bus or NServiceBus (my preference is Rhino Service Bus - I find it much simpler to work with personally). What that allows you to do is separate the functionality down into small chunks, but maintain the transactional nature. Essentially with a service bus, you send a message to initiate a piece of work, the piece of work will be enlisted in a distributed transaction along with receiving the message, so if something goes wrong, the message will not just disappear, leaving your system in a potentially inconsistent state.
Depending on what you need to do, you could send an initial message to start the processing, and then after each step, send a new message to initiate the next step. This can really help to break down the transactions into much smaller pieces of work (and simplify the code). The two service buses I mentioned (there is also Mass Transit), also have things like retries built in, and error handling, so that if something goes wrong, the message ends up in an error queue and you can investigate what went wrong, hopefully fix it, and reprocess the message, thus ensuring your system remains consistent.
Of course whether this is necessary depends on the requirements of your system :)
Another, but more complex solution would be:
You build a background robot application which runs on one of the machines
this background worker robot can be receive "worker jobs" (the one initiated by the user)
then, the robot processes the jobs step & step in the background
Pitfalls are:
- you have to programm this robot very stable
- you need to watch the robot somehow
Sure, this is involves more work - on the flip side you will have the option to integrate more job-types, enabling your system to process different things in the background.
I think the design of your application /SQL statements has a problem , unless you are facebook I dont think any process it should take all this time , it is better to review your design and check where is the bottleneck are, instead of trying to make this long running process continue .
also some times ORM is not good for every scenario , did you try to use SP ?
In my plugin I have a code to check the execution context Depth to avoid infinite loop once the plugin updates itself/entity, but there are cases that entity is being updated from other plugin or workflow with depth 2,3 or 4 and for that specific calls, from that specific plugin I want to process the call and not stop even if the Depth is bigger then 1.
Perhaps a different approach might be better? I've never needed to consider Depth in my plug-ins. I've heard of other people doing the same as you (checking the depth to avoid code from running twice or more) but I usually avoid this by making any changes to the underlying entity in the Pre Operation stage.
If, for example, I have code that changes the name of an Opportunity whenever the opportunity is updated, by putting my code in the post-operation stage of the Update my code would react to the user changing a value by sending a separate Update request back to the platform to apply the change. This new Update itself causes my plug-in to fire again - infinite loop.
If I put my logic in the Pre-Operation stage, I do it differently: the user's change fires the plugin. Before the user's change is committed to the platform, my code is invoked. This time I can look at the Target that was sent in the InputParameters to the Update message. If the name attribute does not exist in the Target (i.e. it wasn't updated) then I can append an attribute called name with the desired value to the Target and this value will get carried through to the platform. In other words, I am injecting my value into the record before it is committed, thereby avoiding the need to issue another Update request. Consequently, my changes causes no further plug-ins to fire.
Obviously I presume that your scenario is more complex but I'd be very surprised if it couldn't fit the same pattern.
I'll start by agreeing with everything that Greg said above - if possible refactor the design to avoid this situation.
If that is not possible you will need to use the IPluginExecutionContext.SharedVariables to communicate between the plug-ins.
Check for a SharedVariable at the start of your plug-in and then set/update it as appropriate. The specific design you'll use will vary based on the complexity you need to manage. I always get use a string with the message and entity ID - easy enough to serialize and deserialize. Then I always know whether I'm already executing the against a certain message for a specific record or not.
I've a server/client architecture implemented, where all state changes are sent to the function, validated and broadcasted to all clients connected. This works rather well, but the system does not maintain synchronization between the client instances of the game as of now.
If there happened to be a 5 second lag between the server and a particular client then he would receive the state change 5 seconds after the rest of the clients thus leaving him with game state out of sync. I've been searching for various ways to implement a synchronization system between the clients but haven't found much so far.
I'm new to network programming, and not so naive to think that I can invent a working system myself without dedicating a severe amount of time to it. The ideas I've been having, however, is to keep some kind of time system, so each state change would be connected to a specific timestamp in the game. That way when a client received a state change, it would know exactly in which period of the game the changed happened, and would in turn be able to correlate for the lag. The problem with this method is that in those n seconds lag the game would have had continued on the client side, and thus the client would have to rollback in time to update for the state change which definitely would get messy.
So I'm looking for papers discussion the subjects or algorithms that solves it. Perhaps my whole design of how the multiplayer system works is flawed, in the sense that a client's game instance shouldn't update unless notion is received from the server? Right now the clients just update themselves in their game loop assuming that any states haven't changed.
The basic approach to this is something called Dead Reckoning and a quite nice article about it can be found here. Basically it is a predication algorithm for where entities positions will be guessed at for the times between server updates.
There are more advanced methodologies that build on this concept, but it is a good starting point.
Also a description of how this is handled in the source engine (Valve's engine for the first Half Life game) can be found here, the principle is basically the same - until the server tells you otherwise use a prediction algorithm to move the entity along an expected path - but this article handles the effect this has on trying to shoot something in more depth.
The best resources I've found in this area are these two articles from Valve Software:
Latency Compensating Methods in Client/Server In-game Protocol Design and Optimization
Source Multiplayer Networking
There will never be a way to guarantee perfect synchronisation across multiple viewpoints in real time - the laws of physics make it impossible. If the sun exploded now, how could you guarantee that observers on Alpha Centauri see the supernova at the same time as we would on Earth? Information takes time to travel.
Therefore, your choices are to either model everything accurately with latency that may differ from viewer to viewer (which is what you have currently), or model them inaccurately without latency and broadly synchronised across viewers (which is where prediction/dead reckoning/extrapolation come in). Slower games like real time strategy tends to go the first route, faster games go the second route.
In particular, you should never assume that the time it takes to travel will be constant. This means that merely sending start and stop messages to move entities will never suffice under either model. You need to send periodic updates of the actual state (typically several times a second for faster games) so that the recipient can correct error in its predictions and interpolations.
If client see events happening at the rate the server is feeding him, which is the normal way to do it (I've worked with protocols of Ultima Online, KalOnline and a little bit of World of Warcraft), then this momentaneous 5 secounds delay would just make him receive this 5 secounds of events all at once and see those events passing really fast or near instantly, as other players would see him "walking" really fast for a short distance if his outputs delay too. After that everything flows normally again. Actually, except for graphic and physics normalization, I can't see any special needs to make it synchronize properly, it just synchronize itself.
If you ever played Valve games in two near computers you would notice they don't care much about minor details like "the exact place where you died" or "where you dead body gibs flyed to". It is all up to client side and totally affected by latency, but this is irrelevant.
After all, lagged players must accept their condition, or close their damn eMule.
Your best option is to send the changes back to the client from the future, thereby arriving at the client at the same point in time it does for other clients that does not have lag problems.