I'm working on a project that is using NSB, really like it but it's my first NSB solution so a bit of a noob. We have a job that needs to run every day that processes members - it is not expected to take long as the work is simple, but will potentially effect thousands of members, and in the future, perhaps tens or hundreds of thousands.
Having it all happen in a single handler in one go feels wrong, but having a handler discover affected members and then fire separate events for each one sounds a bit too much in the opposite direction. I can think of a few other methods of doing it, but was wondering if there is an idiomatic way of dealing with this in NSB?
Edit to clarify: I'm using Schedule to send a command at 3am, the handler for that will query the SQL db for a list of members who need to be processed. Processing will involve updating/inserting one or two rows per member. My question is around how to process that potentially larege list of members within NSB.
Edit part 2: the job now needs to run monthly, not daily.
I would not use a saga for this. Sagas should be lightweight and are designed for orchestration rather than performing work. They are started by messages rather than scheduled.
You can achieve your ends by using the built-in scheduler. I've not used it, but it looks simple enough.
You could do something like:
configure a command message (eg StartJob) to be sent every day at 0300.
StartJob handler will then query the DB to get the work.
Then, depending on your requirements:
If you need all the work done at once, create a single command with all the work in it, and send it to another endpoint for processing. If you use transactional MSMQ then this will succeed or fail as a unit.
If you don't care if only some work succeeds then create a command per unit of work, and dispatch to an endpoint for processing. This has the benefit that you can scale out using the distributor if you needed to.
I'm working on a project that is using NSB...We have a job that needs
to run every day...
Although you can use NSB for this kind of work, it's not really something I would do. There are many other approaches you could use. A SQL job or cron job would be the obvious one (and a hell of a lot quicker to develop, more performant, and simpler).
Even though it does support such use cases, NServiceBus is not really designed for scheduled batch processing. I would seriously question whether you should even use NSB for this task.
You mention a running process and that sounds like a job for a Saga (see https://docs.particular.net/nservicebus/sagas/). You can use saga data and persist checkpoints in different storage mediums (SQL, Mongo etc). But yes, having something long running then dispatch messages from the Saga to individual handlers is definitely something I would do also.
Something else to consider is message deferral (Timeout Managers). So for example, lets say you process x number of users but want to run this again. NServiceBus allows you to defer messages for a defined period and the message will sit in the queue waiting to be dispatched.
Anymore info just shout and I can update my answer.
A real NSB solution would be to get rid of the "batch" job that processes all those records in one run and find out what action(s) would cause each of these records to need processing after all.
When such an action is performed you should publish an NSB event and refactor the batch job to a NSB handler that subscribes to these events so it can do the processing the moment the action is performed, running in parallel with the rest of your proces.
This way there would be no need anymore for a scheduled 'start' message at 3 am, because all the work would already have been done.
Here is how I might model this idiomatically with NServiceBus: there might be a saga called PointsExpirationPolicy, which would be initiated at the moment that any points are awarded to a user. The saga would store the user ID, and number of points awarded, and also calculate the date/time the points should expire. Then it would request a timeout callback message to be sent at the date/time these points should expire. When that callback arrives, the saga sends a command to expire that number of points from the user's account. This would also give you some flexibility around the logic of exactly when and how points expire, and would eliminate the whole batch process.
Related
I have a worker attached to a subscription processing messages. Depending on the message payload, the worker could possibly take days to completely process the message.
I'm a bit confused by different properties that the client can set that control how the Pub/Sub client library automatically extends the deadline so that the worker can process the message without fearing that the message will be redelivered.
Properties from documentation which is not very clear :
MinimumAckDeadline, MaximumAckDeadline, DefaultAckDeadline, MinimumAckExtensionWindow, DefaultAckExtensionWindow, MinimumLeaseExtensionDelay, and DefaultMaxTotalAckExtension
I believe I want to set DefaultMaxTotalAckExtension to a large value (several days) to allow my subscriber to continue working on the message without getting some kind of time out.
But I think I also want to modify the AckDeadline so that Pub/Sub knows that the client is still alive. Not sure which one I would want to modify: MinimumAckDeadline, MaximumAckDeadline, DefaultAckDeadline.
Aside from those properties, I don't know if I need to set MinimumAckExtensionWindow, DefaultAckExtensionWindow, or MinimumLeaseExtensionDelay.
All of the properties you mentioned are default/limit properties for the SubscriberClient class itself. Note that they are static only have getters, not setters. What you want to set is MaxTotalAckExtension, which controls the maximum amount of time a message's lease will be extended.
However, taking days to process a message is considered an anti-pattern for Cloud Pub/Sub and will very likely result in duplicate deliveries. If you are going to take that long to process a message, you probably need to look at other options like persisting it locally (in a file or in a database), acking it, and then processing it. At that point, you may consider just writing directly to a database instead of to Pub/Sub and scanning for rows that need to be processed by your subscribers.
Explanation:
I am developing a simple car business system and I have to implement the following feature:
A very special car model is delivered to a shop. There are a lot of people on waiting list exactly for this model.
When the car arrives the first client receives the right to buy it, he / she has 24 hours to use this opportunity.
I have a special state in the DB that determines if the user is: on waiting list (I have the exact position, as well) or can use opportunity to buy the car. Whenever the car arrives, I run a method that changes the state of the first client on waiting list. And here comes the problem:
Problem:
The client can use his opportunity, during the 24 hours period. But I have to check at the end, if he/she has bought the car. For this reason, I have to schedule a method to run in 24 hours.
Possible solution:
I am thinking about two things. First is using a job scheduler like Hangfire. The problem is that since I do not have any other jobs in my app, I do not want to include a whole package for such a small thing. Second is using making the checking method asynchronous and making the thread sleep for 24 hours before proceeding (I do not feel comfortable in working with threads and this is just an idea). I got the idea from this article. Keep in mind that more than one car can arrive in more than one shop. Does it mean that I should use many threads and how it is going to affect the performance of the system?
Question:
Which of the two solutions is better?
Is there another possibility that you can suggest in this particular case?
I agree. Importing a package for only one job if you aren't going to use it for many jobs is a little bit of overkill.
If you are running SQL server, I'd recommend writing a .NET console application to run on a schedule using the SQL Server Agent. (see image) If you have stored procedures that need to run, you also have the option to run them directly from the SQL job if for some reason you don't want to run them from your .NET application.
Since it sounds like you need this to run on a data driven schedule, you may consider adding a trigger to look for a new record in your database whenever that "special" car is inserted into the database. MSDN SQL Job using Trigger
I've done something similar to this where every morning, an hour prior to business hours starting, I run a .NET executable that checks the latest record in table A and compares it to a value in table B and determines if the record in table A needs to be updated.
I also use SQL Server to run jobs that send emails on a schedule based on data that has been added or modified in a database.
There are advantages to using SQL server to running your jobs as there are many options available to notify you of events, retry running failed jobs, logging and job history. You can specify any type of schedule from repeating frequently to only running once a week.
I'm working on a user story whereby a Task (an entity) is created for a user to work on when a date is overdue and other criteria are met (on a separate entity - let's say a Product).
Ideally I would like a Domain Event to be created in real-time when this "Date" is overdue - however there isn't any trigger I can use in code to do this. I can only really see one type of pattern to use at the moment - that is to have a windows service which is polling every hour (using Topshelf / Quartz for example), pulling back all the records using the Product repository then code to check whether or not the dates are overdue and the criteria are met. If successful, the Domain Event will be triggered and the Task will be created.
As you can imagine, I don't particularly like this. It's not in real-time, and I'm pulling back a lot of data to achieve something relatively simple. Am I missing a trick here? Some kind of state machine / workflow? What architectural patterns / good designs are available for me to utilize in this situation?
Apologies if the question is a little vague, and I'll attempt to clarify if needs be.
If you use a sophisticated scheduler like Quartz anyway, why not just use it to call back to your application at the exact time the Task gets overdue? I've never used Quartz in this way, but I think this should be possible.
To get a robust solution, you might want to consider checking regularly in addition to the on-time callbacks, but I'd expect that these regular checks can run on a low frequency basis.
In any case, when you receive a callback, you need to check which tasks are really overdue. If polling the DB for this check is a performance problem (which I wouldn't expect it is in most cases), you can always cache the upcoming deadlines. Make sure you refresh the cache appropriately, e.g. by listening to the "task published" domain events.
I have this scenario, and I don't really know where to start. Suppose there's a Web service-like app (might be API tho) hosted on a server. That app receives a request to proccess some data (through some method we will call processData(data theData)).
On the other side, there's a robot (might be installed on the same server) that procceses the data. So, The web-service inserts the request on a common Database (both programms have access to it), and it's supposed to wait for that row to change and send the results back.
The robot periodically check the database for new rows, proccesses the data and set some sort of flag to that row, indicating that the data was processed.
So the main problem here is, what should the method proccessData(..) do to check for the changes of the data row?.
I know one way to do it: I can build an iteration block that checks for the row every x secs. But i don't want to do that. What I want to do is to build some sort of event listener, that triggers when the row changes. I know it might involve some asynchronous programming
I might be dreaming, but is that even possible in a web enviroment.?
I've been reading about a SqlDependency class, Async and AWait classes, etc..
Depending on how much control you have over design of this distributed system, it might be better for its architecture if you take a step back and try to think outside the domain of solutions you have narrowed the problem down to so far. You have identified the "main problem" to be finding a way for the distributed services to communicate with each other through the common database. Maybe that is a thought you should challenge.
There are many potential ways for these components to communicate and if your design goal is to reduce latency and thus avoid polling, it might in fact be the right way for the service that needs to be informed of completion of this work item to be informed of it right away. However, if in the future the throughput of this system has to increase, processing work items in bulk and instead poll for the information might become the only feasible option. This is also why I have chosen to word my answer a bit more generically and discuss the design of this distributed system more abstractly.
If after this consideration your answer remains the same and you do want immediate notification, consider having the component that processes a work item to notify the component(s) that need to be notified. As a general design principle for distributed systems, it is best to have the component that is most authoritative for a given set of data to also be the component to answer requests about that data. In this case, the data you have is the completion status of your work items, so the best component to act on this would be the component completing the work items. It might be better for that component to inform calling clients and components of that completion. Here it's also important to know if you only write this data to the database for the sake of communication between components or if those rows have any value beyond the completion of a given work item, such as for reporting purposes or performance indicators (KPIs).
I think there can be valid reasons, though, why you would not want to have such a call, such as reducing coupling between components or lack of access to communicate with the other component in a direct manner. There are many communication primitives that allow such notification, such as MSMQ under Windows, or Queues in Windows Azure. There are also reasons against it, such as dependency on a third component for communication within your system, which could reduce the availability of your system and lead to outages. The questions you might want to ask yourself here are: "How much work can my component do when everything around it goes down?" and "What are my design priorities for this system in terms of reliability and availability?"
So I think the main problem you might want to really try to solve fist is a bit more abstract: how should the interface through which components of this distributed system communicate look like?
If after all of this you remain set on having the interface of communication between those components be the SQL database, you could explore using INSERT and UPDATE triggers in SQL. You can easily look up the syntax of those commands and specify Stored Procedures that then get executed. In those stored procedures you would want to check the completion flag of any new rows and possibly restrain the number of rows you check by date or have an ID for the last processed work item. To then notify the other component, you could go as far as using the built-in stored procedure XP_cmdshell to execute command lines under Windows. The command you execute could be a simple tool that pings your service for completion of the task.
I'm sorry to have initially overlooked your suggestion to use SQL Query Notifications. That is also a feasible way and works through the Service Broker component. You would define a SqlCommand, as if normally querying your database, pass this to an instance of SqlDependency and then subscribe to the event called OnChange. Once you execute the SqlCommand, you should get calls to the event handler you added to OnChange.
I am not sure, however, how to get the exact changes to the database out of the SqlNotificationEventArgs object that will be passed to your event handler, so your query might need to be specific enough for the application to tell that the work item has completed whenever the query changes, or you might have to do another round-trip to the database from your application every time you are notified to be able to tell what exactly has changed.
Are you referring to a Message Queue? The .Net framework already provides this facility. I would say let the web service manage an application level queue. The robot will request the same web service for things to do. Assuming that the data needed for the jobs are small, you can keep the whole thing in memory. I would rather not involve a database, if you don't already have one.
I have an NHibernate MVC application that is using ReadCommitted Isolation.
On the site, there is a certain process that the user could initiate, and depending on the input, may take several minutes. This is because the session is per request and is open that entire time.
But while that runs, no other user can access the site (they can try, but their request won't go through unless the long-running thing is finished)
What's more, I also have a need to have a console app that also performs this long running function while connecting to the same database. It is causing the same issue.
I'm not sure what part of my setup is wrong, any feedback would be appreciated.
NHibernate is set up with fluent configuration and StructureMap.
Isolation level is set as ReadCommitted.
The session factory lifecycle is HybridLifeCycle (which on the web should be Session per request, but on the win console app would be ThreadLocal)
It sounds like your requests are waiting on database locks. Your options are really:
Break the long running process into a series of smaller transactions.
Use ReadUncommitted isolation level most of the time (this is appropriate in a lot of use cases).
Judicious use of Snapshot isolation level (Assuming you're using MS-SQL 2005 or later).
(N.B. I'm assuming the long-running function does a lot of reads/writes and the requests being blocked are primarily doing reads.)
As has been suggested, breaking your process down into multiple smaller transactions will probably be the solution.
I would suggest looking at something like Rhino Service Bus or NServiceBus (my preference is Rhino Service Bus - I find it much simpler to work with personally). What that allows you to do is separate the functionality down into small chunks, but maintain the transactional nature. Essentially with a service bus, you send a message to initiate a piece of work, the piece of work will be enlisted in a distributed transaction along with receiving the message, so if something goes wrong, the message will not just disappear, leaving your system in a potentially inconsistent state.
Depending on what you need to do, you could send an initial message to start the processing, and then after each step, send a new message to initiate the next step. This can really help to break down the transactions into much smaller pieces of work (and simplify the code). The two service buses I mentioned (there is also Mass Transit), also have things like retries built in, and error handling, so that if something goes wrong, the message ends up in an error queue and you can investigate what went wrong, hopefully fix it, and reprocess the message, thus ensuring your system remains consistent.
Of course whether this is necessary depends on the requirements of your system :)
Another, but more complex solution would be:
You build a background robot application which runs on one of the machines
this background worker robot can be receive "worker jobs" (the one initiated by the user)
then, the robot processes the jobs step & step in the background
Pitfalls are:
- you have to programm this robot very stable
- you need to watch the robot somehow
Sure, this is involves more work - on the flip side you will have the option to integrate more job-types, enabling your system to process different things in the background.
I think the design of your application /SQL statements has a problem , unless you are facebook I dont think any process it should take all this time , it is better to review your design and check where is the bottleneck are, instead of trying to make this long running process continue .
also some times ORM is not good for every scenario , did you try to use SP ?