Is it possible to accomplish the following using Quartz.NET :
i have a mysql table named "VODContent" with a field "StreamAt" - datetime - is it possible to monitor this table and 5 minutes before or after the "StreamAt" datetime value, some process (code) is executed ?
i'am avaialable for further details if needed
Thanks in advance
Sure, this is possible:
Create one Quartz.Net Job to poll for the VODContent (there are other ways then polling for other DBMS but this is a simple solution which works for mysql since the mysqlconnector doesn't feature event notification as far as I know). In order to create such a job, you have to implement the IJob interface of Quartz.Net as described here and your program has to schedule it to run every n-seconds/minutes which is also described in the tutorial.
A short tutorial which shows how to connect to, read from and write to a mysql database via the mysql connector can be found here.
How to find out if you already started a job for a specific VODContent depends. You could, for example, modify the specific Row and introduce a new bool value "Processed" which gets updated upon scheduling the specific job. You then have to query only for those values which are not yet processed.
Please read the parts about statefull vs. stateless jobs in Quartz.Net and be sure to understand the difference.
That job created above would then check for changes in the table and schedule a new job which in turn runs your desired process / code according to your rules using the simple trigger instruction as described in the tutorial.
My feeling is that by doing what you asked, you are not using Quartz to its full potential.
An easier way to achieve what you want to do would be that each time you create an entry in VODContent, you also create a trigger scheduled at StreamAt + 5 minutes. This should be done at the level of your applicative code, in the process that is adding the entry in VODContent.
The trigger you create must have a name and a group (so that you can retrieve it at a later point). In your case, I believe the name could be the Id of your VODContent line. Here is how you create such a trigger:
var simpleTrigger = new SimpleTrigger("yourVODContentId", "VODContentStreamAt", startTime);
Then, if at some point you want to cancel the execution of your trigger (let's say for example that you remove an entry in VODContent because your user cancelled his order), you can call the UnscheduleJob on the Scheduler in order to remove your trigger:
_scheduler.UnscheduleJob("yourVODContentId", "VODContentStreamAt");
One last important thing is that your Quartz.NET instance should be configured to AdoJobStore, and not the RAMJobStore. That way, your triggers will be stored in a database, and even if the scheduler is stopped/restarted, you won't lose your scheduled triggers.
Basically, you should try to put yourself in a position where you push to Quartz, instead of having Quartz pull from your application.
Related
I am using Azure Cosmos DB. I have some collections as shown in snapshot below-
Now, I want to create a scheduled task which will retrieve all the data from collection- "CurrentDay" and do some calculation and store the value in another collection- "datewise". Similarly, I will need to retrieve all the data from collection- "datewise" and based one some calculation store the data in "monthwise" and then to "yearly".
I looked for some option in Scheduler in Azure portal and tried creating a scheduler but it seems I don't have sufficient permission license to use that feature. Basically I haven't used that so I am not sure it will work.
Had it been in SQL Server I could have done that using custom code in C#. The only option I currently have is to use REST API calls to fetch data, calculate in C# and Post it back to Azure Cosmos DB. Is there any better way of doing this?
Please let me know if I can provide any details.
I think using a scheduled task (on Azure) and getting the data via the REST API is probably what you want to do. There are several reasons why this isn't as bad as you might think:
Your server and your database are right next to each other in the data centre, so you don't need to pay for data transfer.
Latency is very low and bandwidth is very high, so you'll be limited by the database performance more than anything else (you can run parallel tasks in your scheduled task to make sure of this).
The REST API has a very well supported official C# client library.
Of course it depends on the scale of data we're talking about as to how you should provision your scheduled task.
I'd encapsulate your logic in an Azure WebJob method, and mark the method with a TimerTrigger. The TimerTrigger will call your given method on the schedule that you specify. This has a few less moving parts. If you were to go the scheduler route, you're still going to have to have the scheduler call some endpoint in order to perform the work. Packaging up your logic and schedule in a WebJob will simplify things a bit.
On a side note, if all data lived in the same collection, I'd suggest writing a stored procedure to perform these calculations. But alas, stored procedures in Cosmos are bounded at the collection level.
Explanation:
I am developing a simple car business system and I have to implement the following feature:
A very special car model is delivered to a shop. There are a lot of people on waiting list exactly for this model.
When the car arrives the first client receives the right to buy it, he / she has 24 hours to use this opportunity.
I have a special state in the DB that determines if the user is: on waiting list (I have the exact position, as well) or can use opportunity to buy the car. Whenever the car arrives, I run a method that changes the state of the first client on waiting list. And here comes the problem:
Problem:
The client can use his opportunity, during the 24 hours period. But I have to check at the end, if he/she has bought the car. For this reason, I have to schedule a method to run in 24 hours.
Possible solution:
I am thinking about two things. First is using a job scheduler like Hangfire. The problem is that since I do not have any other jobs in my app, I do not want to include a whole package for such a small thing. Second is using making the checking method asynchronous and making the thread sleep for 24 hours before proceeding (I do not feel comfortable in working with threads and this is just an idea). I got the idea from this article. Keep in mind that more than one car can arrive in more than one shop. Does it mean that I should use many threads and how it is going to affect the performance of the system?
Question:
Which of the two solutions is better?
Is there another possibility that you can suggest in this particular case?
I agree. Importing a package for only one job if you aren't going to use it for many jobs is a little bit of overkill.
If you are running SQL server, I'd recommend writing a .NET console application to run on a schedule using the SQL Server Agent. (see image) If you have stored procedures that need to run, you also have the option to run them directly from the SQL job if for some reason you don't want to run them from your .NET application.
Since it sounds like you need this to run on a data driven schedule, you may consider adding a trigger to look for a new record in your database whenever that "special" car is inserted into the database. MSDN SQL Job using Trigger
I've done something similar to this where every morning, an hour prior to business hours starting, I run a .NET executable that checks the latest record in table A and compares it to a value in table B and determines if the record in table A needs to be updated.
I also use SQL Server to run jobs that send emails on a schedule based on data that has been added or modified in a database.
There are advantages to using SQL server to running your jobs as there are many options available to notify you of events, retry running failed jobs, logging and job history. You can specify any type of schedule from repeating frequently to only running once a week.
Below are my environment settings
SQL Server 2008 R2
ASP.NET MVC
Entity Framework
In my application I want to do post processing of transaction after they are inserted in database. So basically I want a separate application to be listening to database events, and whenever a new row is created in table Orders, do some post processing (calling another webservice to process orders) on it.
I checked with EF and it seems it doesn't provide any such feature.
Few other requirements
at this moment i am not looking for solution involving polling as it will increase db load.
Want some kind of event to get triggered when a row is inserted.
What would be the best way to achieve the same?
Create a T-SQL trigger on that table that inserts a small "command" entry into a separate "Command" table, with all the relevant information (e.g. OrderId and possibly others).
Then have your stand-alone application check that "Command" table on a regular basis (scheduled to check every 10 minutes - or whatever makes sense to you).
Based on the data read from the "Command" table, your application can then do any post-processing necessary, and update the underlying database table with the new information.
Warning: DO NOT put all this post-processing logic directly into the trigger! That would be a cardinal sin - a trigger should never do heavy and lengthy processing since that blocks the currently running transaction and kills your database's system performance for sure!
You can make use of a Service bus, something like MassTransit. Configure it to make use of it's Subscription service.
Once the order has been saved, raise an event on the bus, and have another service listen for this event. Once the other service picks up the event, you can continue processing the order.
This is the MassTransit main page
http://masstransit-project.com/
And this is the configuration for the subscription service
http://docs.masstransit-project.com/en/latest/overview/publishing.html
I'm using ASP.NET and C# to build some 'Social Network' web site,
while adding posts there are to SQL columns that i fill, the date and time when the post was added, and the date and time when the post is expired (It varies between all kind of posts..)
I want some process that constantly checks the SQL database and remove posts with expired date and time.
I've searched for solution and i understand that the 2 most suitable solutions are Semaphores and SQL Jobs (Correct me if i'm wrong).
I hope you could give me a hint about what's the best solution, if it's not one of the two what is it, and some info about the best solution as well..
Thanks!
Just hide posts that have expired based on the current time. For example
WHERE ExpiryDateTime > SYSUTCDATE()
Then you can clean old posts in the background at any frequency you like. Create a Windows Task Scheduler task that calls a special URL of your website. That URL should perform a database cleanup. This is a very simple and clearly correct solution.
If you don't like Windows Task Scheduler (and who really does like it...) you can use a scheduler lib such as Hangfire or Quartz.Net.
Neither.
A semaphore is a way of controlling resource use between multiple threads
An SQL job is a somewhat blunt tool designed to allow db admins to schedule tasks
I would create a separate program 'oldDataDeleter' code up your logic about what you want to delete or archive after how much time and then apply that logic in an atomic way. I would run this as a windows service with a timer, or a console app as a scheduled task
The key is to ensure that the program can run concurrently with itself and only does small atomic changes on a small chunk of data at a time.
You can then fire up multiple instances of this program running at a high frequency.
This ensures don't lock your database with large 'delete all from table X join table Y' statements and that your data is constantly trimmed rather than building up a big overnight job to run.
Edit for 'all code must be in a single website project' restriction
There is another solution which in some ways is better and works with your (slightly odd and very much not best practice) requirement.
That is to delete old entries whenever you make an insertion. So when you code adds a Post "insert into posts..." it also runs the delete "delete from posts where.."
This ensures that your program is self maintaining. However, you do incur a performance hit when adding posts. Given that a large social media site would be continually adding posts and needs to scale with its users. I don't recommend this solution.
However for small projects which don't need to scale it is neater.
I would like to know whether it is possible to run a C# console or ASP.NET application periodically.
My purpose is to automatically do the following steps once a month:
1) Query a Source database.
2) Take the result of the query at (1) and manipulate them by using LINQ and C#, for instance by checking if a certain value is already present in the Destination database.
3) Store the derived data in a Destination database.
The application has to run on a Windows Server 2008, the Source database is in a SQL 2005 Server and the Destination database is in a SQL 2008 Server.
I tried to create for instance a SSIS package but it won't do the job since I cannot add any logic.
Anybody has any suggestion?
You should create a Scheduled Task to perform this. Look here: Control Panel -> Administrative Tools -> Task scheduler
And as you stated, yes - a console app is highly recommended.
Edit
I agree with #andynormancx in that SSIS may be a better way to do this; however, it is commonly accepted to create a console app executed by a scheduled task. This is where it comes down to your resources, time, and expertise: it may or may not be worth the effort to learn enough about SSIS to create a package in SqlServer to do what you need. If someone were to give a complete answer using SSIS to perform this task, I would certainly bow to that expertise.
You can create a new Scheduled Task. It would be much easier and you don't have to re-invent the wheel.
You could create a scheduled task that will call your .exe at pre-defined interval.
Go to your control panel and select Scheduled Task and then add scheduled task
You can do it, of course. But I would recommend making it a Windows service.
If you really want to do it in C# I suggest you write a service, your code is almost Identical to a normal console app.
Check the following link to get started.
C# Service step by step