How can i get notification on every new insertion in sql table, i want to get the top recent entries to show in Grid View against a user, the way i was doing this is calling ajax function using JavaScript on setInterval/setTimeout, but the problem is this its hitting the database in mentioned time interval, and i just want to load the entries (call the function) only when there is new insertion in Database just to minimize the Database Hits.
Thanks in Advance :)
This is possible using triggers and extended stored procedures that can access processes outside the SQL application. I would say though that this is not the right architecture, and extended stored procedures have additional concerns. It would be preferable to use which ever application layer is writing to the database to perform the necessary communications signalling that a new message has arrived.
Related
I have MySQL database, and program in C# and winforms. It will run on few PC's over the lan network. I want to know if theres a way to check if theres changes in database without doing request each second.
Example:
Admin will delete record from database, on workers pc I want to refresh list automatically.
Don't worry about the performance of polling. With a suitable INDEX, MySQL can handle 100 polls/second -- regardless of dataset size.
Let's see SHOW CREATE TABLE and the tentative SELECT to perform the "poll"; I may have further tips.
Also, let's see the admin's query that needs to 'trigger' the workers into action. Surely the admin can do something simple to cause that. Doing a checksum of a table is terribly heavyweight.
You should consider the admin action to be encoded in a Stored Procedure, thereby separating the polling mechanism from the admin action.
JavaScript can use AJAX to do an INSERT (or call a Stored proc).
This mantra may apply: "Don't queue it, just do it." That is, the admin/javascript action can call an application program (PHP/Java/VB/whatever) to actually do the action, no need for queuing. (Please provide more details so we can discuss whether this might be the real solution.)
I am currently working on an asp.net web application and have been interested in implementing a SQL dependency into a part of it. There is a process where users will update some rows in a SQL server database table. A windows service hosted elsewhere will receive a sqldependency query notification, and process the data. What I want is for the web application to post back once it detects that the data processing has completed. The way I'm trying to do this at the moment is to create a SQLdependency with a query that monitors the relevant rows. The only notification I care about is when the query no longer returns rows (meaning the data has finished processing). Is there any clean way to make this work? My current implementation receives notifications for every change made, which come as the data is processed.
I believe you can use a field on a table to track the changes. But you make the field an int (this will be similar to how ##Trancount works). The premise is this:
Each service that begins updates will increase that field by one.
As they finish, they decrease that field by one (and your web app can be the one controlling that aspect)
(NB) On error the services need to be aware to decrease the field by one.
Your SqlDependency will have a query which monitors this table, selecting that field when it changes. When it is zero, it means that no more rows are returned.
Below are my environment settings
SQL Server 2008 R2
ASP.NET MVC
Entity Framework
In my application I want to do post processing of transaction after they are inserted in database. So basically I want a separate application to be listening to database events, and whenever a new row is created in table Orders, do some post processing (calling another webservice to process orders) on it.
I checked with EF and it seems it doesn't provide any such feature.
Few other requirements
at this moment i am not looking for solution involving polling as it will increase db load.
Want some kind of event to get triggered when a row is inserted.
What would be the best way to achieve the same?
Create a T-SQL trigger on that table that inserts a small "command" entry into a separate "Command" table, with all the relevant information (e.g. OrderId and possibly others).
Then have your stand-alone application check that "Command" table on a regular basis (scheduled to check every 10 minutes - or whatever makes sense to you).
Based on the data read from the "Command" table, your application can then do any post-processing necessary, and update the underlying database table with the new information.
Warning: DO NOT put all this post-processing logic directly into the trigger! That would be a cardinal sin - a trigger should never do heavy and lengthy processing since that blocks the currently running transaction and kills your database's system performance for sure!
You can make use of a Service bus, something like MassTransit. Configure it to make use of it's Subscription service.
Once the order has been saved, raise an event on the bus, and have another service listen for this event. Once the other service picks up the event, you can continue processing the order.
This is the MassTransit main page
http://masstransit-project.com/
And this is the configuration for the subscription service
http://docs.masstransit-project.com/en/latest/overview/publishing.html
My current situation is that I have an application that needs to be notified when new data arrives in a database table. The data is coming from an external source (that I have no control over--this is this only integration option). When new data arrives, my application needs to take certain actions--basically query for the new data, handle it, insert the result into a local table, etc.
I want to avoid polling if possible, as the data is expected to be handled in real time. That said, making sure no data ever gets missed is the #1 priority.
My questions:
Is SqlDependency generally considered reliable?
Do I need to be concerned about race conditions, e.g. I am handling one change when another arrives?
What happens when the database gets rebooted? Will my app recover and start receiving changes again, or will I need a fail-safe timer of some sort that will resubscribe to notifications periodically?
Most of the articles I have read on the topic address SQL Server 2005. I am using SQL Server 2008 R2. Is there a newer technique that is preferred over SqlDependency?
(Edit)Also, What if the application goes down? I guess I would have to query for missed data on start up?
1) Yes, I consider it reliable as in it does correctly the purpose was designed to do (cache invalidation)
2) No. This is why you can only subscribe by issuing a query, this ensures that there is no race between the fetching of the data and new updates notifying
3) Database (or instance) restart signals all pending query notifications with an SqlNotificationInfo value of Restart. Read how SqlDependency and is based on Query Notification for a better understanding. As SqlDependency keeps an open connection to the database all the time, a database unavailability will be detected by SqlDependency even before any explicit query notification
4) No. More on this further down...
5) There is no 'missed data'. Query Notification (and hence SqlDependency) never notify you about what data changed. It only notifies you that it changed. You are always supposed to go back and read all the data back to
see what had changed (and I refer you back to question/answer no. 2). A newly started application had not yet queried the data to begin with, so there is no change to be notified of. Only after it has first queried the data can it receive a notification.
From the description of your problem I'm not convinced you need query notifications. It seems to me that you want to act on any change, not matter when it happened, even if your application was not running. This is certainly not cache invalidation, it is change tracking. Therefore you need to deploy a change tracking technology, like Change Data Capture or Change Tracking, both of which are SQL Server 2008 and later only (not available in SQL Server 2005). With SQL Server 2005 is not uncommon to deploy a trigger and queue a message for Service Broker to handle the same problem you are trying to handle (detect changes, react to each row of new data).
Coming at it from the point of view of a .net developer to just wants to use it for cache invalidation it has been a real pain and isn't completely reliable.
Set up and troubleshooting has been particularly painful, we get it working okay in one environment but then it doesn't work in another. Figuring out why has been difficult and time-consuming.
Even when it is all running it isn't completely reliable. SQL Server can drop notifications if it under heavy load and there are known issues with it restarting and notifications not resuming: http://connect.microsoft.com/SQLServer/feedback/details/543921/sqldependency-incorrect-behaviour-after-sql-server-restarts.
I would avoided if there is an alternative technology the does what you want and is less troublesome.
I have an SQL Server 2008 Database and am using C# 4.0 with Linq to Entities classes setup for Database interaction.
There exists a table which is indexed on a DateTime column where the value is the insertion time for the row. Several new rows are added a second (~20) and I need to effectively pull them into memory so that I can display them in a GUI. For simplicity lets just say I need to show the newest 50 rows in a list displayed via WPF.
I am concerned with the load polling may place on the database and the time it will take to process new results forcing me to become a slow consumer (Getting stuck behind a backlog). I was hoping for some advice on an approach. The ones I'm considering are;
Poll the database in a tight loop (~1 result per query)
Poll the database every second (~20 results per query)
Create a database trigger for Inserts and tie it to an event in C# (SqlDependency)
I also have some options for access;
Linq-to-Entities Table Select
Raw SQL Query
Linq-to-Entities Stored Procedure
If you could shed some light on the pros and cons or suggest another way entirely I'd love to hear it.
The process which adds the rows to the table is not under my control, I wish only to read the rows never to modify or add. The most important things are to not overload the SQL Server, keep the GUI up to date and responsive and use as little memory as possible... you know, the basics ;)
Thanks!
I'm a little late to the party here, but if you have the feature on your edition of SQL Server 2008, there is a feature known as Change Data Capture that may help. Basically, you have to enable this feature both for the database and for the specific tables you need to capture. The built-in Change Data Capture process looks at the transaction log to determine what changes have been made to the table and records them in a pre-defined table structure. You can then query this table or pull results from the table into something friendlier (perhaps on another server altogether?). We are in the early stages of using this feature for a particular business requirement, and it seems to be working quite well thus far.
You would have to test whether this feature would meet your needs as far as speed, but it may help maintenance since no triggers are required and the data capture does not tie up your database tables themselves.
Rather than polling the database, maybe you can use the SQL Server Service broker and perform the read from there, even pushing which rows are new. Then you can select from the table.
The most important thing I would see here is having an index on the way you identify new rows (a timestamp?). That way your query would select the top entries from the index instead of querying the table every time.
Test, test, test! Benchmark your performance for any tactic you want to try. The biggest issues to resolve are how the data is stored and any locking and consistency issues you need to deal with.
If you table is updated constantly with 20 rows a second, then there is nothing better to do that pull every second or every few seconds. As long as you have an efficient way (meaning an index or clustered index) that can retrieve the last rows that were inserted, this method will consume the fewest resources.
IF the updates occur in burst of 20 updates per second but with significant periods of inactivity (minutes) in between, then you can use SqlDependency (which has absolutely nothing to do with triggers, by the way, read The Mysterious Notification for to udneratand how it actually works). You can mix LINQ with SqlDependency, see linq2cache.
Do you have to query to be notified of new data?
You may be better off using push notifications from a Service Bus (eg: NServiceBus).
Using notifications (i.e events) is almost always a better solution than using polling.