I am trying to monitor a database table for changes using the SqlDependency class. Though I must be missing something. I have followed all of the examples that I see online and I have reviewed all the questions on this site. I just don't see what I am missing. Here are the initial commands that I ran on the database to enable the service broker and create the queue and service.
CREATE QUEUE ScheduleChangeQueue
GO
CREATE SERVICE ScheduleChangeService ON QUEUE ScheduleChangeQueue ([http://schemas.microsoft.com/SQL/Notifications/PostQueryNotification])
GO
ALTER DATABASE [database] SET ENABLE_BROKER
On the C# side, I have created a class that has a single static Setup method that is called to initiate the process. Here is the code for that:
public class SqlDependencyManager
{
private static bool DoesUserHavePermission()
{
var success = false;
try
{
Program.Log.Info("Retrieving SqlPermission to establish dependency...");
var clientPermission = new SqlClientPermission(System.Security.Permissions.PermissionState.Unrestricted);
// this will throw an error if the user does not have the permissions
clientPermission.Demand();
success = true;
Program.Log.Info("SqlPermission established. Continue setting up dependency.");
}
catch (Exception ex)
{
Program.Log.Error(ex, "SqlPermission not able to be established.");
}
return success;
}
public static void Setup()
{
if (!DoesUserHavePermission())
{
return;
}
var connectionString = ConfigurationManager.ConnectionStrings["ShowMakerPro"].ConnectionString;
// You must stop the dependency before starting a new one.
// You must start the dependency when creating a new one.
SqlDependency.Stop(connectionString);
SqlDependency.Start(connectionString);
using (var cn = new SqlConnection(connectionString))
{
using (var cmd = cn.CreateCommand())
{
cmd.CommandType = CommandType.Text;
//cmd.CommandText = "SELECT MAX(LastChangeTime) FROM Schedule WHERE ChannelID IN ( SELECT ID FROM Channels WHERE Type = 1 ) AND StartTime BETWEEN (GETDATE() - 7) AND (GETDATE() + 30)";
cmd.CommandText = "SELECT LastChangeTime FROM dbo.Schedule";
cmd.Notification = null;
// Creates a new dependency for the SqlCommand. Then creates attaches handler for the notification of data changes
new SqlDependency(cmd).OnChange += SqlDependency_OnChange;
cn.Open();
cmd.ExecuteReader();
}
}
Program.Log.Info("SQL Dependency set. Now monitoring schedule table for changes.");
}
private static void SqlDependency_OnChange(object sender, SqlNotificationEventArgs e)
{
if (e.Type == SqlNotificationType.Change)
{
// this will remove the event handler since the dependency is only for a single notification
((SqlDependency)sender).OnChange -= SqlDependency_OnChange;
ScheduleOutputterService.BuildSchedules();
Program.Log.Info("SQL Dependency triggered schedule rebuild. Resetting SqlDependency to monitor for changes.");
Setup();
}
}
}
I see the code get setup ok and the OnChange method is fired once for the Subscribe but then I never see it fire after that. I manually go into the database and change the LastChangeTime field hoping that it will force the firing of the event but nothing happens.
Can someone please shed some light on where I am screwing up? I see some people saying on line that this works fine in a windows form but they are also having some problems while in a service.
So I finally figured out the answer to my question and I thought I should list all the steps I took to get to this point so someone else coming along behind me will also have another place to look for answers since I seemed unable to find all of my answers in one place.
First off, I noticed in my situation that as soon as the subscription was set the OnChange event would fire right away. That is why I put in the check for change type so I could ignore those events. It turns out that ignoring those events was not a good thing because those events were actually trying to tell me something. Doing a search on my values directed me here:
http://msmvps.com/blogs/siva/archive/2011/11/22/subtle-sqldependency-notification-issue.aspx
This was very valuable because it helped me to see that there must have been a problem with some of my options in the database. Upon further inspection I noticed that my database was set to SQL Server 2000 compatibility. That is obviously my first problem because this is a 2005 and greater feature. So I tried to change my settings to the high version. This worked ok but then I still noticed that I was receiving the same event. So then I checked my database settings and I found that they were not set to match the options required to run service broker. You can see all the required option settings here:
http://msdn.microsoft.com/en-us/library/ms181122(v=SQL.100).aspx
After I inspected all these and tried to do some workarounds to get the right settings all squared away the setup was still failing. But this time it was failing because the updates would not save. It turned out that the client had triggers on the table in question and it's database settings that were being used to execute the trigger were in conflict with the needed settings to run QueryNotifications.
So long story short, the client decided that they did not want to change all the triggers that were being used so they abandoned the effort. But I learned a lot about how to troubleshoot SqlDependency and ServiceBroker. Hopefully these few links I provided will be helpful for someone else. A couple more links that were very helpful were mentioned in the comments but I am going to repost them in this answer so you can have some other items to review.
http://rusanu.com/2006/06/17/the-mysterious-notification/
http://rusanu.com/2005/12/20/troubleshooting-dialogs/
Related
We are running one third party application which uses C#, SQL SERVER. We have created one other application which prints the pass.
Basically it does continuous checking of new entry from third party application in one of the table on remote database. If new entry is present then it prints pass. Accessing network database in such way is not good way and also sometimes application hang.
Instead of continuous loop, I am searching for some other way like: As the new entry comes, it trigger my application for print. Or any other good way to implement.
What you are looking for is the SqlDependency that can help you in
listening to the OnChange event.
Example from msdn:
void Initialization()
{
// Create a dependency connection.
SqlDependency.Start(connectionString, queueName);
}
void SomeMethod()
{
// Assume connection is an open SqlConnection.
// Create a new SqlCommand object.
using (SqlCommand command=new SqlCommand(
"SELECT ShipperID, CompanyName, Phone FROM dbo.Shippers",
connection))
{
// Create a dependency and associate it with the SqlCommand.
SqlDependency dependency=new SqlDependency(command);
// Maintain the refence in a class member.
// Subscribe to the SqlDependency event.
dependency.OnChange+=new
OnChangeEventHandler(OnDependencyChange);
// Execute the command.
using (SqlDataReader reader = command.ExecuteReader())
{
// Process the DataReader.
}
}
}
// Handler method
void OnDependencyChange(object sender,
SqlNotificationEventArgs e )
{
// Handle the event (for example, invalidate this cache entry).
}
void Termination()
{
// Release the dependency.
SqlDependency.Stop(connectionString, queueName);
}
Check it out: http://msdn.microsoft.com/en-us/library/62xk7953.aspx
If any user subsequently changes the underlying data, Microsoft SQL
Server detects that there is a notification pending for such a change,
and posts a notification that is processed and forwarded to the client
through the underlying SqlConnection that was created by calling
SqlDependency.Start. The client listener receives the invalidation
message. The client listener then locates the associated SqlDependency
object and fires the OnChange event.
let me just sum up the data:
you need to give us more information for a more intelligent help here bu basically you can:
Use a timer: that way you are not in an infinite loop and you don't check every few ticks the 3rd party. you can be more sophisticated even and make the time interval grow and shrink according to data changes found or not
if you can change the 3rd party so it'll send events or signal when it has new data then you can use that and register to the events and save a lot of process time
if you can change the DB add triggers and use them to know then new data come and thus you don't need nothing more then that
I implemented windows service with eventLog and FileSystemWatcher that looks for changes in specific directory and writes messages into MyLog.
strange thing 1:
I install it via installUtil.exe (since the VS2012 doesn't have installer templates) and in some situations when I go to "Services" and start the service I get:
The [service name] service on local computer started and then stopped. Some Services stop automatically if they are not in use by another services or programs.
I've already seen this question. 2 answeres from this post why it can be so:
1) There is no thread starting in OnStart() method.
I use the designer and set most of the properties in the Properties window and I never started any thread manually, but in some cases everything was working, so I think this is not the case.
2) An exception occures in OnStart() method. I think it's not the case cause I don't change the code. I just uninstall, build and install again the same service and in some cases it runs, in some not.
When I was stuck for mabby 2 hours with this thing I noticed that the Source property of eventLog is too long: "FilesMonitoringServices". I changed it to "MonitorSource" and everything started to work. Than I reinstalled it cauple of times and got the same warning as the above. I changed the Source property again and now the service runs.
This is the first strange thing.
strange thing 2: worse. Even if it runs it logs only OnStart() and OnStop() methods, I mean the fileSystemWatcher event handler never excutes. It is strange because today I reinstalled this service mabby hundred times and 3 times it was working but after I reinstalled it once again it stoped. And I haven't changed the code between the reinstallations at all.
Here is the methods and constructor from my class (MonitoringService) that inherits ServiceBase:
public MonitoringService()
{
InitializeComponent();
if (!EventLog.SourceExists(eventLog.Source))
{
EventLog.CreateEventSource(eventLog.Source, eventLog.Log);
}
// haven't changed it between the reinstallations
fileWatcher.Path = #"path";
}
protected override void OnStart(string[] args)
{
fileWatcher.EnableRaisingEvents = true;
eventLog.WriteEntry("start", EventLogEntryType.Information);
base.OnStart(args);
}
protected override void OnStop()
{
fileWatcher.EnableRaisingEvents = false;
fileWatcher.Dispose();
eventLog.WriteEntry("stop", EventLogEntryType.Information);
base.OnStop();
}
And file system watcher event handler:
private void fileSystemWatcher1_Changed(object sender, FileSystemEventArgs e)
{
using (var conn = new SqlConnection(GetConnectionString()))
{
conn.Open();
var productId = Convert.ToInt32(Regex.Match(e.Name, #"\d+").Value);
const string cmd = "UPDATE Products SET ImageModifiedDate=#date WHERE ProductId=#productId";
using (var command = new SqlCommand(cmd, conn))
{
command.Parameters.AddWithValue("#productId", productId);
command.Parameters.AddWithValue("#date", DateTime.Now);
command.ExecuteNonQuery();
}
}
eventLog.WriteEntry(string.Format("{0} has been changed: {1}", e.Name, DateTime.Now), EventLogEntryType.Information);
}
Question: it seems to me that this behavior is caused not by my code but rather by operation system settings . Can it be so?
****Edits: just discovered more specific stuff:**
1) If it shows the message (when I want to start the service):
The [service name] service on local computer started and then stopped. ....
I need to change Source property of eventLog, rebuild and reinstall. And this message will not show up; mabby next time.
2) I have the following folders hierarchy: images/prod-images. images and prod-images directories both contain image files. When the service is runing and I change the image from prod-images folder the message is written into the log as I wanted and the database is updated. But after one this event the service stops! (I checked this 3 times). And when I restart it and repeat this again a couple of times it updates database, writes logs and on the 3d time I get
The [service name] service on local computer started and then stopped. ....
But this is not the best part) If I change the image that is in images directory I can do it multiple times and the service doesn't stop. (only images from images/prod-images are bound to entries in the database).
So, mabbe this feature somehow referes to the database accessing?
Edits 2: in visual studio I use DEBUG -> Attach to Process to debug the service. I set up the breakpoints and change the image. First time the event handler executes flawlessly: the database is updated and the log message is written. But than I continue to press F11 (Step Into) and this event handler executes second time. At the line
var productId = Convert.ToInt32(Regex.Match(e.Name, #"\d+").Value);
I get "FormatException was unhandled". After this I stop debugging and the service stops! That's it: the exception occures in event handler.
Do You have any idea why it executes second time? Thanks!
P.S. I've already submited Davut Gürbüz answer cause he pointed me in the right direction.
Anyway, check out my own answer that explains the actual problem.
If you got start-stop error, this means you have an error in constructor.
Put a try catch into your ctor. You may log the error to eventlog in catch block.
Beside this I create a main method and start win service as a console app. If I get an instance of service in my main method I can also Debug it.
//You should select Console Application from Application properties
static void Main(string[] args)
{
MyWindowsService service = new MyWindowsService();
if (Environment.UserInteractive)
{
service.OnStart(args);
Console.WriteLine("Press any key to stop program");
Console.Read();
service.OnStop();
}
else
{
ServiceBase.Run(service);
}
}
Hope helps
The reason why the fileSystemWatcher1_Changed event handler method was executing twice is because I was monitoring images folder included subdirectories. And this event handler was monitoring all LastWrite events.
So, when I changed the image in images/prod-images directory this handler reacted to the image changing and also to the folder changing.
In this situation I either can change the monitoring path to prod-images or insert an if statement when updating the DB.
Such a silly mistake took me a couple of days to find it out))
I'm using SqlDependency to get notification when data in some table are changed.
private void subscribeBroker()
{
using (var conn = new SqlConnection(connString))
{
conn.Open();
var cmd = new SqlCommand("SELECT text FROM dbo.Test");
cmd.Connection = conn;
var dependency = new SqlDependency(cmd);
dependency.OnChange += dependency_OnChange;
SqlDependency.Start(connString);
cmd.ExecuteNonQuery();
}
}
void dependency_OnChange(object sender, SqlNotificationEventArgs e)
{
//Do something...
subscribeBroker();
}
It is working but I have some questions.
1) I didn't find a way how to get information which row was changed. I need to read all data from entire table to see what is different. Is there a way how to get this information? (primary ID, or something) Maybe to use different approach than SqlDependency?
2) What if "somebody" changing data very fast. It is possible that some changes will not being notified? (I'm concerned about time between notification and time when I subscribe it again.
Thank you.
About 1- query notification informs you about the fact, that something is changed. If you want to get what was changed since last time- you could probably use timestamp column.
About 2- query notification informs you about changes and then is dropped. then you again subscribe for notification again. that mean- time between dropping and creation of notifications is that time in which notification about changes is not send.
Query notifications is more for the situations, when your data is not changing frequently. For example- some cashed classification values. So- you subscribe for changes in some table, wait for changes and at the time they happen you get latest version of data. Should consider that query notification also uses server resources, so if you have huge table and want to get changes on some small subset of data, a lot of queries can be affected in terms of performance (something like indexed view).
If you need to take some action based on changed data and each change is important, then i would guess that trigger + service broker could be more effective. Or, depending on your needs, Change Data Capture.
Using Microsoft's EWS, we're able to listen to a mailbox and take actions when a new email comes in. However, I can't figure out how to avoid the connection timing out.
Per Microsoft, here is the constructor for a StreamingSubscriptionConnection:
public StreamingSubscriptionConnection (
ExchangeService service,
int lifetime
)
In my app, I've coded it as follows:
service = new ExchangeService(ExchangeVersion.Exchange2010_SP1);
StreamingSubscriptionConnection conn = new StreamingSubscriptionConnection(service, 30);
In other words, I've got the timeout (lifetime) set to 30 minutes, because that's the highest I've been able to set it. How can I increase this? Or, how can I trick this subscription into staying alive, even if ~45 minutes transpire between incoming emails?
30 minutes is a hard limit. You can not change it to a higher value.
To solve this issue, wire up a handler to the OnDisconnected handler of the OnDisconnect event of the connection instance. Restart the subscription from there (just call connection.Open() from that handler).
If anyone else is interested, this is how I am accomplishing this.
I want to keep the connection open, so I am resetting it in the OnDisconnect handler.
However, before resetting it, I check the private "subscriptions" dictionary on the connection object using reflection.
This allows me to unsubscribe from my connections elsewhere in my code (OnNotificationEvent), and when all subscriptions have been unsubscribed from, I am then able to close the connection.
Here is my Code:
void connection_OnDisconnect(object sender, SubscriptionErrorEventArgs args)
{
var c = (Dictionary<string, StreamingSubscription>)typeof(StreamingSubscriptionConnection).GetField("subscriptions",System.Reflection.BindingFlags.NonPublic | System.Reflection.BindingFlags.Instance).GetValue(sender);
if (c.Count > 0)
{
// reopen the connection
((StreamingSubscriptionConnection)sender).Open();
using (var db = new Metrics_DatabaseEntities())
{
PushNotificationTest pt = new PushNotificationTest();
pt.RetObj = "Connection reset";
db.PushNotificationTests.Add(pt);
db.SaveChanges();
}
}
else
{
using (var db = new Metrics_DatabaseEntities())
{
PushNotificationTest pt = new PushNotificationTest();
pt.RetObj = "Connection closed!";
db.PushNotificationTests.Add(pt);
db.SaveChanges();
}
}
}
Please disregard the poor way that this is written, this is just my first version, as I plan to write this more cleanly soon. I just thought I would share my methodology with folks that might be interested.
If people are interested, here's the little bit of logic that got added.
I added this to my Start method:
conn.OnDisconnect +=
new StreamingSubscriptionConnection.SubscriptionErrorDelegate(OnDisconnect);
I then added the OnDisconnect method:
private void OnDisconnect(object sender, SubscriptionErrorEventArgs args)
{
Start();
}
Ultimately, this still needs improved, because this simply times-out and reconnects every half-hour, regardless of incoming email activity. I'd rather get something in place that resets the counter every time a new message comes in. Then, it would only time-out a couple times per day, instead of 48! Still, this is serving its purpose of keeping my email-listening program online.
I would like to make use of SQL Server notifications to capture insert events at my database within a winforms app. I am attempting to use the SQLDependency object. The MSDN articles make this seem pretty straight forward. So I have created a little example application to give it a try. The event only seems to fire as I enter my application the first time(MessageBox appears). Inserting data into the table does not raise the OnChange event it would seem. Can someone tell me what I'm missing? Thanks!
public Main()
{
InitializeComponent();
var check = EnoughPermission();
SqlDependency.Stop(constr);
SqlDependency.Start(constr);
if(connection == null)
{
connection = new SqlConnection(constr);
}
if(command == null)
{
command = new SqlCommand("Select ID, ChatMessage FROM dbo.Chat",connection);
}
connection.Open();
command.Notification = null;
SqlDependency dependency = new SqlDependency(command);
dependency.OnChange += new OnChangeEventHandler(dependency_OnChange);
command.ExecuteReader();
}
private void dependency_OnChange(object sender, SqlNotificationEventArgs e)
{
MessageBox.Show("Change!");
}
While I was working in the implementation of query notification, I got the exact problem. I checked all configurations, code pieces, and even TCP settings, but nothing helped. Then, I figured out the following query to run on database and it solved my problem. Maybe you can try it.
ALTER AUTHORIZATION ON DATABASE::[Your DB] TO sa;
Your first notification is the only notification you'll get. Query Notifications are not a subscription for changes, once a notification is fired it is also invalidate. You are supposed to re-submit a new notification subscription.
If your query is notified immedeatly it means you did not get a notification for a change, but one for an invalid query. Check the values of the SqlNotificationEventArgs argument you receive. Check the Info to be Insert/Update/Delete, check the Source to be Data, check the Type to be Change.
Have a look at the Watcher Application example to better understand how you are supposed to re-subscribe when notified. For a better understanding of how the Query Notifications work, see The Mysterious Notification.