Quick summary with what I now know
I've got an EventWaitHandle that I created and then closed. When I try to re-create it with this ctor, an "Access to the path ... is denied" exception is thrown. This exception is rare, most of the times it just re-creates the EventWaitHandle just fine. With the answer posted below (by me), I'm able to successfully call EventWaitHandle.OpenExisting and continue on in the case that an exception was thrown, however, the ctor for EventWaitHandle should have done this for me, right? Isn't that what the out parameter, createdNew is for?
Initial question
I've got the following architecture, a windows service and a web service on the same server. The web service tells the windows service that it has to do work by opening and setting the wait handle that the windows service is waiting on.
Normally everything is flawless and I'm able to start / stop the windows service without any issue popping up. However, some times when I stop the web service and then start it up again, it will be completely unable to create the wait handle, breaking the whole architecture.
I specifically need to find out what is breaking the event wait handle and stop it. When the wait handle "breaks", I have to reboot windows before it will function properly again and thats obviously not ideal.
UPDATE: Exception thrown & Log of Issue
I rebooted the windows service while the web service was doing work in hopes of causing the issue and it did! Some of the class names have been censored for corporate anonymity
12:00:41,250 [7] - Stopping execution due to a ThreadAbortException
System.Threading.ThreadAbortException: Thread was being aborted.
at System.Threading.Thread.SleepInternal(Int32 millisecondsTimeout)
at OurCompany.OurProduct.MyClass.MyClassCore.MonitorRequests()
12:00:41,328 [7] - Closing Event Wait Handle
12:00:41,328 [7] - Finally block reached
12:00:42,781 [6] - Application Start
12:00:43,031 [6] - Creating EventWaitHandle: Global\OurCompany.OurProduct.MyClass.EventWaitHandle
12:00:43,031 [6] - Creating EventWaitHandle with the security entity name of : Everyone
12:00:43,078 [6] - Unhandled Exception
System.UnauthorizedAccessException: Access to the path 'Global\OurCompany.OurProduct.MyClass.EventWaitHandle' is denied.
at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
at System.Threading.EventWaitHandle..ctor(Boolean initialState, EventResetMode mode, String name, Boolean& createdNew, EventWaitHandleSecurity eventSecurity)
at OurCompany.OurProduct.MyClassLibrary.EventWaitHandleFactory.GetNewWaitHandle(String handleName, String securityEntityName, Boolean& created)
at OurCompany.OurProduct.MyClassLibrary.EventWaitHandleFactory.GetNewEventWaitHandle()
at OurCompany.OurProduct.MyClass.MyClassCore..ctor()
Rough timeline:
11:53:09,937: The last thread on the web service to open that existing wait handle, COMPLETED its work (as in terminated connection with the client)
12:00:30,234: The web service gets a new connection, not yet using the wait handle. The thread ID for this connection is the same as the thread ID for the last connection at 11:53
12:00:41,250: The windows service stops
12:00:42,781: The windows service starts up
12:00:43,078: The windows service finished crashing
12:00:50,234: The web service was actually able to open the wait handle call Set() on it without any exception thrown etc.
12:02:00,000: I tried rebooting the windows service, same exception
12:36:57,328: After arbitrarily waiting 36 minutes, I was able to start the windows service up without a full system reboot.
Windows Service Code
Initialization:
// I ran into security issues so I open the global EWH
// and grant access to Everyone
var ewhSecurity = new EventWaitHandleSecurity();
ewhSecurity.AddAccessRule(
new EventWaitHandleAccessRule(
"Everyone",
EventWaitHandleRights.Synchronize | EventWaitHandleRights.Modify,
AccessControlType.Allow));
this.ewh = new EventWaitHandle(
false,
EventResetMode.AutoReset,
#"Global\OurCompany.OurProduct.MyClass.EventWaitHandle",
out created,
ewhSecurity);
// the variable "created" is logged
Utilization:
// wait until the web service tells us to loop again
this.ewh.WaitOne();
Disposal / closing:
try
{
while (true)
{
// entire service logic here
}
}
catch (Exception e)
{
// should this be in a finally, instead?
if (this.ewh != null)
{
this.ewh.Close();
}
}
Web Service Code
Initialization:
// NOTE: the wait handle is a member variable on the web service
this.existing_ewh = EventWaitHandle.OpenExisting(
#"Global\OurCompany.OurProduct.MyClass.EventWaitHandle");
Utilization:
// wake up the windows service
this.existing_ewh.Set();
Since the EventWaitHandle is a member variable on the web service, I don't have any code that specifically closes it. Actually, the only code that interacts with the EventWaitHandle on the web service is posted above.
Looking back, I should probably have put the Close() that is in the catch block, in a finally block instead. I probably should have done the same for the web service but I didn't think that it was needed.
At any rate, can anyone see if I'm doing anything specifically wrong? Is it crucially important to put the close statements within a finally block? Do I need to manually control the Close() of the existing_ewh on the web service?
Also, I know this is a slightly complex issue so let me know if you need any additional info, I'll be monitoring it closely and add any needed information or explanations.
Reference material
EventWaitHandleSecurity Class
EventWaitHandleAccessRule Class
EventWaitHandle Class
In the code that creates the wait handle on the windows service, if it fails (as in access denied), you could try to "open an existing wait handle" via
EventWaitHandle.OpenExisting(
#"Global\OurCompany.OurProduct.MyClass.EventWaitHandle",
EventWaitHandleRights.Synchronize | EventWaitHandleRights.Modify);
Though, I'm not entirely sure if the behaviour would stay the same at that point.
Note: I'd appreciate feedback. Its a potential answer so I'm answering my own question, again, plenty of comments are quite welcome!
Note 2: Amazingly, applying EventWaitHandleRights.FullControl instead of the above flags (Synchronize + Modify) doesn't work well. You must use the sample above.
MSDN says:
UnauthorizedAccessException - The named event exists and has access control security, but the user does not have EventWaitHandleRights.FullControl.
and
The caller has full control over the newly created EventWaitHandle object even if eventSecurity denies or fails to grant some access rights to the current user.
Your service has no rights to get the existing event via EventWaitHandle constructor. (EventWaitHandleRights.FullControl is not specified. And your named event exists while it has opened handles on it.) You can open the existing event using EventWaitHandle.OpenExisting.
Related
We have an application which read messages from IBM MQ Topic and interact with users via SignalR WebSockets.
Case:
Open iis asp.net application web.config
Change and save it (this causing appdomain restart)
Repeate step 2 10 times
After that we can see many Application_Start/Dispose events in logs but at ONE of appdomain restart iterations haven't Dispose call. Cause that out IBM MQ listener handling message from old AppDomain therefore we have duplicate handling and business errors.
It seems like something constraint AppDomain from unload.
I know what it's very hard to say what's there happening, but maybe anybody knows how can we trace that problem.
Disable Overlapped Recycle is true
Shutdown Time Limit is 3s
what I have do in similar situation - on global.asax use this call
void Application_End(object sender, EventArgs e)
{
// here signaling the listener to close - and wait until they do
// also raise the shutdown time limit to more than 3 seconds, give them time to close
}
I have a Topshelf C# service that must restore adapter DNS settings when exiting. My stop/start methods work just fine and this code works:
ManagementObject.InvokeMethod("SetDNSServerSearchOrder", DNS, null);
Shutdown, however, is a problem, even with RequestAdditionalTime
I log the following error:
2016-11-30 15:10:53,427 [7] TRACE MyDNSService - DNSService Shutdown command received.
2016-11-30 15:10:53,677 [7] DEBUG MyDNSService - DNSService Error setting DNS: A system shutdown is in progress. (Exception from HRESULT: 0x8007045B)
So it appears that the OS is blocking my call to ManagementObject.InvokeMethod
I'm stymied. Is there a way around this issue? On startup my service detects the anomaly and recovers, but that takes too long. I'd really like to be able to shutdown gracefully.
Rocky, I just re-created the functional elements of your code (logging what's happening) but I'm not getting the error. I'm setting the DNSServerSearchOrder to null and feeding that to the SetDNSServerSearchOrder method of the management object. https://github.com/paulsbruce/StackOverflowExamples/blob/master/PriorityShutdown/PriorityShutdown/MyService.cs
My only additional recommendation is that you can try changing the priority of the shutdown order of your service to see if that has any effect. See this thread: .NET Windows Services stopping order when the system shutdown
I am having MSMQ on windows 2008. Messages are available in private queue. I have one WCF subscriber (written in C#) which is installed as windows service. Now problem is that sometimes the WCF subscriber stops picking messages from Queue. If I restart service again it works fine. Now I attached IError Handler to log the reason and exception.
Now to Handle this issue what I wanted to do is, I will set the recovery property to restart service on first failure and now problem is how to throw the error from HandleError() method of IErrorHandler class?
Please tell me best way to throw an exception in a window service so it can be restarted.
While it is probably better to address the underlying cause of your exceptions, it is certainly valid in certain scenarios to implement a fail fast methodology. Indeed, this ability to kill processes which have become "flawed" in some manner is critical to the concept of fault tolerance.
So, to make a windows service commit suicide:
void KillSelf()
{
try
{
// Code to close open connections/dispose
// of unmanaged resources etc
...
}
finally
{
Environment.Exit(1);
}
}
Service recovery options should be set to restart automatically. This will ensure your service comes straight back up again.
As far as I know one cannot throw an exception to restart a windows service.
I usually encapsulate a try catch (with logging) to prevent any exceptions crashing the service, which is the opposite to what you are suggesting.
It may be that you can catch an error and stop the service (not sure) and configure the service to restart if it stops?
I got a C# program which opens a EventWaitHandle like this to get triggered by a Windows Service.
EventWaitHandle sampleEventHandle = new EventWaitHandle(false, EventResetMode.AutoReset, "Global\\sampleEvent");
When the program now gets killed (or dies due to a unexpected error), the EventWaitHandle is not being closed and when restarting the program the following error occurs:
System.UnauthorizedAccessException: Access to the path is denied.
at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
at System.Threading.EventWaitHandle..ctor(Boolean initialState, EventResetMode mode, String name)
After a minute, the EventWaitHandle is getting closed and you can restart the Application.
Any ideas how to fix this issue?
This happens because the service has a handle opened on the same object. Which is to be expected, after all you are using this to implement signaling between the service and your program. The physical underlying Windows named object doesn't get removed from global namespace until the last handle is closed.
So getting an exception on this gives you a real problem diagnostic, the service is using the wrong handle and can never communicate with you when you restart your program. How this gets resolved after a minute is hard to guess, I have to assume that the service periodically calls OpenExisting().
The solution is simple: it should be the service that creates the wait handle and your UI program should call OpenExisting(). The event now always exists, at least as long as the service is alive. And it if doesn't then OpenExisting() gives you an excellent diagnostic of that with a WaitHandleCannotBeOpenedException
I am creating a Windows Service app that I would like to have programmatically pause when either a system error, odbc connection, or missing file error occur while . I was wondering if anyone knows how to do this? The Windows service app uses an odbc connection and datareader to connect to an MS Access database and an Oracle table, so there are the probable errors that I would be handling with those, I just want to allow a pause for the user handle the errors if/when they occur.
ServiceController service = new ServiceController(serviceName);
TimeSpan timeout = TimeSpan.FromMilliseconds(timeoutValue);
service.Pause(); //or whatever you want here.
sevice.WaitForStatus(ServiceControllerStatus.Paused, timeout);
...
Then to restart, do the same thing except for
service.Continue();
sevice.WaitForStatus(ServiceControllerStatus.Running, timeout);
You can do this for any state you want. Check out the msdn documentation by googling SeviceController. It will be the first result returned.
Also, you will need to handle the OnPause and OnContinue events in your service.
Have you tried?
System.Threading.Thread.Sleep(1000); // sleep for 1 second
Adjust the 1000 to 1000 times however long you want it to sleep in seconds.
Assuming that your service has a continual loop that checks for data, add a check to an external source for pause/continue commands. This source can be a message queue like MSMQ or a database table.
I implemented something along like this by having my service continually check a table for commands, and reporting its status in another table. When it gets a start command it launches a processing loop on another thread. A stop command causes it to signal the thread to gracefully exit. The service core never stops running.
The user interacts via a separate app with a UI that lets them view the service's status and submit commands. Since the app does its control via a database it doesn't have to run on the same machine that the service is running on.