Develop Reference

Threads hang until i attach debugger

I have a service using WCF. Internally it has a dictionary with lists that you can add to or get a subset of from different endpoints.
The code is something like this:
List<Data> list = null;
try
{
locker.EnterReadLock();
list = internalData[Something].Where(x => x.hassomething()).ToList();
}
finally
{
locker.ExitReadLock();
}
foreach (var y in list)
{
result[y.proprty1].Add(y.property2); // <-- here it hangs
}
return result;
So the internalData is locked with a ReaderWriterLockSlim for all operations, readerlock for reading and writerlock for adding. I make a copy of the items inside the lock and work on this copy later.
The issue is after a while, more and more cpu-cores goes to 100% and finally is uses all cores. It can run perfectly for days and million calls before it stops.
Attaching debugger and pausing shows that one treads hang on adding to the result dictionary. But as soon as I resume all threads will continue and a lot of memory is being released.
Is there something special happening when a debugger is attached, pausing and resuming that will release something like this?

I changed my locks to lock(something) { code... } and the problem with hangs went away. So it looks lilke i ran into the issue Steffen Winkler pointed out in his comment. http://joeduffyblog.com/2007/02/07/introducing-the-new-readerwriterlockslim-in-orcas/

Terminating Thread Running an Event

I wrote an API that automates a certain website. However, on the testing stage, I noticed that (not very sure), my thread is not being terminated correctly.
I am using the WebBrowser object to navigate inside a thread, so that it works synchronously with my program:
private void NavigateThroughTread(string url)
{
Console.WriteLine("Defining thread...");
var th = new Thread(() =>
{
_wb = new WebBrowser();
_wb.DocumentCompleted += PageLoaded;
_wb.Visible = true;
_wb.Navigate(url);
Console.WriteLine("Web browser navigated.");
Application.Run();
});
Console.WriteLine("Thread defined.");
th.SetApartmentState(ApartmentState.STA);
Console.WriteLine("Before thread start...");
th.Start();
Console.WriteLine("Thread started.");
while (th.IsAlive) { }
Console.WriteLine("Journey ends.");
}
private void PageLoaded(object sender, WebBrowserDocumentCompletedEventArgs e)
{
Console.WriteLine("Pages loads...");
.
.
.
switch (_action)
{
.
.
.
case ENUM.FarmActions.Idle:
_wb.Navigate(new Uri("about:blank"));
_action = ENUM.FarmActions.Exit;
return;
case ENUM.FarmActions.Exit:
Console.WriteLine("Disposing wb...");
_wb.DocumentCompleted -= PageLoaded;
_wb.Dispose();
break;
}
Application.ExitThread(); // Stops the thread
}
Here is how I call this function:
public int Attack(int x, int y, ArmyBuilder army)
{
// instruct to attack the village
_action = ENUM.FarmActions.Attack;
//get the army and coordinates
_army = army;
_enemyCoordinates[X] = x;
_enemyCoordinates[Y] = y;
//Place the attack command
_errorFlag = true; // the action is not complated, the flag will set as false once action is complete
_attackFlag = false; // attack is not made yet
Console.WriteLine("Journey starts");
NavigateThroughTread(_url.GetUrl(ENUM.Screens.RallyPoint));
return _errorFlag ? -1 : CalculateDistance();
}
So the problem is, when I call the Attack function, couple times like this:
_command.Attack(509, 355, new ArmyBuilder(testArmy_lc));
_command.Attack(509, 354, new ArmyBuilder(testArmy_lc));
_command.Attack(505, 356, new ArmyBuilder(testArmy_lc));
_command.Attack(504, 356, new ArmyBuilder(testArmy_lc));
_command.Attack(504, 359, new ArmyBuilder(testArmy_lc));
_command.Attack(505, 356, new ArmyBuilder(testArmy_lc));
_command.Attack(504, 356, new ArmyBuilder(testArmy_lc));
_command.Attack(504, 359, new ArmyBuilder(testArmy_lc));
My application most of the times, gets stuck in one of these function (usually happens after the 4th or 5th). When it gets stuck the last log that I see is
Web browser navigated.
I assume it is something to do with termination of my thread. Can someone show me how I can run a thread which runs the DocumentCompleted event ?

I don't see any obvious reason for deadlock, nor did it reproduce at all when testing the code. There are a number of flaws in the code but nothing that yells "here!" loudly. I can only make recommendations:
Consider that you do not need a thread at all. The while (th.IsAlive) { } hot loop blocks your main thread while you wait for the browser code to finish the job. That is not a useful way to use a thread, you might as well use your main thread. This instantly eliminates a large number of potential hang causes.
The state logic in PageLoaded is risky. We cannot see all of it but one glaring issue is that you dispose the WebBrowser twice. If you have a case where you use return without a Navigate() call then you'll hang as described. No need to unsubscribe the event but same story, if you do unsubscribe but don't all Application.Exit() then you'll hang as described. State machines can be hard to debug, thorough logging is necessary. Minimize the risk by moving the Dispose() call and unsubscribing the event out of the logic, it doesn't belong there. And you need to test what happens when any Navigate() call ends up in failure, redirecting to a page you did not expect.
The _wb.Dispose() call is risky. Note that you destroy the WebBrowser while its DocumentCompleted event is in flight. Technically that can return code execution to code that is no longer alive or present. That can trip a race condition in the browser. As well as in the debugger, there is a dedicated MDA that checks for this problem. It is trivially avoided by moving the Dispose() call after the Application.Run() call where it belongs.
The while-loop burns 100% core, potentially starving the worker thread. Not a good enough reason to explain deadlock, but certainly unnecessary. Use Thread.Join() instead.
You create a lot of WebBrowser objects in this code. It is a very heavy object, as you can imagine, you need to keep an eye on memory usage in your program. Especially the unmanaged kind. If the browser leaks, like they so often do, you could technically create a scenario where the WB initializes okay but does not have enough memory left to load the page. Strongly favor using only one WB.
You need to consider that this might well be an environmental problem. On the top of that list is forever anti-malware and firewall, they always have a very good reason to treat a browser specially since that is the most common malware injection vector. You'll need to run your test with anti-malware and firewall disabled to ensure that it is not the cause of the hang.
Another environmental problem is one I noticed while testing this code, Google got sulky about me hitting it so often and started to throttle the requests, greatly slowing down the code. Talk to the web site owner and ask if he's got similar blocking or throttling counter-measures in place, most do. You need to test your state logic to verify that it still works properly when the browser redirects to an error page.
Yet another environmental issue is the WB will display a dialog itself in certain cases. This can deadlock in 3rd party code, very hard to diagnose. You should at least set the WebBrower.ScriptErrorsSuppressed to true but beware of Javascript code in the web page you load that itself creates new windows or displays alert dialogs. Using one WB is the workaround.
Keep in mind that your program can only be as reliable as your Internet connection and the web page server. That's not a terribly good place to be of course, both are quite out of your reach and you don't get nice exceptions to help you diagnose such a failure. And consider that you probably have not yet tested your program well enough yet to check if it can survive such a failure, it doesn't happen enough.
Quite a laundry list, focus first on eliminating the unnecessary thread and temporarily suppressing anti-malware. That's quick, focus next on using only one WebBrowser.

Hans thank you, I was able to fix this issue with one of your ideas. As you spent your time giving me a long answer, I wanted respond in same manner.
2 - I built the state machine structure carefully and with a lot logs (you can see it from my git account) also did a lot of debugs. I am sure that after I'm done navigating, I use Application.ExitThread() and wb.Dispose() only once.
3 - I tried placing the wb.Dispose() outside the event, however I couldn't find any other place where the Thread is still alive. If I try disposing WebBrowser outside the thread which is created inside the thread, the application gives me an error.
4 - I changed the code while (th.IsAlive) { } with th.Join(2000) this is absolutely a better idea but did not change anything. It optimized the code and as you mentioned, it prevented burning 100% core of my CPU.
5 - I tried using a single WebBrowser object which is instantiated in the constructor. However when I tried to navigate inside the thread, the application wouldnt even fire the events anymore. For some reason, I couldn't make it running whit a single WB object.
6,7 - I tested my application with different PC's and diffrent networks(with firewall and non-firewall protection). I changed windows firewall options as well but no travail. On my original code I do have _wb.ScriptErrorsSuppressed = true; so this shouldn't also be the issue.
8,9 - If these are the reasons, I can't do anything about it. But I doubt the real problem is caused because of them.
1 - This one was a good suggestion. I tried implementing my code without using a thread and it is now working fine. Here is how it looks like (still needs a lot optimization)
// Constructer
public FarmActions(string token)
{
// set the urls using the token
_url = new URL(token);
// define web browser properties
_wb = new WebBrowser();
_wb.DocumentCompleted += PageLoaded;
_wb.Visible = true;
_wb.AllowNavigation = true;
_wb.ScriptErrorsSuppressed = true;
}
public int Attack(int x, int y, ArmyBuilder army)
{
// instruct to attack the village
_action = ENUM.FarmActions.Attack;
//get the army and coordinates
_army = army;
_enemyCoordinates[X] = x;
_enemyCoordinates[Y] = y;
//Place the attack command
_errorFlag = true; // the action is not complated, the flag will set as false once action is complete
_attackFlag = false; // attack is not made yet
_isAlive = true;
Console.WriteLine("-------------------------");
Console.WriteLine("Journey starts");
NavigateThroughTread(_url.GetUrl(ENUM.Screens.RallyPoint));
return _errorFlag ? -1 : CalculateDistance();
}
private void NavigateThroughTread(string url)
{
Console.WriteLine("Defining thread...");
_wb.Navigate(url);
while (_isAlive) Application.DoEvents();
}
private void PageLoaded(object sender, WebBrowserDocumentCompletedEventArgs e)
{
Console.WriteLine("Pages loads...");
.
.
.
switch (_action)
{
.
.
.
case ENUM.FarmActions.Idle:
_wb.Navigate(new Uri("about:blank"));
_action = ENUM.FarmActions.Exit;
return;
case ENUM.FarmActions.Exit:
break;
}
_isAlive = false;
}
This is how I was able to wait without using a thread.
The main problem was probably as you mentioned in number 3 or 5. But I wasn't able to fix the problem as I spent couple of hours.
Anyway thanks for your help it works.

CPU is 100% at multithreading

First I've read all the posts here regarding this issue and I manged to progress a bit. However it seems I do need your help :)
I have a program with several threads, sometimes (not always) the CPU usage of the program is increasing up to 100% and never reduced until I shut down the program.
As I read in other similar posts, I ran the app using the visual studio (2012 - Ultimate).
I paused the app, and open the threads window.
There I pauses the threads until I've found the 4 threads which stuck the app.
The all refer to the same line of code (a call for constructor).
I checked the constructor inside and outside and couldn't find any loop which could cause it.
To be more careful I've added break point to almost every line of code and resume the app. None of them have been triggered.
This is the line of code:
public static void GenerateDefacementSensors(ICrawlerManager cm)
{
m_SensorsMap = new Dictionary<DefacementSensorType, DefacementSensor>();
// Create instance of all sensors
// For any new defacement sensor, don't forget to add an appropriate line here
// m_SensorsMap.add(DefacementSensorType.[Type], new [Type]Sensor())
try
{
if (m_SensorsMap.Count <= 0)
{
m_SensorsMap.Add(DefacementSensorType.BackgroundSensor, new BackgroundSensor());
m_SensorsMap.Add(DefacementSensorType.TaglinesSensor, new TaglinesSensor(cm.Database));
m_SensorsMap.Add(DefacementSensorType.SingleImageSensor, new SingleImageSensor());
}
}
catch (Exception)
{
Console.WriteLine("There was a problem initializing defacement sensors");
}
}
The second "m_SensorsMap.Add" is marked with green arrow, as I understand it, it means it's still waiting to the first line to finish.
By the way, the m_SensorsMap.Count value is 3.
How can I find the problem?
Is it a loop?
Or maybe a deadlock (not make sense because it shouldn't be 100% cpu, right?)
It's pointless to upload a code because this is a huge project.
I need more general help like how to debug?
Is it could something else than a loop?
Because it's a bug that returns every while and than I'm not closing the app until I found the problem :)
Thanks in advance!!
Edit:
The constructors:
public TaglinesSensor(IDatabase db)
{
m_DB = db;
}

I couldn't found the problem so I've changed the design on order not to call those constructors anymore.
Thanks for the guys who tried to help.
Shaul

.NET Denying Access to Directories After Copy Operations

I'm performing a "safe" copy of a directory over another directory as follows:
Given the source C:\Source and target C:\Target
Copy C:\Source to C:\Target-incoming
Move C:\Target (if it exists) to C:\Target-outgoing
Move C:\Target-incoming to C:\Target
Delete C:\Target-outgoing (if it exists)
If any of the first three steps fail, I'll attempt to put things back as they were to prevent data loss.
However, the move of C:\Target-incoming to C:\Target fails with "Access to the path C:\Target-incoming is denied" most of the time.
At the moment, inserting Thread.Sleep(100) just before the move operation fixes the problem for me. However, waiting .1 of a second seems ridiculous to me. Thread.Sleep(10) isn't enough to fix it. I also have the sinking feeling that the value I have to wait depends on the speed of disk IO.
So, my questions:
Can I prevent this from happening?
If not, is there a way of finding out when the lock on the directory is released after copying it?
Edit: For clarity, I'm doing all these operations in one method on one thread, and I'm just using Thread.Sleep() to pause code flow for a moment. The moves and copies are being done with standard .NET Directory.Move(), Directory.CreateDirectory() and File.CopyTo() methods. It would appear that the .NET methods are returning before the locks on the respective files are being released, causing the necessity to wait an amount of time before continuing.

What could be happening is probably that your thread is trying to "Move C:\Target-incoming to C:\Target" WHILE the "Move C:\Target to C:\Target-outgoing" is NOT finished YET.
This track is confirmed by the success of your process after short Thread Sleep.
Try to Chain your processes, i.e : Divide each step into specific methods, and call the methods one after the other (sync'ing the start of a method to the end of the previous one)
There are various ways to do that (among others syncing/locking/chaining different threads per process/step)
You could check Thread Synchronization in .NET
But of course, this is not the only possible cause for your problem.

After a bunch of testing, it seems like the very act of trying to move a locked folder gets the OS to hurry up and release the lock, even if the first attempt fails.
I wrote this extension method to DirectoryInfo:
public static void TryToMoveTo(this DirectoryInfo o, string targetPath) {
int attemptsRemaining = 5;
while (true) {
try {
o.MoveTo(targetPath);
break;
} catch (Exception) {
if (attemptsRemaining == 0) {
throw;
} else {
attemptsRemaining--;
System.Threading.Thread.Sleep(10);
}
}
}
}
While debugging the original problem, I settled on waiting for 100ms as anything less seemed to cause exceptions (I tried 10, 25, 50, 75 and 100ms). However, in the method above I wait 10ms before retrying, and I never, ever got more than one exception thrown in each of my hundreds of test runs.

You can always try waiting in a loop, up till a maximum # of tries. You can check to see if the directory is locked by calling CreateFile and checking it's return code. Be sure to read through the "flags" section of the docs because you need to pass in a special flag to open a directory.
Someone else mentioned in a comment that you may want to try Transactional NTFS. If you can, you might want to try that.

check wethere source and target directories exist before copying or moving using io.directory.exists
the access deneied error is caused by either source or target are not found.

Process.Kill() related crashes

Or not!
I have a fairly simple application timer program. The program will launch a user selected (from file dialog) executable and then terminate the process after the user specified number of minutes. During testing I found that a crash occurs when I call the Process.Kill() method and the application is minimized to the system tray.
The executable in question is Frap.exe which I use frequently and is the reason I wrote the app timer in the first place. I always minimize fraps to the tray, and this is when the crash occurs.
My use of Kill() is straight forward enough...
while (true)
{
//keep checking if timer expired or app closed externally (ie. by user)
if (dtEndTime <= DateTime.Now || p.HasExited)
{
if (!p.HasExited)
p.Kill();
break;
}
System.Threading.Thread.Sleep(500);
}
In searching for alternatives methods to close an external application programmatically, I found only Close() and Kill() (CloseMainWindow is not helpful to me at all). I tried using Close(), which works providing the application is minimized the tray. If the app is minimized, Close() doesn't cause a crash but the app remains open and active.
One thing I noticed in a few posts posts regarding closing external applications was the comment: "Personally I'd try to find a more graceful way of shutting it down though." made in THIS thread found here at stack flow (no offense to John). Thing is, I ran across comments like that on a few sites, with no attempt at what a graceful or elegant (or crash-free!!) method might be.
Any suggestions?
The crash experienced is not consistant and I've little to offer as to details. I am unable to debug using VS2008 as I get message - cant debug crashing application (or something similar), and depending on what other programs I have running at the time, when the Kill() is called some of them also crash (also programs only running in the tray) so I'm thinking this is some sort of problem specifically related to the system tray.

Is it possible that your code is being executed in a way such that the Kill() statement could sometimes be called twice? In the docs for Process.Kill(), it says that the Kill executes asynchronously. So, when you call Kill(), execution continues on your main thread. Further, the docs state that Kill will throw a Win32Exception if you call it on an app that is already in the process of closing. The docs state that you can use WaitForExit() to wait for the process to exit. What happens if you put a call to WaitForExit() immediately following the call to Kill(). The loop looks ok (with the break statement). Is it possible that you have code entering that loop twice?
If that's not the problem, maybe there is another way to catch that exception:
Try hooking the AppDomain.CurrentDomain.UnhandledException event
(currentDomain is a static member)
The problem is that Kill runs asynchronously, so if it's throwing an exception, it's occurring on a different thread. That's why your exception handler doesn't catch it. Further (I think) that an unhandled async exception (which is what I believe you have) will cause an immediate unload of your application (which is what is happening).
Edit: Example code for hooking the UnhandledExceptionEvent
Here is a simple console application that demonstrates the use of AppDomain.UnhandledException:
using System;
public class MyClass
{
public static void Main()
{
System.AppDomain.CurrentDomain.UnhandledException += MyExceptionHandler;
System.Threading.ThreadPool.QueueUserWorkItem(DoWork);
Console.ReadLine();
}
private static void DoWork(object state)
{
throw new ApplicationException("Test");
}
private static void MyExceptionHandler(object sender, System.UnhandledExceptionEventArgs e)
{
// get the message
System.Exception exception = e.ExceptionObject as System.Exception;
Console.WriteLine("Unhandled Exception Detected");
if(exception != null)
Console.WriteLine("Message: {0}", exception.Message);
// for this console app, hold the window open until I press enter
Console.ReadLine();
}
}

My first thought is to put a try/catch block around the Kill() call and log the exception you get, if there is one. It might give you a clue what's wrong. Something like:
try
{
if(!p.HasExited)
{
p.Kill();
}
break;
}
catch(Exception ex)
{
System.Diagnostics.Trace.WriteLine(String.Format("Could not kill process {0}, exception {1}", p.ToString(), ex.ToString()));
}

I dont think I should claim this to be "THE ANSWER" but its a decent 'work around'. Adding the following to lines of code...
p.WaitForInputIdle(10000);
am.hWnd = p.MainWindowHandle;
...stopped the crashing issue. These lines were placed immediately after the Process.Start() statement. Both lines are required and in using them I opened the door to a few other questions that I will be investigating over the next few days. The first line is just an up-to 10 second wait for the started process to go 'idle' (ie. finish starting). am.hWnd is a property in my AppManagement class of type IntPtr and this is the only usage of both sides of the assignment. For lack of better explaination, these two lines are analguous to a debouncing method.
I modified the while loop only slightly to allow for a call to CloseMainWindow() which seems to be the better route to take - though if it fails I then Kill() the app:
while (true)
{
//keep checking if timer expired or app closed externally (ie. by user)
if (dtEndTime <= DateTime.Now || p.HasExited) {
try {
if (!p.HasExited) // if the app hasn't already exitted...
{
if (!p.CloseMainWindow()) // did message get sent?
{
if (!p.HasExited) //has app closed yet?
{
p.Kill(); // force app to exit
p.WaitForExit(2000); // a few moments for app to shut down
}
}
p.Close(); // free resources
}
}
catch { // blah blah }
break;
}
System.Threading.Thread.Sleep(500);
}
My initial intention for getting the MainWindowHandle was to maximize/restore an app if minimized and I might implement that in the near future. I decided to see if other programs that run like Fraps (ie, a UI but mostly run in the system tray (like messanger services such as Yahoo et al.)). I tested with XFire and nothing I could do would return a value for the MainWindowHandle. Anyways, this is a serperate issue but one I found interesting.
PS. A bit of credit to JMarsch as it was his suggestion RE: Win32Exception that actually lead me to finding this work around - as unlikely as it seems it true.