Develop Reference

Terminating Thread Running an Event

I wrote an API that automates a certain website. However, on the testing stage, I noticed that (not very sure), my thread is not being terminated correctly.
I am using the WebBrowser object to navigate inside a thread, so that it works synchronously with my program:
private void NavigateThroughTread(string url)
{
Console.WriteLine("Defining thread...");
var th = new Thread(() =>
{
_wb = new WebBrowser();
_wb.DocumentCompleted += PageLoaded;
_wb.Visible = true;
_wb.Navigate(url);
Console.WriteLine("Web browser navigated.");
Application.Run();
});
Console.WriteLine("Thread defined.");
th.SetApartmentState(ApartmentState.STA);
Console.WriteLine("Before thread start...");
th.Start();
Console.WriteLine("Thread started.");
while (th.IsAlive) { }
Console.WriteLine("Journey ends.");
}
private void PageLoaded(object sender, WebBrowserDocumentCompletedEventArgs e)
{
Console.WriteLine("Pages loads...");
.
.
.
switch (_action)
{
.
.
.
case ENUM.FarmActions.Idle:
_wb.Navigate(new Uri("about:blank"));
_action = ENUM.FarmActions.Exit;
return;
case ENUM.FarmActions.Exit:
Console.WriteLine("Disposing wb...");
_wb.DocumentCompleted -= PageLoaded;
_wb.Dispose();
break;
}
Application.ExitThread(); // Stops the thread
}
Here is how I call this function:
public int Attack(int x, int y, ArmyBuilder army)
{
// instruct to attack the village
_action = ENUM.FarmActions.Attack;
//get the army and coordinates
_army = army;
_enemyCoordinates[X] = x;
_enemyCoordinates[Y] = y;
//Place the attack command
_errorFlag = true; // the action is not complated, the flag will set as false once action is complete
_attackFlag = false; // attack is not made yet
Console.WriteLine("Journey starts");
NavigateThroughTread(_url.GetUrl(ENUM.Screens.RallyPoint));
return _errorFlag ? -1 : CalculateDistance();
}
So the problem is, when I call the Attack function, couple times like this:
_command.Attack(509, 355, new ArmyBuilder(testArmy_lc));
_command.Attack(509, 354, new ArmyBuilder(testArmy_lc));
_command.Attack(505, 356, new ArmyBuilder(testArmy_lc));
_command.Attack(504, 356, new ArmyBuilder(testArmy_lc));
_command.Attack(504, 359, new ArmyBuilder(testArmy_lc));
_command.Attack(505, 356, new ArmyBuilder(testArmy_lc));
_command.Attack(504, 356, new ArmyBuilder(testArmy_lc));
_command.Attack(504, 359, new ArmyBuilder(testArmy_lc));
My application most of the times, gets stuck in one of these function (usually happens after the 4th or 5th). When it gets stuck the last log that I see is
Web browser navigated.
I assume it is something to do with termination of my thread. Can someone show me how I can run a thread which runs the DocumentCompleted event ?

I don't see any obvious reason for deadlock, nor did it reproduce at all when testing the code. There are a number of flaws in the code but nothing that yells "here!" loudly. I can only make recommendations:
Consider that you do not need a thread at all. The while (th.IsAlive) { } hot loop blocks your main thread while you wait for the browser code to finish the job. That is not a useful way to use a thread, you might as well use your main thread. This instantly eliminates a large number of potential hang causes.
The state logic in PageLoaded is risky. We cannot see all of it but one glaring issue is that you dispose the WebBrowser twice. If you have a case where you use return without a Navigate() call then you'll hang as described. No need to unsubscribe the event but same story, if you do unsubscribe but don't all Application.Exit() then you'll hang as described. State machines can be hard to debug, thorough logging is necessary. Minimize the risk by moving the Dispose() call and unsubscribing the event out of the logic, it doesn't belong there. And you need to test what happens when any Navigate() call ends up in failure, redirecting to a page you did not expect.
The _wb.Dispose() call is risky. Note that you destroy the WebBrowser while its DocumentCompleted event is in flight. Technically that can return code execution to code that is no longer alive or present. That can trip a race condition in the browser. As well as in the debugger, there is a dedicated MDA that checks for this problem. It is trivially avoided by moving the Dispose() call after the Application.Run() call where it belongs.
The while-loop burns 100% core, potentially starving the worker thread. Not a good enough reason to explain deadlock, but certainly unnecessary. Use Thread.Join() instead.
You create a lot of WebBrowser objects in this code. It is a very heavy object, as you can imagine, you need to keep an eye on memory usage in your program. Especially the unmanaged kind. If the browser leaks, like they so often do, you could technically create a scenario where the WB initializes okay but does not have enough memory left to load the page. Strongly favor using only one WB.
You need to consider that this might well be an environmental problem. On the top of that list is forever anti-malware and firewall, they always have a very good reason to treat a browser specially since that is the most common malware injection vector. You'll need to run your test with anti-malware and firewall disabled to ensure that it is not the cause of the hang.
Another environmental problem is one I noticed while testing this code, Google got sulky about me hitting it so often and started to throttle the requests, greatly slowing down the code. Talk to the web site owner and ask if he's got similar blocking or throttling counter-measures in place, most do. You need to test your state logic to verify that it still works properly when the browser redirects to an error page.
Yet another environmental issue is the WB will display a dialog itself in certain cases. This can deadlock in 3rd party code, very hard to diagnose. You should at least set the WebBrower.ScriptErrorsSuppressed to true but beware of Javascript code in the web page you load that itself creates new windows or displays alert dialogs. Using one WB is the workaround.
Keep in mind that your program can only be as reliable as your Internet connection and the web page server. That's not a terribly good place to be of course, both are quite out of your reach and you don't get nice exceptions to help you diagnose such a failure. And consider that you probably have not yet tested your program well enough yet to check if it can survive such a failure, it doesn't happen enough.
Quite a laundry list, focus first on eliminating the unnecessary thread and temporarily suppressing anti-malware. That's quick, focus next on using only one WebBrowser.

Hans thank you, I was able to fix this issue with one of your ideas. As you spent your time giving me a long answer, I wanted respond in same manner.
2 - I built the state machine structure carefully and with a lot logs (you can see it from my git account) also did a lot of debugs. I am sure that after I'm done navigating, I use Application.ExitThread() and wb.Dispose() only once.
3 - I tried placing the wb.Dispose() outside the event, however I couldn't find any other place where the Thread is still alive. If I try disposing WebBrowser outside the thread which is created inside the thread, the application gives me an error.
4 - I changed the code while (th.IsAlive) { } with th.Join(2000) this is absolutely a better idea but did not change anything. It optimized the code and as you mentioned, it prevented burning 100% core of my CPU.
5 - I tried using a single WebBrowser object which is instantiated in the constructor. However when I tried to navigate inside the thread, the application wouldnt even fire the events anymore. For some reason, I couldn't make it running whit a single WB object.
6,7 - I tested my application with different PC's and diffrent networks(with firewall and non-firewall protection). I changed windows firewall options as well but no travail. On my original code I do have _wb.ScriptErrorsSuppressed = true; so this shouldn't also be the issue.
8,9 - If these are the reasons, I can't do anything about it. But I doubt the real problem is caused because of them.
1 - This one was a good suggestion. I tried implementing my code without using a thread and it is now working fine. Here is how it looks like (still needs a lot optimization)
// Constructer
public FarmActions(string token)
{
// set the urls using the token
_url = new URL(token);
// define web browser properties
_wb = new WebBrowser();
_wb.DocumentCompleted += PageLoaded;
_wb.Visible = true;
_wb.AllowNavigation = true;
_wb.ScriptErrorsSuppressed = true;
}
public int Attack(int x, int y, ArmyBuilder army)
{
// instruct to attack the village
_action = ENUM.FarmActions.Attack;
//get the army and coordinates
_army = army;
_enemyCoordinates[X] = x;
_enemyCoordinates[Y] = y;
//Place the attack command
_errorFlag = true; // the action is not complated, the flag will set as false once action is complete
_attackFlag = false; // attack is not made yet
_isAlive = true;
Console.WriteLine("-------------------------");
Console.WriteLine("Journey starts");
NavigateThroughTread(_url.GetUrl(ENUM.Screens.RallyPoint));
return _errorFlag ? -1 : CalculateDistance();
}
private void NavigateThroughTread(string url)
{
Console.WriteLine("Defining thread...");
_wb.Navigate(url);
while (_isAlive) Application.DoEvents();
}
private void PageLoaded(object sender, WebBrowserDocumentCompletedEventArgs e)
{
Console.WriteLine("Pages loads...");
.
.
.
switch (_action)
{
.
.
.
case ENUM.FarmActions.Idle:
_wb.Navigate(new Uri("about:blank"));
_action = ENUM.FarmActions.Exit;
return;
case ENUM.FarmActions.Exit:
break;
}
_isAlive = false;
}
This is how I was able to wait without using a thread.
The main problem was probably as you mentioned in number 3 or 5. But I wasn't able to fix the problem as I spent couple of hours.
Anyway thanks for your help it works.

Faster way of searching for Top Level Windows in WPF CodedUI

I have several parts of my application that are supposed to close a window.
Checking that these windows have been closed using Coded-UI is incredibly slow. Right now my code looks like this:
Assert.IsFalse(UIMap.SomeWindow.TryFind(),
"X Window found when should be closed");
The problem is, this takes around 30s to search, and there are around 5 times this is used, and I have around 10 similar windows all being tested. I'd like to trim this time if possible, as it's making my tests slow.
I have also tried a dynamic solution (which is basically identical to the UIMap implementation):
var window = new WpfWindow();
window.SearchProperties.Add(UITestControl.PropertyNames.Name, "Window Title");
Assert.IsFalse(window.TryFind());
This is just as slow. It would be nice to use ApplicationUnderTest as a search parent, but as the window is Top Level, it doesn't seem to work.
Surely it shouldn't be too hard just to look at the open windows on my system (5), and check their titles against the search parameter?
Edit: Using SearchConfiguration.VisibleOnly doesn't seem to help either.

Found my answer, surprisingly on LinkedIn.
Now using:
Playback.PlaybackSettings.SearchTimeout = 1000; //in ms
Playback.PlaybackSettings.ShouldSearchFailFast = true;
Source: https://www.linkedin.com/grp/post/3828241-5843659258196959234
(C&P):
//Search Settings
// Value is in milliseconds. Default search timeout is 2 minutes.
// The search engine will continue making passes until the timeout has expired
// or the window has been found.
settings.SearchTimeout = 10000;
// Default search will make 3 attempts.
// If true the default 3 attempts is applied. If false then only one attempt should take place.
settings.ShouldSearchFailFast = true;

I think there might be a better answer. If set a global configuration like that and have to deal with a WPF Table and find a specific cell you might not find it working.
Using Window name generally isn't a great idea if there is any dynamic titles, Control Name is a good constant. It sounds to me like you are passing bad SearchProperties. You can use DrawHighlight() to see if CUI is actually finding your controls. First pass in the main parent window to your close window method, then use it as a try.
public static WinWindow _mainParent(string MainParentCtlName)
{
var _mainForm = new WinWindow();
_mainForm.SearchProperties.Add("ControlName", MainParentCtlName);
return _mainForm;
}
public static void CloseWindow(string MainWinCtlName)
{
var close = new WinButton(_mainParent(MainWinCtlName));
close.SearchProperties.Add("Name", "Close");
Mouse.Click(close);
}
try
{CloseWindow("MainWindowForm")}
catch{}

Monitor web pages access

I hope I can get some help.
I’m trying to create an host based application using C# (in the simplest way) to monitor access to a web page from the computer that hosts the application, if this web page is accessed while the program is running an event should rise.
So far I have used the SHDocVw.ShellWindows() but it works only if the web page has already been accessed not while is being accessed.
It monitors Windows Internet Explorer
I have also researched the httplistener but to no avail.
Do you have any solution?
Please let me know if you require more details

This may or may not be valid for your situation, but I had to do something similar with an Intranet website (cross-browser so it was a little harder than just with IE) recently. My solution was to setup a client-side application which hosts a WCF service. Then, when the user clicks a link on the web page (or raises any event, such as, $(document).ready) it sends an message back to the server telling the server to connect to the IP address associated with the current session (really just the IP Address on the request) on a known port. This connection is made to the client side application which is listening at that IP address and port for instructions on what to do (in my case it is dynamically compiling code in the request and running it).
That of course will only work for Intranet websites. A more general approach that will work for IE across the internet, is to create a IE extension (or maybe a Silverlight application) that talks on localhost. I've never done it, so I can't tell you how or if it is actually possible (but in principle seems possible).
If you don't have access to the website at all then perhaps using SharpPCAP or the Fiddler API would work for you.

Assuming the question is "I want to know when a program on my local computer accesses a give web page": A transparent http proxy is likely approach you want to take. Check out Fiddler to see if it is exactly what you want.
If your question is more "I want to know when a particular page is hit on my remote server": There are plenty of monitoring tools that parse web server logs and event logs to know state of the server. If you want to do something yourself and control the server's code - collect hit information for the page you are interested and provide some page that reports this data.

After few hours of work I have found a solution, not the most elegant one so far,(and at times causes a memory dump) but it does what I need.
Thanks
Just last edit, I solved the crash issue by adding a time so it checks the page every sec or so.
once again thanks for your iterest in my question.
class wait
{
private static System.Timers.Timer aTimer;
public void timed1()
{
// Create a timer with a ten second interval.
aTimer = new System.Timers.Timer(10000);
// Hook up the Elapsed event for the timer.
aTimer.Elapsed += new ElapsedEventHandler(OnTimedEvent);
// Set the Interval to 2 seconds (2000 milliseconds).
aTimer.Interval = 2000;
aTimer.Enabled = true;
Console.WriteLine("Press the Enter key to exit the program.");
Console.ReadLine();
}
private static void OnTimedEvent(object source, ElapsedEventArgs e)
{
//NetKeyLogger klog = new NetKeyLogger();
// Console.WriteLine("The Elapsed event was raised at {0}", e.SignalTime);
Kelloggs.Program KKA = new Kelloggs.Program();
SHDocVw.ShellWindows shellWindows = new SHDocVw.ShellWindows();
string filename;
foreach (SHDocVw.InternetExplorer ie in shellWindows)
{
filename = Path.GetFileNameWithoutExtension(ie.FullName).ToLower();
if (filename.Equals("iexplore"))
{
string ddd = (ie.LocationURL);
// Console.WriteLine(ddd);
if (ie.LocationURL == "http://www.testPage.com/")
{
Console.WriteLine("Page found");
// Console.ReadLine();
aTimer.Enabled = false;
KKA.Maino();
}
}

Trying to run multiple HTTP requests in parallel, but being limited by Windows (registry)

I'm developing an application (winforms C# .NET 4.0) where I access a lookup functionality from a 3rd party through a simple HTTP request. I call an url with a parameter, and in return I get a small string with the result of the lookup. Simple enough.
The challenge is however, that I have to do lots of these lookups (a couple of thousands), and I would like to limit the time needed. Therefore I would like to run requests in parallel (say 10-20). I use a ThreadPool to do this, and the short version of my code looks like this:
public void startAsyncLookup(Action<LookupResult> returnLookupResult)
{
this.returnLookupResult = returnLookupResult;
foreach (string number in numbersToLookup)
{
ThreadPool.QueueUserWorkItem(lookupNumber, number);
}
}
public void lookupNumber(Object threadContext)
{
string numberToLookup = (string)threadContext;
string url = #"http://some.url.com/?number=" + numberToLookup;
WebClient webClient = new WebClient();
Stream responseData = webClient.OpenRead(url);
LookupResult lookupResult = parseLookupResult(responseData);
returnLookupResult(lookupResult);
}
I fill up numbersToLookup (a List<String>) from another place, call startAsyncLookup and provide it with a call-back function returnLookupResult to return each result. This works, but I found that I'm not getting the throughput I want.
Initially I thought it might be the 3rd party having a poor system on their end, but I excluded this by trying to run the same code from two different machines at the same time. Each of the two took as long as one did alone, so I could rule out that one.
A colleague then tipped me that this might be a limitation in Windows. I googled a bit, and found amongst others this post saying that by default Windows limits the number of simultaneous request to the same web server to 4 for HTTP 1.0 and to 2 for HTTP 1.1 (for HTTP 1.1 this is actually according to the specification (RFC2068)).
The same post referred to above also provided a way to increase these limits. By adding two registry values to [HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Internet Settings] (MaxConnectionsPerServer and MaxConnectionsPer1_0Server), I could control this myself.
So, I tried this (sat both to 20), restarted my computer, and tried to run my program again. Sadly though, it didn't seem to help any. I also kept an eye on the Resource Monitor while running my batch lookup, and I noticed that my application (the one with the title blacked out) still only was using two TCP connections.
So, the question is, why isn't this working? Is the post I linked to using the wrong registry values? Is this perhaps not possible to "hack" in Windows any longer (I'm on Windows 7)?
And just in case anyone should wonder, I have also tried with different settings for MaxThreads on ThreadPool (everything from 10 to 100), and this didn't seem to affect my throughput at all, so the problem shouldn't be there either.

It is matter of ServicePoint. Which provides connection management for HTTP connections.
The default maximum number of concurrent connections allowed by a ServicePoint object is 2.
So if you need to increase it you can use ServicePointManager.DefaultConnectionLimit property. Just check the link in MSDN there you can see a sample. And set the value you need.

For quicker reference for someone. To increase the connection limit per host you can do this in your Main() or anytime before you begin making the HTTP requests.
System.Net.ServicePointManager.DefaultConnectionLimit = 1000; //or some other number > 4

Fire and forget this method from your main method. Icognito user is correct, only 2 threads are allowed to play at the same time.
private static void openServicePoint()
{
ServicePointManager.UseNagleAlgorithm = true;
ServicePointManager.Expect100Continue = true;
ServicePointManager.CheckCertificateRevocationList = true;
ServicePointManager.DefaultConnectionLimit = 10000;
Uri MS = new Uri("http://My awesome web site");
ServicePoint servicePoint = ServicePointManager.FindServicePoint(MS);
}

For Internet Explorer 8:
Run Registry Editor and navigate to following key HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Internet Explorer\MAIN\FeatureControl\FEATURE_MAXCONNECTION SPERSERVER
and
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Internet Explorer\MAIN\FeatureControl\FEATURE_MAXCONNECTION SPER1_0SERVER
If FEATURE_MAXCONNECTIONSPERSERVER and FEATURE_MAXCONNECTIONSPER1_0SERVER are missing then create them. Now create DWORD Value called iexplore.exe for both sub keys (listed above) and set their value to 10 or whatever number desired.

Better way to "dock" a third party running application inside a windows.forms panel?

I am currently doing this as follows:
// _Container is the panel that the program is to be displayed in.
System.Diagnostics.Process procTest = new System.Diagnostics.Process();
procTest.StartInfo.FileName = "TEST.EXE";
procTest.StartInfo.CreateNoWindow = false;
procTest.StartInfo.WindowStyle = ProcessWindowStyle.Normal;
procTest.Start();
procTest.WaitForInputIdle();
SetParent(procTest.MainWindowHandle, _Container.Handle);
MoveWindow(procTest.MainWindowHandle,
0, 0, _Container.Width, _Container.Height, true);
The problem I am having with this code is that some parts of the application UI no longer function properly once I change the MainWindowHandle (ie: buttons missing text).
Is there a way to do this without causing issues with the docked application? (Either through .net or user32)?

First of all, instead of simply waiting 1.5 seconds, try calling procTest.WaitForInputIdle to wait until its message loop is free. You already are.
In general, I don't think it's possible to do this without modifying the program that you're hosting.
EDIT: You could try to keep the other program above your hosting area by hiding in from the taskbar, removing its title bar, moving it as your program moves, etc. However, this still wouldn't work perfectly; I recommend that you try to find some alternative.
Try contacting the original developers of the third-party application and asking for their advice.