I've got a single statement running on a ASP.NET page that takes a long time (by long I mean 100ms, which is too long if I want this page to be lightning fast) , and I don't care when it executes as long as executes.
What is the best (and hopefully easiest) way to accomplish this?
The easiest way is probably to get it to execute in the threadpool. For example, to make this asynchronous:
using System;
using System.Threading;
class Test
{
static void ExecuteLongRunningTask()
{
Console.WriteLine("Sleeping...");
Thread.Sleep(1000);
Console.WriteLine("... done");
}
static void Main()
{
ExecuteLongRunningTask();
Console.WriteLine("Finished");
Console.ReadLine();
}
}
Change the first line of Main to:
ThreadPool.QueueUserWorkItem(x => ExecuteLongRunningTask());
Be aware that if you pass any arguments in, they'll be captured variables - that could have subtle side-effects in some cases (particularly if you use it in a loop).
If it's 100ms, then don't bother. Your users can't detect a 100ms delay.
Edit: some explanations.
If I remember correctly, 100 milliseconds (1/10 second) is near the minimum amount of time that a human can perceive. So, for the purpose of discussion, let me grant that this 100ms can be perceived by the users of the OP's site, and that it is worthwhile to improve performance by 100ms. I assumed from the start that the OP had correctly identified this "long-running" task as a potential source of a 100ms improvement. So, why did I suggest he ignore it?
Dealing with multiple threads properly is not easy, and is a source of bugs that are difficult to track down. Adding threads to a problem is usually not a solution, but is rather a source of other problems you simply don't find right away (*).
I had the opportunity once to learn this the hard way, with a bug that could only be reproduced on the fastest eight-cpu system available at the time, and then only by pounding on the thing relentlessly, while simulating a degree of network failure that would have caused the network administrators to be lined up and shot, if it had happened in real life. The bug turned out to be in the Unix OS kernel handling of signals, and was a matter of the arrangement of a handful of instructions.
Granted I've never seen anything that bad since then, I've still seen many developers tripped up by multithreading bugs. This question, seemed to be, on the one hand, asking for an "easy way out" via threading, and on the other hand, the benefit was only 100ms. Since it did not appear that the OP already had a well-tested threading infrastructure, it seemed to me that it made better sense to ignore the 100ms, or perhaps to pick up performance some other way.
(*) Of course, there are many circumstances where an algorithm can profitably be made parallel and executed by multiple threads, running on multiple cores. But it does not sound like the OP has such a case.
There's a background worker thread that can be very useful when processing information in the background of a .NET application. A bit more context would help answer the question better.
// This is the delegate to use to execute the process asynchronously
public delegate void ExecuteBackgroundDelegate();
//
[STAThread]
static void Main(string[] args)
{
MyProcess proc = new MyProcess();
// create an instance of our execution delegate
ExecuteBackgroundDelegate asynchExec = new ExecuteBackgroundDelegate(proc.Execute);
// execute this asynchronously
asynchExec.BeginInvoke(null, null);
}
If you mean to say that the response can be sent before the command is ran, you could use ThreadPool.QueueUserWorkItem to run it on another thread without blocking your request.
you can use a ThreadPool.
ThreadPool.QueueUserWorkItem( o => Thread.Sleep(1000) /*your long task*/ );
class Test
{
void LongRunningTask()
{
Console.WriteLine("Sleeping...");
Thread.Sleep(10000);
Console.WriteLine("... done");
}
static void Main()
{
Test t = new Test();
new Action(() => t.LongRunningTask()).BeginInvoke(null, t);
}
}
I wouldn't bother with any of that threading stuff for a 100ms delay. This will do fine:
protected void Page_Unload(object sender, EventArgs e)
{
HttpContext.Current.Response.Flush();
HttpContext.Current.Response.Close();
// Your code here
}
(copied from my earlier answer for this question)
Since the connection to the client downloading your page will be closed, their browser will stop displaying a loading message. Your code will continue execution normally however.
Related
I was having a discussion with a teammate about locking in .NET. He's a really bright guy with an extensive background in both lower-level and higher-level programming, but his experience with lower level programming far exceeds mine. Anyway, He argued that .NET locking should be avoided on critical systems expected to be under heavy-load if at all possible in order to avoid the admittedly small possibility of a "zombie thread" crashing a system. I routinely use locking and I didn't know what a "zombie thread" was, so I asked. The impression I got from his explanation is that a zombie thread is a thread that has terminated but somehow still holds onto some resources. An example he gave of how a zombie thread could break a system was a thread begins some procedure after locking on some object, and then is at some point terminated before the lock can be released. This situation has the potential to crash the system, because eventually, attempts to execute that method will result in the threads all waiting for access to an object that will never be returned, because the thread that is using the locked object is dead.
I think I got the gist of this, but if I'm off base, please let me know. The concept made sense to me. I wasn't completely convinced that this was a real scenario that could happen in .NET. I've never previously heard of "zombies", but I do recognize that programmers who have worked in depth at lower levels tend to have a deeper understanding of computing fundamentals (like threading). I definitely do see the value in locking, however, and I have seen many world class programmers leverage locking. I also have limited ability to evaluate this for myself because I know that the lock(obj) statement is really just syntactic sugar for:
bool lockWasTaken = false;
var temp = obj;
try { Monitor.Enter(temp, ref lockWasTaken); { body } }
finally { if (lockWasTaken) Monitor.Exit(temp); }
and because Monitor.Enter and Monitor.Exit are marked extern. It seems conceivable that .NET does some kind of processing that protects threads from exposure to system components that could have this kind of impact, but that is purely speculative and probably just based on the fact that I've never heard of "zombie threads" before. So, I'm hoping I can get some feedback on this here:
Is there a clearer definition of a "zombie thread" than what I've explained here?
Can zombie threads occur on .NET? (Why/Why not?)
If applicable, How could I force the creation of a zombie thread in .NET?
If applicable, How can I leverage locking without risking a zombie thread scenario in .NET?
Update
I asked this question a little over two years ago. Today this happened:
Is there a clearer definition of a "zombie thread" than what I've explained here?
Seems like a pretty good explanation to me - a thread that has terminated (and can therefore no longer release any resources), but whose resources (e.g. handles) are still around and (potentially) causing problems.
Can zombie threads occur on .NET? (Why/Why not?)
If applicable, How could I force the creation of a zombie thread in .NET?
They sure do, look, I made one!
[DllImport("kernel32.dll")]
private static extern void ExitThread(uint dwExitCode);
static void Main(string[] args)
{
new Thread(Target).Start();
Console.ReadLine();
}
private static void Target()
{
using (var file = File.Open("test.txt", FileMode.OpenOrCreate))
{
ExitThread(0);
}
}
This program starts a thread Target which opens a file and then immediately kills itself using ExitThread. The resulting zombie thread will never release the handle to the "test.txt" file and so the file will remain open until the program terminates (you can check with process explorer or similar). The handle to "test.txt" won't be released until GC.Collect is called - it turns out it is even more difficult than I thought to create a zombie thread that leaks handles)
If applicable, How can I leverage locking without risking a zombie thread scenario in .NET?
Don't do what I just did!
As long as your code cleans up after itself correctly (use Safe Handles or equivalent classes if working with unmanaged resources), and as long as you don't go out of your way to kill threads in weird and wonderful ways (safest way is just to never kill threads - let them terminate themselves normally, or through exceptions if necessary), the only way that you are going to have something resembling a zombie thread is if something has gone very wrong (e.g. something goes wrong in the CLR).
In fact its actually surprisingly difficult to create a zombie thread (I had to P/Invoke into a function that esentially tells you in the documentation not to call it outside of C). For example the following (awful) code actually doesn't create a zombie thread.
static void Main(string[] args)
{
var thread = new Thread(Target);
thread.Start();
// Ugh, never call Abort...
thread.Abort();
Console.ReadLine();
}
private static void Target()
{
// Ouch, open file which isn't closed...
var file = File.Open("test.txt", FileMode.OpenOrCreate);
while (true)
{
Thread.Sleep(1);
}
GC.KeepAlive(file);
}
Despite making some pretty awful mistakes, the handle to "test.txt" is still closed as soon as Abort is called (as part of the finalizer for file which under the covers uses SafeFileHandle to wrap its file handle)
The locking example in C.Evenhuis answer is probably the easiest way to fail to release a resource (a lock in this case) when a thread is terminated in a non-weird way, but thats easily fixed by either using a lock statement instead, or putting the release in a finally block.
See also
Subtleties of C# IL
codegen
for a very subtle case where an exception can prevent a lock from
being released even when using the lock keyword (but only in .Net 3.5 and earlier)
Locks and exceptions do not
mix
I've cleaned up my answer a bit, but left the original one below for reference
It’s the first time I've heard of the term zombies so I'll assume its definition is:
A thread that has terminated without releasing all of its resources
So given that definition, then yes, you can do that in .NET, as with other languages (C/C++, java).
However, I do not think this as a good reason not to write threaded, mission critical code in .NET. There may be other reasons to decide against .NET but writing off .NET just because you can have zombie threads somehow doesn't make sense to me. Zombie threads are possible in C/C++ (I'd even argue that it’s a lot easier to mess up in C) and a lot of critical, threaded apps are in C/C++ (high volume trading, databases etc).
Conclusion
If you are in the process of deciding on a language to use, then I suggest you take the big picture into consideration: performance, team skills, schedule, integration with existing apps etc. Sure, zombie threads are something that you should think about, but since it’s so difficult to actually make this mistake in .NET compared to other languages like C, I think this concern will be overshadowed by other things like the ones mentioned above. Good luck!
Original Answer
Zombies† can exist if you don't write proper threading code. The same is true for other languages like C/C++ and Java. But this is not a reason not to write threaded code in .NET.
And just like with any other language, know the price before using something. It also helps to know what is happening under the hood so you can foresee any potential problems.
Reliable code for mission critical systems is not easy to write, whatever language you're in. But I'm positive it’s not impossible to do correctly in .NET. Also AFAIK, .NET threading is not that different from threading in C/C++, it uses (or is built from) the same system calls except for some .net specific constructs (like the light weight versions of RWL and event classes).
†first time I've heard of the term zombies but based on your description, your colleague probably meant a thread that terminated without release all resources. This could potentially cause a deadlock, memory leak or some other bad side effect. This is obviously not desirable but singling out .NET because of this possibility is probably not a good idea since it’s possible in other languages too. I'd even argue that it’s easier to mess up in C/C++ than in .NET (especially so in C where you don't have RAII) but a lot of critical apps are written in C/C++ right? So it really depends on your individual circumstances. If you want to extract every ounce of speed from your application and want to get as close to bare metal as possible, then .NET might not be the best solution. If you are on a tight budget and do a lot of interfacing with web services/existing .net libraries/etc then .NET may be a good choice.
Right now most of my answer has been corrected by the comments below. I won't delete the answer because I need the reputation points because the information in the comments may be valuable to readers.
Immortal Blue pointed out that in .NET 2.0 and up finally blocks are immune to thread aborts. And as commented by Andreas Niedermair, this may not be an actual zombie thread, but the following example shows how aborting a thread can cause problems:
class Program
{
static readonly object _lock = new object();
static void Main(string[] args)
{
Thread thread = new Thread(new ThreadStart(Zombie));
thread.Start();
Thread.Sleep(500);
thread.Abort();
Monitor.Enter(_lock);
Console.WriteLine("Main entered");
Console.ReadKey();
}
static void Zombie()
{
Monitor.Enter(_lock);
Console.WriteLine("Zombie entered");
Thread.Sleep(1000);
Monitor.Exit(_lock);
Console.WriteLine("Zombie exited");
}
}
However when using a lock() { } block, the finally would still be executed when a ThreadAbortException is fired that way.
The following information, as it turns out, is only valid for .NET 1 and .NET 1.1:
If inside the lock() { } block an other exception occurs, and the ThreadAbortException arrives exactly when the finally block is about to be ran, the lock is not released. As you mentioned, the lock() { } block is compiled as:
finally
{
if (lockWasTaken)
Monitor.Exit(temp);
}
If another thread calls Thread.Abort() inside the generated finally block, the lock may not be released.
This isn't about Zombie threads, but the book Effective C# has a section on implementing IDisposable, (item 17), which talks about Zombie objects which I thought you may find interesting.
I recommend reading the book itself, but the gist of it is that if you have a class either implementing IDisposable, or containing a Desctructor, the only thing you should be doing in either is releasing resources. If you do other things here, then there is a chance that the object will not be garbage collected, but will also not be accessible in any way.
It gives an example similar to below:
internal class Zombie
{
private static readonly List<Zombie> _undead = new List<Zombie>();
~Zombie()
{
_undead.Add(this);
}
}
When the destructor on this object is called, a reference to itself is placed on the global list, meaning it stays alive and in memory for the life of the program, but isn't accessible. This may mean that resources (particularly unmanaged resources) may not be fully released, which can cause all sorts of potential issues.
A more complete example is below. By the time the foreach loop is reached, you have 150 objects in the Undead list each containing an image, but the image has been GC'd and you get an exception if you try to use it. In this example, I am getting an ArgumentException (Parameter is not valid) when I try and do anything with the image, whether I try to save it, or even view dimensions such as height and width:
class Program
{
static void Main(string[] args)
{
for (var i = 0; i < 150; i++)
{
CreateImage();
}
GC.Collect();
//Something to do while the GC runs
FindPrimeNumber(1000000);
foreach (var zombie in Zombie.Undead)
{
//object is still accessable, image isn't
zombie.Image.Save(#"C:\temp\x.png");
}
Console.ReadLine();
}
//Borrowed from here
//http://stackoverflow.com/a/13001749/969613
public static long FindPrimeNumber(int n)
{
int count = 0;
long a = 2;
while (count < n)
{
long b = 2;
int prime = 1;// to check if found a prime
while (b * b <= a)
{
if (a % b == 0)
{
prime = 0;
break;
}
b++;
}
if (prime > 0)
count++;
a++;
}
return (--a);
}
private static void CreateImage()
{
var zombie = new Zombie(new Bitmap(#"C:\temp\a.png"));
zombie.Image.Save(#"C:\temp\b.png");
}
}
internal class Zombie
{
public static readonly List<Zombie> Undead = new List<Zombie>();
public Zombie(Image image)
{
Image = image;
}
public Image Image { get; private set; }
~Zombie()
{
Undead.Add(this);
}
}
Again, I am aware you were asking about zombie threads in particular, but the question title is about zombies in .net, and I was reminded of this and thought others may find it interesting!
On critical systems under heavy load, writing lock-free code is better primarily because of the performance improvments. Look at stuff like LMAX and how it leverages "mechanical sympathy" for great discussions of this. Worry about zombie threads though? I think that's an edge case that's just a bug to be ironed out, and not a good enough reason not to use lock.
Sounds more like your friend is just being fancy and flaunting his knowledege of obscure exotic terminology to me! In all the time I was running the performance labs at Microsoft UK, I never came across an instance of this issue in .NET.
1.Is there a clearer definition of a "zombie thread" than what I've explained here?
I do agree that "Zombie Threads" exist, it's a term to refer to what happens with Threads that are left with resources that they don't let go of and yet don't completely die, hence the name "zombie," so your explanation of this referral is pretty right on the money!
2.Can zombie threads occur on .NET? (Why/Why not?)
Yes they can occur. It's a reference, and actually referred to by Windows as "zombie": MSDN uses the Word "Zombie" for Dead processes/threads
Happening frequently it's another story, and depends on your coding techniques and practices, as for you that like Thread Locking and have done it for a while I wouldn't even worry about that scenario happening to you.
And Yes, as #KevinPanko correctly mentioned in the comments, "Zombie Threads" do come from Unix which is why they are used in XCode-ObjectiveC and referred to as "NSZombie" and used for debugging. It behaves pretty much the same way... the only difference is an object that should've died becomes a "ZombieObject" for debugging instead of the "Zombie Thread" which might be a potential problem in your code.
I can make zombie threads easily enough.
var zombies = new List<Thread>();
while(true)
{
var th = new Thread(()=>{});
th.Start();
zombies.Add(th);
}
This leaks the thread handles (for Join()). It's just another memory leak as far as we are concerned in the managed world.
Now then, killing a thread in a way that it actually holds locks is a pain in the rear but possible. The other guy's ExitThread() does the job. As he found, the file handle got cleaned up by the gc but a lock around an object wouldn't. But why would you do that?
I have two different result from exchanging two lines of code ( done = true with Console.Write() one )
If I put done = true, firstly, the result will be:
True
Else If I put Console.WriteLine() firstly, the result will be:
False
False
Why? ( see carefully, that bool variable is static! )
using System;
using System.Threading;
class Program
{
static bool done;
static void Main(string[] args)
{
new Thread(test).Start();
test();
}
static void test()
{
if (!done)
{
done = true;
Console.WriteLine(done);
}
}
}
My bet is that the Console.WriteLine will be enough work to keep the thread busy while the second call to test() has a chance to execute.
So basically the call to WriteLine delays the setting of done long enough for the second call to test to be able to test done and find it is still set as false.
If you leave it as shown, with done = true; before the write to the console then this will be set almost instantly and thus the second call to test will find done set to true and will therefore not perform the Console.WriteLine.
Hope that all makes sense.
I just found this which contains code very much like your question. If you didn't get your question from this page already, then I would suggest having a read as it explains in much more detail the cause of this effect.
With the follow key extract:
On a single-processor computer, a thread scheduler performs
time-slicing — rapidly switching execution between each of the active
threads. Under Windows, a time-slice is typically in the
tens-of-milliseconds region — much larger than the CPU overhead in
actually switching context between one thread and another (which is
typically in the few-microseconds region).
So essentially the call to Console.WriteLine is taking long enough for the processor to decide that it is time for the main thread to have another go before your extra thread is permitted to continue (and ultimate set the done flag)
Your code isn't thread safe, and the results will be unpredictable.
You need to lock access when reading / writing to the static boolean, like so:
static bool done;
static readonly object _mylock = new object();
static void Main()
{
//Application.EnableVisualStyles();
//Application.SetCompatibleTextRenderingDefault(false);
//Application.Run(new Form1());
new Thread(test).Start();
test();
Console.ReadKey();
}
static void test()
{
lock (_mylock)
{
if (!done)
{
Console.WriteLine(done);
done = true;
}
}
}
Edit : readonly thanks #d4wn
Looks like the scheduler just cut the CPU time from one thread after it's call of Console.Writeline and then gave it to the other thread, all before done was set to true.
Are you certain that it always prints False\nFalse when you call Console.Writeline before assigning done = true;? To my understanding, this should be quite random.
Each time a shared variable is accessed by one of the sharing threads must be protected by one of the syncronization techniques explicitly. The environment (clr..) doesn't do it for us, cause in the whole possible complexity of multithreading it would be impossible. So this definetely responsible and not easy task must be done by the developer, writing multithreading code.
I guess there you can find a great deal of necessary information:
Thread Synchronization (C# Programming Guide)
Here is the code that I was trying on my workstation.
class Program
{
public static volatile bool status = true;
public static void Main()
{
Thread FirstStart = new Thread(threadrun);
FirstStart.Start();
Thread.Sleep(200);
Thread thirdstart = new Thread(threadrun2);
thirdstart.Start();
Console.ReadLine();
}
static void threadrun()
{
while (status)
{
Console.WriteLine("Waiting..");
}
}
static void threadrun2()
{
status = false;
Console.WriteLine("the bool value is now made FALSE");
}
}
As you can see I have fired three threads in Main. Then using breakpoints I tracked the threads. My initial conception was all the three threads will be fired simultaneously, but my breakpoint flow showed that the thread-execution-flow followed one after other (and so was the output format i.e. Top to bottom execution of threads). Guys why is that happening ?
Additionally I tried to run the same program without using the volatile keyword in declaration, and I found no change in program execution. I doubt the volatile keyword is of no practical live use. Am I going wrong somewhere?
Your method of thinking is flawed.
The very nature of threading related issues is that they're non-deterministic. This means that what you have observed is potentially no indicator of what may happen in the future.
This is the very nature of why multithreaded programming is "hard." It often defies ad hoc testing, or even most unit testing. The only way to do it effectively is to understand your entire software and hardware stack, and diagram every possible occurrence through use of state machines.
In summary, threaded programming is not about what you've seen happen, it's about what might possibly happen, no matter how improbable.
Ok I will try to explain a very long story as short as possible:
Number 1: Trying to inspect the behavior of threads with the debugger is as useful as repeatedly running a multithreaded program and concluding that it works fine because out of 100 tests none failed: WRONG! Threads behave in a completely nondeterministic (some would say random) way and you need different methods to make sure such a program will run correctly.
Number 2: The use of volatile will become clear once you remove it and then run your program in Debug mode and then switch to Release mode. I think you will have a surprise... What happens in Release mode is that the compiler will optimize code (this includes reordering instructions and caching of values). Now, if your two threads run on different processor cores, then the core executing the thread that is checking for the value of status will cache its value instead of repeatedly checking for it. The other thread will set it but the first one will never see the change: deadlock! volatile prevents this kind of situation from occurring.
In a sense, volatile is a guard in case the code does not actually (and most likely will not) run as you think it will in a multithreaded scenario.
The fact that your simple code doesn't behave dirrefently with volatile doesn't mean anything. Your code is too simple and has nothing to do with volatile. You need to write very computation-intensive code to create a clearly visible memory race condition.
Also, volatile keyword may be useful on other platforms than x86/x64 with other memory models. (I mean like for example Itanium.)
Joe Duffy wrote interesting information about volatile on his blog. I strongly recommend to read it.
Then using breakpoints I tracked the threads. My initial conception
was all the three threads will be fired simultaneously, but my
breakpoint flow showed that the thread-execution-flow followed one
after other (and so was the output format i.e. Top to bottom execution
of threads). Guys why is that happening?
The debugger is temporarily suspending the threads to make it easier to debug.
I doubt the volatile keyword is of no practical live use. Am I going
wrong somewhere?
The Console.WriteLine calls are very likely fixing masking the problem. They are most likely generating the necessary memory barrier for you implicitly. Here is a really simple snippet of code that demonstrates that there is, in fact, a problem when volatile is not used to declare the stop variable.
Compile the following code with the Release configuration and run it outside of the debugger.
class Program
{
static bool stop = false;
public static void Main(string[] args)
{
var t = new Thread(() =>
{
Console.WriteLine("thread begin");
bool toggle = false;
while (!stop)
{
toggle = !toggle;
}
Console.WriteLine("thread end");
});
t.Start();
Thread.Sleep(1000);
stop = true;
Console.WriteLine("stop = true");
Console.WriteLine("waiting...");
t.Join();
}
}
Consider the following piece of code:
void MyRunningThread()
{
while(counter>0) // counter is a Class member that can get modified from external //threads
{
Thread.Sleep(2000);
}
}
Suppose I start a Thread on the above function.
Now, is it a bad to have a never ending loop like this inside a Thread?
I have added Thread.Sleep() with the assumption that context switching to this function will occur at a less frequency due to the Sleep and hence reduce Resource consumption by this Thread.
Could someone verify my points.
It is far from optimal, wasting a Thread .
You should consider using a Timer or a WaitHandle or something. Without more details it's not possible to be precise.
But your current approach is not disastrous or anything, provided that that other thread uses lock or InterLocked to change the counter.
Try to use System.Threading.SpinWait instead of sleep, this article explains how to use it http://www.emadomara.com/2011/08/spinwait-and-lock-free-code.html
Could you elaborate your problem more? Why do you need to wait in another thread? Anyway, if you need to wait until some counter becomes zero you may consider to use CountdownEvent
Alright...I've given the site a fair search and have read over many posts about this topic. I found this question: Code for a simple thread pool in C# especially helpful.
However, as it always seems, what I need varies slightly.
I have looked over the MSDN example and adapted it to my needs somewhat. The example I refer to is here: http://msdn.microsoft.com/en-us/library/3dasc8as(VS.80,printer).aspx
My issue is this. I have a fairly simple set of code that loads a web page via the HttpWebRequest and WebResponse classes and reads the results via a Stream. I fire off this method in a thread as it will need to executed many times. The method itself is pretty short, but the number of times it needs to be fired (with varied data for each time) varies. It can be anywhere from 1 to 200.
Everything I've read seems to indicate the ThreadPool class being the prime candidate. Here is what things get tricky. I might need to fire off this thing say 100 times, but I can only have 3 threads at most running (for this particular task).
I've tried setting the MaxThreads on the ThreadPool via:
ThreadPool.SetMaxThreads(3, 3);
I'm not entirely convinced this approach is working. Furthermore, I don't want to clobber other web sites or programs running on the system this will be running on. So, by limiting the # of threads on the ThreadPool, can I be certain that this pertains to my code and my threads only?
The MSDN example uses the event drive approach and calls WaitHandle.WaitAll(doneEvents); which is how I'm doing this.
So the heart of my question is, how does one ensure or specify a maximum number of threads that can be run for their code, but have the code keep running more threads as the previous ones finish up until some arbitrary point? Am I tackling this the right way?
Sincerely,
Jason
Okay, I've added a semaphore approach and completely removed the ThreadPool code. It seems simple enough. I got my info from: http://www.albahari.com/threading/part2.aspx
It's this example that showed me how:
[text below here is a copy/paste from the site]
A Semaphore with a capacity of one is similar to a Mutex or lock, except that the Semaphore has no "owner" – it's thread-agnostic. Any thread can call Release on a Semaphore, while with Mutex and lock, only the thread that obtained the resource can release it.
In this following example, ten threads execute a loop with a Sleep statement in the middle. A Semaphore ensures that not more than three threads can execute that Sleep statement at once:
class SemaphoreTest
{
static Semaphore s = new Semaphore(3, 3); // Available=3; Capacity=3
static void Main()
{
for (int i = 0; i < 10; i++)
new Thread(Go).Start();
}
static void Go()
{
while (true)
{
s.WaitOne();
Thread.Sleep(100); // Only 3 threads can get here at once
s.Release();
}
}
}
Note: if you are limiting this to "3" just so you don't overwhelm the machine running your app, I'd make sure this is a problem first. The threadpool is supposed to manage this for you. On the other hand, if you don't want to overwhelm some other resource, then read on!
You can't manage the size of the threadpool (or really much of anything about it).
In this case, I'd use a semaphore to manage access to your resource. In your case, your resource is running the web scrape, or calculating some report, etc.
To do this, in your static class, create a semaphore object:
System.Threading.Semaphore S = new System.Threading.Semaphore(3, 3);
Then, in each thread, you do this:
System.Threading.Semaphore S = new System.Threading.Semaphore(3, 3);
try
{
// wait your turn (decrement)
S.WaitOne();
// do your thing
}
finally {
// release so others can go (increment)
S.Release();
}
Each thread will block on the S.WaitOne() until it is given the signal to proceed. Once S has been decremented 3 times, all threads will block until one of them increments the counter.
This solution isn't perfect.
If you want something a little cleaner, and more efficient, I'd recommend going with a BlockingQueue approach wherein you enqueue the work you want performed into a global Blocking Queue object.
Meanwhile, you have three threads (which you created--not in the threadpool), popping work out of the queue to perform. This isn't that tricky to setup and is very fast and simple.
Examples:
Best threading queue example / best practice
Best method to get objects from a BlockingQueue in a concurrent program?
It's a static class like any other, which means that anything you do with it affects every other thread in the current process. It doesn't affect other processes.
I consider this one of the larger design flaws in .NET, however. Who came up with the brilliant idea of making the thread pool static? As your example shows, we often want a thread pool dedicated to our task, without having it interfere with unrelated tasks elsewhere in the system.