Develop Reference

Large Number of Timers

I need to write a component that receives an event (the event has a unique ID). Each event requires me to send out a request. The event specifies a timeout period, which to wait for a response from the request.
If the response comes before the timer fires, great, I cancel the timer.
If the timer fires first, then the request timed out, and I want to move on.
This timeout period is specified in the event, so it's not constant.
The expected timeout period is in the range of 30 seconds to 5 minutes.
I can see two ways of implementing this.
Create a timer for each event and put it into a dictionary linking the event to the timer.
Create an ordered list containing the DateTime of the timeout, and a new thread looping every 100ms to check if something timed out.
Option 1 would seem like the easiest solution, but I'm afraid that creating so many timers might not be a good idea because timers might be too expensive. Are there any pitfalls when creating a large number of timers? I suspect that in the background, the timer implementation might actually be an efficient implementation of Option 2. If this option is a good idea, which timer should I use? System.Timers.Timer or System.Threading.Timer.
Option 2 seems like more work, and may not be an efficient solution compared to Option 1.
Update
The maximum number of timers I expect is in the range of 10000, but more likely in the range of 100. Also, the normal case would be the timer being canceled before firing.
Update 2
I ran a test using 10K instances of System.Threading.Timer and System.Timers.Timer, keeping an eye on thread count and memory. System.Threading.Timer seems to be "lighter" compared to System.Timers.Timer judging by memory usage, and there was no creation of excessive number of threads for both timers (ie - thread pooling working properly). So I decided to go ahead and use System.Threading.Timer.

I do this a lot in embedded systems (pure c), where I can't burn a lot of resources (e.g. 4k of RAM is the system memory). This is one approach that has been used (successfully):
Create a single system timer (interrupt) that goes off on a periodic basis (e.g. every 10 ms).
A "timer" is an entry in a dynamic list that indicates how many "ticks" are left till the timer goes off.
Each time the system timer goes off, iterate the list and decrement each of the "timers". Each one that is zero is "fired". Remove it from the list and do whatever the timer was supposed to do.
What happens when the timer goes off depends on the application. It may be a state machine gets run. It may be a function gets called. It may be an enumeration telling the execution code what to do with the parameter sent it the "Create Timer" call. The information in the timer structure is whatever is necessary in the context of the design. The "tick count" is the secret sauce.
We also have created this returning an "ID" for the timer (usually the address of the timer structure, which is drawn from a pool) so it can be cancelled or status on it can be obtained.
Convenience functions convert "seconds" to "ticks" so the API of creating the timers is always in terms of "seconds" or "milliseconds".
You set the "tick" interval to a reasonable value for granularity tradeoff.
I have done other implementations of this in C++, C#, objective-C, with little change in the general approach. It is a very general timer subsystem design/architecture. You just need something to create the fundamental "tick".
I even did it once with a tight "main" loop and a stopwatch from the high-precision internal timer to create my own "simulated" tick when I did not have a timer. I do not recommend this approach; I was simulating hardware in a straight console app and did not have access to the system timers, so it was a bit of an extreme case.
Iterating over a list of a hundreds of timers 10 times a second is not that big a deal on a modern processor. There are ways you can overcome this as well by inserting the items with "delta seconds" and putting them into the list in sorted order. This way you only have to check the ones at the front of the list. This gets you past scaling issues, at least in terms of iterating the list.
Was this helpful?

You should do it the simplest way possible. If you are concerned about performance, you should run your application through a profiler and determine the bottlenecks. You might be very surprised to find out it was some code which you least expected, and you had optimized your code for no reason. I always write the simplest code possible as this is the easiest. See PrematureOptimization
I don't see why there would be any pitfalls with a large number of timers. Are we talking about a dozen, or 100, or 10,000? If it's very high you could have issues. You could write a quick test to verify this.
As for which of those Timer classes to use: I don't want to steal anyone elses answer who probably did much more research: check out this answer to that question`

The first option simply isn't going to scale, you are going to need to do something else if you have a lot of concurrent timeouts. (If you don't know if how many you have is enough to be a problem though, feel free to try using timers to see if you actually have a problem.)
That said, your second option would need a bit of tweaking. Rather than having a tight loop in a new thread, just create a single timer and set its interval (each time it fires) to be the timespan between the current time and the "next" timeout time.

Let me propose a different architecture: for each event, just create a new Task and send the request and wait1 for the response there.
The ~1000 tasks should scale just fine, as shown in this early demo. I suspect ~10000 tasks would still scale, but I haven't tested that myself.
1 Consider implementing the wait by attaching a continuation on Task.Delay (instead of just Thread.Sleep), to avoid under-subscription.

I think Task.Delay is a really good option. Here is the test code for measuring how many concurrent tasks can be executed in different delay times. This code is also calculating error statistics for waiting time accuracy.
static async Task Wait(int delay, double[] errors, int index)
{
var sw = new Stopwatch();
sw.Start();
await Task.Delay(delay);
sw.Stop();
errors[index] = Math.Abs(sw.ElapsedMilliseconds - delay);
}
static void Main(string[] args)
{
var trial = 100000;
var minDelay = 1000;
var maxDelay = 5000;
var errors = new double[trial];
var tasks = new Task[trial];
var rand = new Random();
var sw = new Stopwatch();
sw.Start();
for (int i = 0; i < trial; i++)
{
var delay = rand.Next(minDelay, maxDelay);
tasks[i] = Wait(delay, errors, i);
}
sw.Stop();
Console.WriteLine($"{trial} tasks started in {sw.ElapsedMilliseconds} milliseconds.");
Task.WaitAll(tasks);
Console.WriteLine($"Avg Error: {errors.Average()}");
Console.WriteLine($"Min Error: {errors.Min()}");
Console.WriteLine($"Max Error: {errors.Max()}");
Console.ReadLine();
}
You may change the parameters to see different results. Here are several results in milliseconds:
100000 tasks started in 9353 milliseconds.
Avg Error: 9.10898
Min Error: 0
Max Error: 110

WithDegreeOfParallelism(N>CPU count)

System.Threading.ThreadPool.SetMaxThreads(50, 50);
File.ReadLines().AsParallel().WithDegreeOfParallelism(100).ForAll((s)->{
/*
some code which is waiting external API call
and do not utilize CPU
*/
});
I have never got threads count more than CPU count in my system.
Can I use PLINQ and get more than one thread per CPU?

If you're calling external web API, you might be hitting the limit of concurrent simultaneous connections, which is set to 2. In the begining of your application do the following:
System.Net.ServicePointManager.DefaultConnectionLimit = 4096;
System.Net.ServicePointManager.Expect100Continue = false;
Try if that helps. If not, there might be some other bottleneck within the routine you're trying to parallelize.
Also, just like other responders said, ThreadPool decides how many threads to spin up based on load. In my experience with TPL I've seen that thread cound increases by time: longer the app runs, and heavier load gets, more threads are spun up.

PLINQ uses a hill-climbing algorithm to determine the optimum size of the thread pool which is used by the TPL. I think that if you put a lot of I/O in your tasks, seeing more threads than the cpu count is likeable.
That said, I've never seen more threads than the cpu count :) . But maybe I never had the right situation.

I tested this with the following code:
var lines = Enumerable.Range(0, 200).ToArray();
int currentThreads = 0;
int maxThreads = 0;
object l = new object();
lines.AsParallel().WithDegreeOfParallelism(100).ForAll(
s =>
{
lock (l)
{
currentThreads++;
if (currentThreads > maxThreads)
{
maxThreads = currentThreads;
Console.WriteLine(maxThreads);
}
}
Thread.Sleep(3000);
lock (l)
{
currentThreads--;
}
});
Console.WriteLine();
Console.WriteLine(maxThreads);
Basically, it records the current number of concurrently executing iterations and then saves the maximum encountered value.
The results vary quite a bit, between 15 and 25, but it's always much more than the number of CPUs my computer has (4). Increasing the sleep time increases the maximum number of concurrent threads. So it looks like the limiting factor here is the ThreadPool: it will create new threads slowly, especially when jobs are being completed relatively quickly.
If you want to increase the number of threads used, you would need to use SetMinThreads() (not SetMaxThreads()). If I set the minimum to 50, the number of threads actually used is around 60.
But having dozens of threads that do nothing but wait is quite inefficient, especially when it comes to memory consumption. You should consider using asynchronous methods instead.

PLINQ does not fit in this case.
I have found next article useful for me.
http://msdn.microsoft.com/en-us/library/hh228609(v=vs.110).aspx

Short answer: nope.
The amount of threading is simply up to the .Net Framework runtime. There is no developer control for controlling the number of threads for TPL (Task Parallel Library) usage.
EDIT
Thanks to some other feedback: it is actually possible--but not recommended--to manually control the number of threads in the ThreadPool, which PLINQ and TPL use.
It's my opinion that any parallelization problem needs to be carefully thought out, and carefully constructed and tested. There's a lot of subtlety in this.

Reactive Extensions Test Scheduler Simulating Time elapse

I am working with RX scheduler classes using the .Schedule(DateTimeOffset, Action>) stuff. Basically I've a scheduled action that can schedule itself again.
Code:
public SomeObject(IScheduler sch, Action variableAmountofTime)
{
this.sch = sch;
sch.Schedule(GetNextTime(), (Action<DateTimeOffset> runAgain =>
{
//Something that takes an unknown variable amount of time.
variableAmountofTime();
runAgain(GetNextTime());
});
}
public DateTimeOffset GetNextTime()
{
//Return some time offset based on scheduler's
//current time which is irregular based on other inputs that i have left out.
return this.sch.now.AddMinutes(1);
}
My Question is concerning simulating the amount of time variableAmountofTime might take and testing that my code behaves as expected and only triggers calling it as expected.
I have tried advancing the test scheduler's time inside the delegate but that does not work. Example of code that I wrote that doesnt work. Assume GetNextTime() is just scheduleing one minute out.
[Test]
public void TestCallsAppropriateNumberOfTimes()
{
var sch = new TestScheduler();
var timesCalled = 0;
var variableAmountOfTime = () =>
{
sch.AdvanceBy(TimeSpan.FromMinutes(3).Ticks);
timescalled++;
};
var someObject = new SomeObject(sch, variableAmountOfTime);
sch.AdvanceTo(TimeSpan.FromMinutes(3).Ticks);
Assert.That(timescalled, Is.EqualTo(1));
}
Since I am wanting to go 3 minutes into the future but the execution takes 3 minutes, I want to see this only trigger 1 time..instead it triggers 3 times.
How can I simulate something taking time during execution using the test scheduler.

Good question. Unfortunately, this is currently not supported in Rx v1.x and Rx v2.0 Beta (but read on). Let me explain the complication of nested Advance* calls to you.
Basically, Advance* implies starting the scheduler to run work till the point specified. This involves running the work in order on a single logical thread that represents the flow of time in the virtual scheduler. Allowing nested Advance* calls raises a few questions.
First of all, should a nested Advance* call cause a nested worker loop to be run? If that were the case, we're no longer mimicking a single logical thread of execution as the current work item would be interrupted in favor of running the inner loop. In fact, Advance* would lead to an implicit yield where the rest of the work (that was due now) after the Advance* call would not be allowed to run until all nested work has been processed. This leads to the situation where future work cannot depend on (or wait for) past work to finish its execution. One way out is to introduce real physical concurrency, which defeats various design points of the virtual time and historical schedulers to begin with.
Alternatively, should a nested Advance* call somehow communicate to the top-most worker loop dispatching call (Advance* or Start) it may need to extend its due time because a nested invocation has asked to advance to a point beyond the original due time. Now all sorts of things are getting weird though. The clock doesn't reflect the changes after returning from Advance* and the top-most call no longer finishes at a predictable time.
For Rx v2.0 RC (coming next month), we took a look at this scenario and decided Advance* is not the right thing to emulate "time slippage" because it'd need an overloaded meaning depending on the context where it's invoked from. Instead, we're introducing a Sleep method that can be used to slip time forward from any context, without the side-effect of running work. Think of it as a way to set the Clock property but with safeguarding against going back in time. The name also reflects the intent clearly.
In addition to the above, to reduce the surprise factor of nested Advance* calls having no effect, we made it detect this situation and throw an InvalidOperationException in a nested context. Sleep, on the other hand, can be called from anywhere.
One final note. It turns out we needed exactly the same feature for work we're doing in Rx v2.0 RC with regards to our treatment of time. Several tests required a deterministic way to emulate slippage of time due to the execution of user code that can take arbitrarily long (think of the OnNext handler to e.g. Observable.Interval).
Hope this helps... Stay tuned for our Rx v2.0 RC release in the next few weeks!
-Bart (Rx team)

Is this a good way of checking the amount of time it took to run some code in C#?

What I mean is...
get the time, run the code, get the time, compare the time and get the seconds out:
am I doing this right?
DateTime timestamp = DateTime.Now;
//...do the code...
DateTime endstamp = DateTime.Now;
string results = ((endstamp.ticks - timestamp.ticks)/10000000).ToString();

You should use Stopwatch for this, for example:
var sw = Stopwatch.StartNew();
//...do the code...
sw.Stop();
var result = sw.ElapsedTicks; //ticks it took
//or less accurate/for bigger tasks, sw.ElapsedMilliseconds
Edited to include #Brian's improvement from comments.

As many people have noted, the high-precision Stopwatch class is designed for answering the question "how long did this take?" whereas the DateTime class is designed for answering the question "when does Doctor Who start?" Use the right tool for the job.
However, there is more to the problem of correctly measuring elapsed time than simply getting the timer right. You've also got to make sure that you're measuring what you really want to measure. For example, consider:
// start the timer
M();
// stop the timer
// start another timer
M();
// stop the timer
Is there going to be a significant difference between the timings of the two calls? Possibly yes. Remember, the first time a method is called the jitter has to compile it from IL into machine code. That takes time. The first call to a method can be in some cases many times longer than every subsequent call put together.
So which measurement is "right"? The first measurement? The second? An average of them? It depends on what you are trying to optimize for. If you are optimizing for fast startup then you care very very much about the jit time. If you are optimizing for number of identical pages served per second on a warmed-up server then you don't care at all about jit time and should be designing your tests to not measure it. Make sure you are measuring the thing you are actually optimizing for.

No. Use the System.Diagnostics.Stopwatch class instead. DateTime.Now doesn't have the level of precision that you desire (although the DateTime struct is plenty precise, in and of itself).
Stopwatch watch = new Stopwatch();
watch.Start();
// do stuff
watch.Stop();
long ticks = watch.ElapsedTicks;

The suggestions given in previous answers will work for simple measurements. If you need something more advanced, you might want to use a profiler (there are commercial ones and free ones such as equatec).

Obviously arbitary processes executing on your machine will likely distort the result you get.
A Stopwatch is a good solution, as stated in MSDN:
The Stopwatch measures elapsed time by counting timer ticks in the underlying timer mechanism. If the installed hardware and operating system support a high-resolution performance counter, then the Stopwatch class uses that counter to measure elapsed time. Otherwise, the Stopwatch class uses the system timer to measure elapsed time. Use the Frequency and IsHighResolution fields to determine the precision and resolution of the Stopwatch timing implementation.
The Stopwatch class assists the manipulation of timing-related performance counters within managed code. Specifically, the Frequency field and GetTimestamp method can be used in place of the unmanaged Win32 APIs QueryPerformanceFrequency and QueryPerformanceCounter.

A better idea is to use the System.Diagnostics.StopWatch class http://msdn.microsoft.com/en-us/library/system.diagnostics.stopwatch.aspx

It's probably ok, but note that there are several means for "the amount of time it took to run some code". What you've got there is wall clock time: the amount of time that passed in the world between the first and second calls to DateTime.Now (approximately). That will include time spent waiting on locks or disk access, time spent running other threads that don't contain your code, etc.

That will inflate your times just slightly, but in general it works. You can also follow aspect-oriented principles and adopt something like log4net to get this funcationality without having to code it up everywhere.
Here is an article on it.

C# thread pool limiting threads

Alright...I've given the site a fair search and have read over many posts about this topic. I found this question: Code for a simple thread pool in C# especially helpful.
However, as it always seems, what I need varies slightly.
I have looked over the MSDN example and adapted it to my needs somewhat. The example I refer to is here: http://msdn.microsoft.com/en-us/library/3dasc8as(VS.80,printer).aspx
My issue is this. I have a fairly simple set of code that loads a web page via the HttpWebRequest and WebResponse classes and reads the results via a Stream. I fire off this method in a thread as it will need to executed many times. The method itself is pretty short, but the number of times it needs to be fired (with varied data for each time) varies. It can be anywhere from 1 to 200.
Everything I've read seems to indicate the ThreadPool class being the prime candidate. Here is what things get tricky. I might need to fire off this thing say 100 times, but I can only have 3 threads at most running (for this particular task).
I've tried setting the MaxThreads on the ThreadPool via:
ThreadPool.SetMaxThreads(3, 3);
I'm not entirely convinced this approach is working. Furthermore, I don't want to clobber other web sites or programs running on the system this will be running on. So, by limiting the # of threads on the ThreadPool, can I be certain that this pertains to my code and my threads only?
The MSDN example uses the event drive approach and calls WaitHandle.WaitAll(doneEvents); which is how I'm doing this.
So the heart of my question is, how does one ensure or specify a maximum number of threads that can be run for their code, but have the code keep running more threads as the previous ones finish up until some arbitrary point? Am I tackling this the right way?
Sincerely,
Jason
Okay, I've added a semaphore approach and completely removed the ThreadPool code. It seems simple enough. I got my info from: http://www.albahari.com/threading/part2.aspx
It's this example that showed me how:
[text below here is a copy/paste from the site]
A Semaphore with a capacity of one is similar to a Mutex or lock, except that the Semaphore has no "owner" – it's thread-agnostic. Any thread can call Release on a Semaphore, while with Mutex and lock, only the thread that obtained the resource can release it.
In this following example, ten threads execute a loop with a Sleep statement in the middle. A Semaphore ensures that not more than three threads can execute that Sleep statement at once:
class SemaphoreTest
{
static Semaphore s = new Semaphore(3, 3); // Available=3; Capacity=3
static void Main()
{
for (int i = 0; i < 10; i++)
new Thread(Go).Start();
}
static void Go()
{
while (true)
{
s.WaitOne();
Thread.Sleep(100); // Only 3 threads can get here at once
s.Release();
}
}
}

Note: if you are limiting this to "3" just so you don't overwhelm the machine running your app, I'd make sure this is a problem first. The threadpool is supposed to manage this for you. On the other hand, if you don't want to overwhelm some other resource, then read on!
You can't manage the size of the threadpool (or really much of anything about it).
In this case, I'd use a semaphore to manage access to your resource. In your case, your resource is running the web scrape, or calculating some report, etc.
To do this, in your static class, create a semaphore object:
System.Threading.Semaphore S = new System.Threading.Semaphore(3, 3);
Then, in each thread, you do this:
System.Threading.Semaphore S = new System.Threading.Semaphore(3, 3);
try
{
// wait your turn (decrement)
S.WaitOne();
// do your thing
}
finally {
// release so others can go (increment)
S.Release();
}
Each thread will block on the S.WaitOne() until it is given the signal to proceed. Once S has been decremented 3 times, all threads will block until one of them increments the counter.
This solution isn't perfect.
If you want something a little cleaner, and more efficient, I'd recommend going with a BlockingQueue approach wherein you enqueue the work you want performed into a global Blocking Queue object.
Meanwhile, you have three threads (which you created--not in the threadpool), popping work out of the queue to perform. This isn't that tricky to setup and is very fast and simple.
Examples:
Best threading queue example / best practice
Best method to get objects from a BlockingQueue in a concurrent program?

It's a static class like any other, which means that anything you do with it affects every other thread in the current process. It doesn't affect other processes.
I consider this one of the larger design flaws in .NET, however. Who came up with the brilliant idea of making the thread pool static? As your example shows, we often want a thread pool dedicated to our task, without having it interfere with unrelated tasks elsewhere in the system.