I am trying to implement some kind of task queue with different task weights allowing different amount of tasks to run concurrently based on their weights.
There are two types of tasks: long task and short task.
Maximum of N short tasks can be executed together.
When the long task appears, it should start immediately if there are no other long task running or wait for it to complete otherwise.
Concurrent short task count limit should be reduced to M if there is long task running.
Short tasks running already should keep running to completion; but no new short tasks should start if the current limit is less or equal than current running short tasks amount.
As it seems, I basically need the ability to dynamically change semaphore "capacity".
It would be easy to decrease/increase the capacity by just taking/freeing (N - M) "slots" when needed, but that would cause "hanging" of the queue before (N - M) short tasks complete if there are N short tasks already running.
I could also implement some kind of "scheduler" awaking every 100 ms (for example) and checking the queue for any tasks that can start now. Disadvantage of this approach is having up to 100 ms delay between enqueueing the task and starting it.
So I am stuck with that puzzle and hope that someone will have some fresh idea about how to implement this.
Update:
Tasks are not going to create any significant CPU load.
They are HTTP requests in fact. Long requests are uploading files and short requests are common HTTP requests.
I answered a very similar question a few days ago, the solution for you is pretty much the exact same, use QueuedTaskScheduler
from "ParallelExtensionsExtras"
private static void Main(string[] args)
{
int highPriorityMaxConcurrancy = 1
QueuedTaskScheduler qts = new QueuedTaskScheduler();
var highPriortiyScheduler = qts.ActivateNewQueue(0);
var lowPriorityScheduler = qts.ActivateNewQueue(1);
BlockingCollection<HttpRequestWrapper> fileRequest= new BlockingCollection<Foo>();
BlockingCollection<HttpRequestWrapper> commonRequest= new BlockingCollection<Foo>();
List<Task> processors = new List<Task>(2);
processors.Add(Task.Factory.StartNew(() =>
{
Parallel.ForEach(fileRequest.GetConsumingPartitioner(), //.GetConsumingPartitioner() is also from ParallelExtensionExtras, it gives better performance than .GetConsumingEnumerable() with Parallel.ForEeach(
new ParallelOptions() { TaskScheduler = highPriortiyScheduler, MaxDegreeOfParallelism = highPriorityMaxConcurrancy },
ProcessWork);
}, TaskCreationOptions.LongRunning));
processors.Add(Task.Factory.StartNew(() =>
{
Parallel.ForEach(commonRequest.GetConsumingPartitioner(),
new ParallelOptions() { TaskScheduler = lowPriorityScheduler},
ProcessWork);
}, TaskCreationOptions.LongRunning));
//Add some work to do here to the fileRequest or commonRequest collections
//Lets the blocking collections know we are no-longer going to be adding new items so it will break out of the `ForEach` once it has finished the pending work.
fileRequest.CompleteAdding();
commonRequest.CompleteAdding();
//Waits for the two collections to compleatly empty before continueing
Task.WaitAll(processors.ToArray());
}
private static void ProcessWork(HttpRequestWrapper request)
{
//...
}
Related
I am using multiple thread for invoking a function as below.
At table level there is queue number. According to the queue
Number the MAX_Download_Thread will be set. So that much thread
will be created and work it continuously. When I put
MAX_Download_Thread as 4 it consuming 30% of CPU out of 8 processors.
When make it as 10 it almost consuming 70%. Just want to know whether
any better method to reduce this or this is normal.
protected override void OnStart(string[] args)
{
for (int i = 1; i <= MAX_Download_THREAD; i++)
{
int j = i;
var fileDownloadTread = new Thread(() =>
{
new Repo.FILE_DOWNLOAD().PIC_FILE_DOWNLOAD(j);
});
fileDownloadTread.Start();
}
}
The point of using multiple Threads is to achieve a higher utilization of multicore CPUs to complete a Task in a shorter amount of time. The higher usage is to be expected, nay desired.
If the work your threads are doing is I/O-Bound (e.g. writing large amounts of Data to a Disk) it might end up taking longer than with fewer threads as data can only be written to a disk in series.
Using multiple threads does use more CPU. In this case though, that's simply wasted. IO operations like file operations and downloads are carried out by the OS itself, not application code. The application threads just wait for the OS to retrieve data over the network.
To execute multiple downloads concurrently you need asynchronous operations. One way to do this is to use Parallel.ForeachAsync. The following code will download files in parallel from a list and save them locally:
HttpClient _client=new HttpClient();
async Task DownloadUrl(Uri url,DirectoryInfo folder)
{
var fileName=url.Segments.Last();
var newPath=Path.Combine(folder.FullName,fileName);
var stream=await _client.GetStreamAsync(url);
using var localStream=File.Create(newPath);
await stream.CopyToAsync(localStream);
}
async Task DonwloadConcurrently(IEnumerable<Uri> urls,DirectoryInfo folder)
{
await Parallel.ForEachAsync(urls,async (url,_)=>{
await DownloadUrl(url,folder);
});
}
This will executeDownloadUrl multiple times in parallel. The default number of concurrent tasks is equal to Environment.ProcessorCount but can be adjusted
ParallelOptions parallelOptions = new()
{
MaxDegreeOfParallelism = 3
};
await Parallel.ForEachAsync(urls, parallelOptions, async (url,_) =>{
await DownloadUrl(url,folder);
});
I have an ASP.NET 5 Web API application which contains a method that takes objects from a List<T> and makes HTTP requests to a server, 5 at a time, until all requests have completed. This is accomplished using a SemaphoreSlim, a List<Task>(), and awaiting on Task.WhenAll(), similar to the example snippet below:
public async Task<ResponseObj[]> DoStuff(List<Input> inputData)
{
const int maxDegreeOfParallelism = 5;
var tasks = new List<Task<ResponseObj>>();
using var throttler = new SemaphoreSlim(maxDegreeOfParallelism);
foreach (var input in inputData)
{
tasks.Add(ExecHttpRequestAsync(input, throttler));
}
List<ResponseObj> resposnes = await Task.WhenAll(tasks).ConfigureAwait(false);
return responses;
}
private async Task<ResponseObj> ExecHttpRequestAsync(Input input, SemaphoreSlim throttler)
{
await throttler.WaitAsync().ConfigureAwait(false);
try
{
using var request = new HttpRequestMessage(HttpMethod.Post, "https://foo.bar/api");
request.Content = new StringContent(JsonConvert.SerializeObject(input, Encoding.UTF8, "application/json");
var response = await HttpClientWrapper.SendAsync(request).ConfigureAwait(false);
var responseBody = await response.Content.ReadAsStringAsync().ConfigureAwait(false);
var responseObject = JsonConvert.DeserializeObject<ResponseObj>(responseBody);
return responseObject;
}
finally
{
throttler.Release();
}
}
This works well, however I am looking to limit the total number of Tasks that are being executed in parallel globally throughout the application, so as to allow scaling up of this application. For example, if 50 requests to my API came in at the same time, this would start at most 250 tasks running parallel. If I wanted to limit the total number of Tasks that are being executed at any given time to say 100, is it possible to accomplish this? Perhaps via a Queue<T>? Would the framework automatically prevent too many tasks from being executed? Or am I approaching this problem in the wrong way, and would I instead need to Queue the incoming requests to my application?
I'm going to assume the code is fixed, i.e., Task.Run is removed and the WaitAsync / Release are adjusted to throttle the HTTP calls instead of List<T>.Add.
I am looking to limit the total number of Tasks that are being executed in parallel globally throughout the application, so as to allow scaling up of this application.
This does not make sense to me. Limiting your tasks limits your scaling up.
For example, if 50 requests to my API came in at the same time, this would start at most 250 tasks running parallel.
Concurrently, sure, but not in parallel. It's important to note that these aren't 250 threads, and that they're not 250 CPU-bound operations waiting for free thread pool threads to run on, either. These are Promise Tasks, not Delegate Tasks, so they don't "run" on a thread at all. It's just 250 objects in memory.
If I wanted to limit the total number of Tasks that are being executed at any given time to say 100, is it possible to accomplish this?
Since (these kinds of) tasks are just in-memory objects, there should be no need to limit them, any more than you would need to limit the number of strings or List<T>s. Apply throttling where you do need it; e.g., number of HTTP calls done simultaneously per request. Or per host.
Would the framework automatically prevent too many tasks from being executed?
The framework has nothing like this built-in.
Perhaps via a Queue? Or am I approaching this problem in the wrong way, and would I instead need to Queue the incoming requests to my application?
There's already a queue of requests. It's handled by IIS (or whatever your host is). If your server gets too busy (or gets busy very suddenly), the requests will queue up without you having to do anything.
If I wanted to limit the total number of Tasks that are being executed at any given time to say 100, is it possible to accomplish this?
What you are looking for is to limit the MaximumConcurrencyLevel of what's called the Task Scheduler. You can create your own task scheduler that regulates the MaximumCongruencyLevel of the tasks it manages. I would recommend implementing a queue-like object that tracks incoming requests and currently working requests and waits for the current requests to finish before consuming more. The below information may still be relevant.
The task scheduler is in charge of how Tasks are prioritized, and in charge of tracking the tasks and ensuring that their work is completed, at least eventually.
The way it does this is actually very similar to what you mentioned, in general the way the Task Scheduler handles tasks is in a FIFO (First in first out) model very similar to how a ConcurrentQueue<T> works (at least starting in .NET 4).
Would the framework automatically prevent too many tasks from being executed?
By default the TaskScheduler that is created with most applications appears to default to a MaximumConcurrencyLevel of int.MaxValue. So theoretically yes.
The fact that there practically is no limit to the amount of tasks(at least with the default TaskScheduler) might not be that big of a deal for your case scenario.
Tasks are separated into two types, at least when it comes to how they are assigned to the available thread pools. They're separated into Local and Global queues.
Without going too far into detail, the way it works is if a task creates other tasks, those new tasks are part of the parent tasks queue (a local queue). Tasks spawned by a parent task are limited to the parent's thread pool.(Unless the task scheduler takes it upon itself to move queues around)
If a task isn't created by another task, it's a top-level task and is placed into the Global Queue. These would normally be assigned their own thread(if available) and if one isn't available it's treated in a FIFO model, as mentioned above, until it's work can be completed.
This is important because although you can limit the amount of concurrency that happens with the TaskScheduler, it may not necessarily be important - if for say you have a top-level task that's marked as long running and is in-charge of processing your incoming requests. This would be helpful since all the tasks spawned by this top-level task will be part of that task's local queue and therefor won't spam all your available threads in your thread pool.
When you have a bunch of items and you want to process them asynchronously and with limited concurrency, the SemaphoreSlim is a great tool for this job. There are two ways that it can be used. One way is to create all the tasks immediately and have each task acquire the semaphore before doing it's main work, and the other is to throttle the creation of the tasks while the source is enumerated. The first technique is eager, and so it consumes more RAM, but it's more maintainable because it is easier to understand and implement. The second technique is lazy, and it's more efficient if you have millions of items to process.
The technique that you have used in your sample code is the second (lazy) one.
Here is an example of using two SemaphoreSlims in order to impose two maximum concurrency policies, one per request and one globally. First the eager approach:
private const int maxConcurrencyGlobal = 100;
private static SemaphoreSlim globalThrottler
= new SemaphoreSlim(maxConcurrencyGlobal, maxConcurrencyGlobal);
public async Task<ResponseObj[]> DoStuffAsync(IEnumerable<Input> inputData)
{
const int maxConcurrencyPerRequest = 5;
var perRequestThrottler
= new SemaphoreSlim(maxConcurrencyPerRequest, maxConcurrencyPerRequest);
Task<ResponseObj>[] tasks = inputData.Select(async input =>
{
await perRequestThrottler.WaitAsync();
try
{
await globalThrottler.WaitAsync();
try
{
return await ExecHttpRequestAsync(input);
}
finally { globalThrottler.Release(); }
}
finally { perRequestThrottler.Release(); }
}).ToArray();
return await Task.WhenAll(tasks);
}
The Select LINQ operator provides an easy and intuitive way to project items to tasks.
And here is the lazy approach for doing exactly the same thing:
private const int maxConcurrencyGlobal = 100;
private static SemaphoreSlim globalThrottler
= new SemaphoreSlim(maxConcurrencyGlobal, maxConcurrencyGlobal);
public async Task<ResponseObj[]> DoStuffAsync(IEnumerable<Input> inputData)
{
const int maxConcurrencyPerRequest = 5;
var perRequestThrottler
= new SemaphoreSlim(maxConcurrencyPerRequest, maxConcurrencyPerRequest);
var tasks = new List<Task<ResponseObj>>();
foreach (var input in inputData)
{
await perRequestThrottler.WaitAsync();
await globalThrottler.WaitAsync();
Task<ResponseObj> task = Run(async () =>
{
try
{
return await ExecHttpRequestAsync(input);
}
finally
{
try { globalThrottler.Release(); }
finally { perRequestThrottler.Release(); }
}
});
tasks.Add(task);
}
return await Task.WhenAll(tasks);
static async Task<T> Run<T>(Func<Task<T>> action) => await action();
}
This implementation assumes that the await globalThrottler.WaitAsync() will never throw, which is a given according to the documentation. This will no longer be the case if you decide later to add support for cancellation, and you pass a CancellationToken to the method. In that case you would need one more try/finally wrapper around the task-creation logic. The first (eager) approach could be enhanced with cancellation support without such considerations. Its existing try/finally infrastructure is
already sufficient.
It is also important that the internal helper Run method is implemented with async/await. Eliding the async/await would be an easy mistake to make, because in that case any exception thrown synchronously by the ExecHttpRequestAsync method would be rethrown immediately, and it would not be encapsulated in a Task<ResponseObj>. Then the task returned by the DoStuffAsync method would fail without releasing the acquired semaphores, and also without awaiting the completion of the already started operations. That's another argument for preferring the eager approach. The lazy approach has too many gotchas to watch for.
I'm determining between using TPL Dataflow blocks or some sort of producer/consumer approach for these tests. Producing a list of tasks will be super-fast, as each task will just be a string containing a list of test parameters such as the setup parameters, the measurements required, and time between measurements. This list of tasks will simply be files loaded through the GUI (1 file per test).
When at test is started, it should start right away. The tests could be very long and very asynchronous in that an action could take seconds or tens of minutes (e.g. heating up a device), followed by a measurement that takes a few seconds (or minutes), followed by a long period of inaction (24 hours) before the test is repeated again.
I could have up to 16 tests running at the same time, but I need the flexibility to be able to cancel any one of those tests at any time. I also need to be able to ADD a new test at any time (i.e. try to picture testing of 16 devices, or the span of a month in which individual test devices are added and removed throughout the month).
(Visual C#) I tried this example code for TPL dataflow where I tell it to run 32 simple tasks all at the same time. Each task is just a 5 second delay to simulate work. It appears to be processing the tasks in parallel as the time to complete the tasks took 15 seconds. I assume all 32 tasks did not finish in 5 seconds due to scheduling and any other overhead, but I am a bit worried that some task might of been blocked.
class Program
{
// Performs several computations by using dataflow and returns the elapsed
// time required to perform the computations.
static TimeSpan TimeDataflowComputations(int messageCount)
{
// Create an ActionBlock<int> that performs some work.
var workerBlock = new ActionBlock<int>(
// Simulate work by suspending the current thread.
millisecondsTimeout => Thread.Sleep(millisecondsTimeout),
// Specify a maximum degree of parallelism.
new ExecutionDataflowBlockOptions
{
MaxDegreeOfParallelism = messageCount
});
// Compute the time that it takes for several messages to
// flow through the dataflow block.
Stopwatch stopwatch = new Stopwatch();
stopwatch.Start();
for (int i = 0; i < messageCount; i++)
{
workerBlock.Post(5000); // simulated work: a delay of 5 seconds.
}
workerBlock.Complete();
// Wait for all messages to propagate through the network.
workerBlock.Completion.Wait();
// Stop the timer and return the elapsed number of milliseconds.
stopwatch.Stop();
return stopwatch.Elapsed;
}
static void Main(string[] args)
{
int messageCount = 32;
TimeSpan elapsed;
// set processors maximum degree of parallelism. This causes
// multiple messages to be processed in parallel.
Console.WriteLine("START:\r\n");
elapsed = TimeDataflowComputations(messageCount);
Console.WriteLine("message count = {1}; " +
"elapsed time = {2}ms.", messageCount,
(int)elapsed.TotalMilliseconds);
Console.ReadLine();
}
}
The demo seems to work, but I am not sure if any of the tasks were blocked until one or more of the 5 second tasks were completed. I am also not sure how one would go about identifying each action block in order to cancel a specific one.
The reason that you don't get the expected performance is because your workload is synchronous and blocks the thread-pool threads. Do you expect to actually have synchronous (blocking) workload in your production environment? If yes, you could try boosting the ThreadPool reserve of available threads before starting the TPL Dataflow pipeline:
ThreadPool.SetMinThreads(workerThreads: 100, completionPortThreads: 100);
If your actual workload is asynchronous, then you could better simulate it with Task.Delay instead of Thread.Sleep.
var workerBlock = new ActionBlock<int>(async millisecondsTimeout =>
{
await Task.Delay(millisecondsTimeout);
}, new ExecutionDataflowBlockOptions
{
MaxDegreeOfParallelism = messageCount
});
I didn't test it, but you should get completion times at around 5 sec with both these approaches.
private async Task MainTask(CancellationToken token)
{
List<Task> tasks = new List<Task>();
do
{
var data = StaticVariables.AllData;
foreach (var dataPiece in data)
{
tasks.Add((new Task(() => DoSomething(data))));
}
Parallel.ForEach(tasks, task => task.Start());
await Task.WhenAll(tasks);
tasks.Clear();
await Task.Delay(2000);
} while (!token.IsCancellationRequested);
}
The above function is supposed to start a number of DoSomething(task) methods and run them at the same time. DoSomething has a timeout of 2 sec before it returns false. After some testing, it seems that the part between
await Task.WhenAll(tasks);
and
tasks.Clear()
is taking roughly 2 sec * number of tasks. So it would seem they do it like that:
Start task
do it or abort after 2 sec
start next task
...
How could I do it so that they all start at the same time and perform their operations simultaneously?
EDIT
Doing it like so:
await Task.WhenAll(data.Select(dataPiece => Task.Run(() => DoSomething(dataPiece)))
results in horrible performance (around 25 sec to complete the old code, 115 sec to complete this)
The issue you are seeing here is due to the fact that the thread pool maintains a minimum number of threads ready to run. If the thread pool needs to create more threads than that minimum, it introduces a deliberate 1 second delay between creating each new thread.
This is done to prevent things like "thread stampedes" from swamping the system with many simultaneous thread creations.
You can change the minimum thread limit using the ThreadPool.SetMinThreads() method. However, it is not recommended to do this, since it is subverting the expected thread pool operation and may cause other processes to slow down.
If you really must do it though, here's an example console application:
using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.Threading;
using System.Threading.Tasks;
namespace ConsoleApp3
{
class Program
{
static Stopwatch sw = Stopwatch.StartNew();
static void Main()
{
runTasks();
setMinThreadPoolThreads(30);
runTasks();
}
static void setMinThreadPoolThreads(int count)
{
Console.WriteLine("\nSetting min thread pool threads to {0}.\n", count);
int workerThreads, completionPortThreads;
ThreadPool.GetMinThreads(out workerThreads, out completionPortThreads);
ThreadPool.SetMinThreads(count, completionPortThreads);
}
static void runTasks()
{
var sw = Stopwatch.StartNew();
Console.WriteLine("\nStarting tasks.");
var task = test(20);
Console.WriteLine("Waiting for tasks to finish.");
task.Wait();
Console.WriteLine("Finished after " + sw.Elapsed);
}
static async Task test(int n)
{
var tasks = new List<Task>();
for (int i = 0; i < n; ++i)
tasks.Add(Task.Run(new Action(task)));
await Task.WhenAll(tasks);
}
static void task()
{
Console.WriteLine("Task starting at time " + sw.Elapsed);
Thread.Sleep(5000);
Console.WriteLine("Task stopping at time " + sw.Elapsed);
}
}
}
If you run it, you'll see from the output that running test() before setting the minimum thread pool size the tasks will take around 10 seconds (and you'll see the delay between the task start times increases after the first few tasks).
After setting the minimum thread pool threads to 30, the delay between new tasks starting is much shorter, and the overall time to run test() drops to around 5 seconds (on my PC - yours may be different!).
However, I just want to reiterate that setting the minimum thread pool size is not a normal thing to do, and should be approached with caution. As the Microsoft documentation says:
By default, the minimum number of threads is set to the number of processors on a system. You can use the SetMinThreads method to increase the minimum number of threads. However, unnecessarily increasing these values can cause performance problems. If too many tasks start at the same time, all of them might appear to be slow. In most cases, the thread pool will perform better with its own algorithm for allocating threads. Reducing the minimum to less than the number of processors can also hurt performance.
First of all, you should utilize Task.Run instead of creating and starting tasks in separate steps.
You can do so inside the loop or Linq style. If you use Linq, just ensure that you are not stuck with deferred execution, where the second task only starts after the first one is completed. Create a list, array or some other persistent collection of your selected tasks:
await Task.WhenAll(data.Select(dataPiece => Task.Run(() => DoSomething(dataPiece)).ToList());
The other problem is with the content of DoSomething. As long as this is a synchronous method, it will block its executing thread until it is done. For an inherently asynchronous operation (like pinging some network address), redesigning the method can prevent this thread blocking behavior.
Another option, as answered by Matthew Watson is to increase the amount of available threads, so each task can run in its own thread. This is not the best option, but if you have many tasks that have long blocking time without doing actual work, more threads will help to get the work done.
More threads will not help if the tasks are actually using the available physical resources, CPU or IO bound work.
I have a requirement to fire off web service requests to an online api and I thought that Parallel Extensions would be a good fit for my needs.
The web service in question is designed to be called repeatedly, but has a mechanism that charges you if you got over a certain number of calls per second. I obviously want to minimize my charges and so was wondering if anyone has seen a TaskScheduler that can cope with the following requirements:
Limit the number of tasks scheduled per timespan. I guess if the number of requests exceeded this limit then it would need to throw away the task or possibly block? (to stop a back log of tasks)
Detect if the same request is already in the scheduler to be executed but hasn't been yet and if so not queue the second task but return the first instead.
Do people feel that these are the sorts of responsibilities a task scheduler should be dealing with or am i barking up the wrong tree? If you have alternatives I am open to suggestions.
I agree with others that TPL Dataflow sounds like a good solution for this.
To limit the processing, you could create a TransformBlock that doesn't actually transform the data in any way, it just delays it if it arrived too soon after the previous data:
static IPropagatorBlock<T, T> CreateDelayBlock<T>(TimeSpan delay)
{
DateTime lastItem = DateTime.MinValue;
return new TransformBlock<T, T>(
async x =>
{
var waitTime = lastItem + delay - DateTime.UtcNow;
if (waitTime > TimeSpan.Zero)
await Task.Delay(waitTime);
lastItem = DateTime.UtcNow;
return x;
},
new ExecutionDataflowBlockOptions { BoundedCapacity = 1 });
}
Then create a method that produces the data (for example integers starting from 0):
static async Task Producer(ITargetBlock<int> target)
{
int i = 0;
while (await target.SendAsync(i))
i++;
}
It's written asynchronously, so that if the target block isn't able to process the items right now, it will wait.
Then write a consumer method:
static void Consumer(int i)
{
Console.WriteLine(i);
}
And finally, link it all together and start it up:
var delayBlock = CreateDelayBlock<int>(TimeSpan.FromMilliseconds(500));
var consumerBlock = new ActionBlock<int>(
(Action<int>)Consumer,
new ExecutionDataflowBlockOptions { MaxDegreeOfParallelism = DataflowBlockOptions.Unbounded });
delayBlock.LinkTo(consumerBlock, new DataflowLinkOptions { PropagateCompletion = true });
Task.WaitAll(Producer(delayBlock), consumerBlock.Completion);
Here, delayBlock will accept at most one item every 500 ms and the Consumer() method can run multiple times in parallel. To finish processing, call delayBlock.Complete().
If you want to add some caching per your #2, you could create another TransformBlock do the work there and link it to the other blocks.
Honestly I would work at a higher level of abstraction and use the TPL Dataflow API for this. The only catch is you would need to write a custom block that will throttle the requests at the rate at which you need because, by default, blocks are "greedy" and will just process as fast as possible. The implementation would be something like this:
Start with a BufferBlock<T> which is the logical block that you would post to.
Link the BufferBlock<T> to a custom block which has the knowledge of requests/sec and throttling logic.
Link the custom block from 2 to to your ActionBlock<T>.
I don't have the time to write the custom block for #2 right this second, but I will check back later and try to fill in an implementation for you if you haven't already figured it out.
I haven't used RX much, but AFAICT the Observable.Window method would work fine for this.
http://msdn.microsoft.com/en-us/library/system.reactive.linq.observable.window(VS.103).aspx
It would seem to be a better fit than Throttle which seems to throw elements away, which I'm guessing is not what you want
If you need to throttle by time, you should check out Quartz.net. It can facilitate consistent polling. If you care about all requests, you should consider using some sort of queueing mechanism. MSMQ is probably the right solution but there are many specific implementations if you want to go bigger and use an ESB like NServiceBus or RabbitMQ.
Update:
In that case, TPL Dataflow is your preferred solution if you can leverage the CTP. A throttled BufferBlock is the solution.
This example comes from the documentation provided by Microsoft:
// Hand-off through a bounded BufferBlock<T>
private static BufferBlock<int> m_buffer = new BufferBlock<int>(
new DataflowBlockOptions { BoundedCapacity = 10 });
// Producer
private static async void Producer()
{
while(true)
{
await m_buffer.SendAsync(Produce());
}
}
// Consumer
private static async Task Consumer()
{
while(true)
{
Process(await m_buffer.ReceiveAsync());
}
}
// Start the Producer and Consumer
private static async Task Run()
{
await Task.WhenAll(Producer(), Consumer());
}
Update:
Check out RX's Observable.Throttle.