Develop Reference

Asynchronous operation of the BatchBlock in BackgroundService

I need to receive messages from a queue broker and, after some processing, store them in a database. In order to save records in blocks, i use a BatchBlock.
All this works through the BackgroundService
Code:
public class Worker : BackgroundService
{
private readonly IServiceProvider _serviceProvider;
private readonly BatchBlock<Comment> _batchBlock;
private readonly ActionBlock<Comment[]> _importer;
private readonly ConnectionFactory _factory;
private readonly IModel _channel;
private const string _queueName = "Comment";
private static int count;
public Worker(IConfiguration configuration, IServiceProvider serviceProvider)
{
_serviceProvider = serviceProvider;
_factory = new ConnectionFactory
{ Uri = new Uri(configuration.GetSection("RabbitMqConnection").Value), DispatchConsumersAsync = true };
var connection = _factory.CreateConnection();
_channel = connection.CreateModel();
_channel.QueueDeclare(queue: _queueName, durable: true, exclusive: false, autoDelete: false, arguments: null);
_batchBlock = new BatchBlock<Comment>(50);
_importer = new ActionBlock<Comment[]>(SaveCommentsToDb);
_batchBlock.LinkTo(_importer, new DataflowLinkOptions { PropagateCompletion = true });
}
protected override Task ExecuteAsync(CancellationToken stoppingToken)
{
stoppingToken.ThrowIfCancellationRequested();
var consumer = new AsyncEventingBasicConsumer(_channel);
consumer.Received += async (_, ea) =>
await StartCommentHandlerGroup(ea);
_channel.BasicConsume(_queueName, false, consumer);
return Task.CompletedTask;
}
private async Task StartCommentHandlerGroup(BasicDeliverEventArgs message)
{
try
{
var content = Encoding.UTF8.GetString(message.Body.ToArray());
var comment = JsonConvert.DeserializeObject<Comment>(content);
await _batchBlock.SendAsync(comment);
}
catch (Exception e)
{
Log.Error($"!!!Handler error!!! {e.Message}{Environment.NewLine} !!!Message!!!{message.Body}");
throw;
}
}
private async Task SaveCommentsToDb(Comment[] comments)
{
using var scope = _serviceProvider.CreateScope();
var dbContext = scope.ServiceProvider.GetRequiredService<Context>();
await dbContext.AddMessagesToDb(comments);
}
}
It works very weird.....
If i set the BatchSize parameter to 10, about 200 records get into the database, if i set it to 2, about 300 hits. I set the BatchSize to 50 and higher and nothing gets into the database.
In this case, the queue is consumed correctly. SendAsync works correctly in the BatchBlock, the Task returns true.
After a certain part of the processing is done, i get the InputCount parameters in the _importer - 0 and the status is IsCompleted - true.
No matter how many new messages from the broker i receive in the future, this does not change the situation in any way and the ActionBlock is not called.
await dbContext.AddMessagesToDb(comments) contains:
foreach (var model in models)
await DbContext.AddAsync(model);
await DbContext.SaveChangesAsync();
I tried adding the BoundedCapacity parameter, but it didn't work.
batchBlock = new BatchBlock<Comment>(10, new GroupingDataflowBlockOptions(){BoundedCapacity = 50});

Message is published from API #1, how do I create a listener in another API?

I've two APIs.
When one of the endpoints are called in API #1, message is sent to queue in Azure Service Bus.
API #2 should listen and make some changes in DB after this message appears in queue.
Message is sent to queue successfully.
Listener part doesn't work because the listener method is never called (and I do not understand how to call it).
Listener in API #2 :
public class MessageConsumer : IMessageConsumer
{
const string connectionString = "stringTakenFromAzure";
private static IQueueClient queueClient;
private CartingDbContext context;
public MessageConsumer(CartingDbContext context)
{
this.context = context;
}
public async Task Consume()
{
queueClient = new QueueClient(connectionString, "cartqueue");
var options = new MessageHandlerOptions(ExceptionReceivedHandler)
{
MaxConcurrentCalls = 1,
AutoComplete = false
};
queueClient.RegisterMessageHandler(ProcessMessageAsync, options);
await queueClient.CloseAsync();
}
private async Task ProcessMessageAsync(Microsoft.Azure.ServiceBus.Message message, CancellationToken cancellationToken)
{
var jsonBody = Encoding.UTF8.GetString(message.Body);
var categoryItem = JsonSerializer.Deserialize<CategoryItem>(jsonBody);
//update item in DB.
var categoryItemInDb = context.CategoryItems.Where(x => x.Id == categoryItem.Id).FirstOrDefault();
if (categoryItemInDb == null)
{
context.CategoryItems.Add(categoryItem);
}
else
{
context.CategoryItems.Update(categoryItem);
}
context.SaveChanges();
await queueClient.CompleteAsync(message.SystemProperties.LockToken);
}
private static Task ExceptionReceivedHandler(ExceptionReceivedEventArgs args)
{
return Task.CompletedTask;
}
}
Program.cs
builder.Services.AddTransient<IMessageConsumer, MessageConsumer>();

Windows services to consume Azure service bus message from queue

I have created a simple windows services to consume messages from Azure service bus queue. I used the Topshelf to create windows service. Code snipped below following example from here: https://learn.microsoft.com/en-us/azure/service-bus-messaging/service-bus-dotnet-get-started-with-queues
var hf = HostFactory.New(x =>
{
x.Service<ServiceBusHelper>(s =>
{
s.ConstructUsing(serviceProvider.GetService<ServiceBusHelper>);
s.WhenStarted(async service => await service.ReceiveMessagesAsync());
s.WhenStopped(async service => await service.Stop());
});
x.RunAsNetworkService()
.StartAutomatically()
.EnableServiceRecovery(rc => rc.RestartService(1));
x.SetServiceName("MyWindowsService");
x.SetDisplayName("MyWindowsService");
x.SetDescription("MyWindowsService");
});
hf.Run();
ServiceBusHelper class:
public async Task ReceiveMessagesAsync()
{
var connectionString = _configuration.GetValue<string>("ServiceBusConnectionString");
var queueName = _configuration.GetValue<string>("ServiceBusQueueName");
await using (ServiceBusClient client = new ServiceBusClient(connectionString))
{
ServiceBusProcessor processor = client.CreateProcessor(queueName, new ServiceBusProcessorOptions());
processor.ProcessMessageAsync += MessageHandler;
processor.ProcessErrorAsync += ErrorHandler;
await processor.StartProcessingAsync();
System.Threading.Thread.Sleep(1000);//Wait for a minute before stop processing
await processor.StopProcessingAsync();
}
}
public async Task MessageHandler(ProcessMessageEventArgs args)
{
string body = args.Message.Body.ToString();
var messageBytes = Encoding.ASCII.GetBytes(body);
ProcessMessage(messageBytes);
await args.CompleteMessageAsync(args.Message);
}
public Task ErrorHandler(ProcessErrorEventArgs args)
{
return Task.CompletedTask;
}
public Task Stop()
{
return Task.CompletedTask;
}
Window service gets installed successfully and the status show running. However, it would not automatically consume the message from the service bus. If I manually stop and start the service it will pick up the message from the queue. Not sure what am I missing with this implementation. Any suggestions appreciated.

.NetCore 3.1 introduced a new extension to work along side Microsoft.AspNetCore.Hosting
Adding NuGet package Microsoft.Extensions.Hosting.WindowsServices
you can add
.UseWindowsService(). this will allow you run this as a windows service or Console app.
public static IHostBuilder CreateHostBuilder(string[] args) =>
Host.CreateDefaultBuilder(args)
.UseWindowsService()
.ConfigureAppConfiguration((context, config) =>
{
// configure the app here.
})
.ConfigureServices((hostContext, services) =>
{
services.AddHostedService<QueueWorker>();
}).UseSerilog();
}
you can then create a background worker to start and stop processing the servicebus queue. Here is my implementaion:
public class QueueWorker : BackgroundService, IDisposable
{
protected ILogger<QueueWorker> _logger;
protected IQueueMessageReceiver _queueProcessor;
public QueueWorker()
{
}
public QueueWorker(ILogger<QueueWorker> logger, IQueueMessageReceiver queueMessageReceiver)
{
_logger = logger;
_queueProcessor = queueMessageReceiver;
}
protected override async Task ExecuteAsync(CancellationToken stoppingToken)
{
await Task.CompletedTask.ConfigureAwait(false);
}
public override Task StartAsync(CancellationToken cancellationToken)
{
_logger.LogInformation("Service Starting");
var task = _queueProcessor.StartProcessor(cancellationToken);
task.Wait();
if (task.IsFaulted)
{
throw new Exception("Unable to start Processor");
}
return base.StartAsync(cancellationToken);
}
public override async Task StopAsync(CancellationToken cancellationToken)
{
_logger.LogInformation("Stopping Service");
await _queueProcessor.StopProcessor().ConfigureAwait(false);
await base.StopAsync(cancellationToken).ConfigureAwait(false);
}
public override void Dispose()
{
_logger.LogInformation("Disposing Service");
var loopCount = 0;
while (_queueProcessor != null && !_queueProcessor.IsClosedOrClosing() && loopCount < 5)
{
var task = Task.Delay(600);
task.Wait();
loopCount++;
}
base.Dispose();
GC.SuppressFinalize(this);
}
And The actual processor:
public class QueueMessageReceiver : IQueueMessageReceiver
{
private readonly ServiceBusClient _queueClient;
private ServiceBusProcessor _processor;
private readonly ReceiverConfiguration _configuration;
private readonly ILogger _logger;
private readonly ILoggerFactory _loggerFactory;
private Dictionary<string, string> _executionMatrix;
private readonly IServiceProvider _provider;
private CancellationToken _cancellationToken;
public QueueMessageReceiver(ReceiverConfiguration configuration, ILogger<QueueMessageReceiver> logger, IExecutionMatrix executionMatrix, ILoggerFactory loggerFactory, IServiceProvider serviceProvider)
{
if (configuration == null) throw new ArgumentException($"Configuration is missing from the expected ");
_configuration = configuration;
_logger = logger;
_loggerFactory = loggerFactory;
_executionMatrix = executionMatrix.GetExecutionMatrix();
_provider = serviceProvider;
_queueClient = new ServiceBusClient(_configuration.ConnectionString);
if (string.IsNullOrWhiteSpace(configuration.ConnectionString)) throw new ArgumentException($"ServiceBusConnectionString Object missing from the expected configuration under ConnectionStrings ");
if (configuration.QueueName == null) throw new ArgumentException($"Queue Name value missing from the expected configuration");
}
public async Task StartProcessor(CancellationToken cancellationToken)
{
if (!IsClosedOrClosing())
{
throw new FatalSystemException("ServiceBusProcessor is already running. ");
}
_cancellationToken = cancellationToken;
var options = new ServiceBusProcessorOptions
{
AutoCompleteMessages = _configuration.AutoComplete,
MaxConcurrentCalls = _configuration.MaxConcurrentCalls,
MaxAutoLockRenewalDuration = _configuration.MaxAutoRenewDuration
};
_processor = _queueClient.CreateProcessor(_configuration.QueueName, options);
_processor.ProcessMessageAsync += ProcessMessagesAsync;
_processor.ProcessErrorAsync += ProcessErrorAsync;
await _processor.StartProcessingAsync().ConfigureAwait(false);
}
public async Task StopProcessor()
{
await _processor.StopProcessingAsync();
await _processor.CloseAsync();
}
private Task ProcessErrorAsync(ProcessErrorEventArgs args)
{
_logger.LogError(args.Exception, "Uncaught handled exception", args.ErrorSource, args.FullyQualifiedNamespace, args.EntityPath);
return Task.CompletedTask;
}
private async Task ProcessMessagesAsync(ProcessMessageEventArgs args)
{
var message = args.Message;
// Process the message.
var sbMessage = $"Received message: SequenceNumber:{message.SequenceNumber} Body:{Encoding.UTF8.GetString(message.Body)}";
_logger.LogInformation(sbMessage);
//Handle your message
}
public bool IsClosedOrClosing()
{
return ((_processor == null) || _processor.IsClosed || !_processor.IsProcessing);
}
}

Create delay between two message reads of a Queue?

I am using Azure Queues to perform a bulk import.
I am using WebJobs to perform the process in the background.
The queue dequeues very frequently. How do I create a delay between 2 message
reads?
This is how I am adding a message to the Queue
public async Task<bool> Handle(CreateFileUploadCommand message)
{
var queueClient = _queueService.GetQueueClient(Constants.Queues.ImportQueue);
var brokeredMessage = new BrokeredMessage(JsonConvert.SerializeObject(new ProcessFileUploadMessage
{
TenantId = message.TenantId,
FileExtension = message.FileExtension,
FileName = message.Name,
DeviceId = message.DeviceId,
SessionId = message.SessionId,
UserId = message.UserId,
OutletId = message.OutletId,
CorrelationId = message.CorrelationId,
}))
{
ContentType = "application/json",
};
await queueClient.SendAsync(brokeredMessage);
return true;
}
And Below is the WebJobs Function.
public class Functions
{
private readonly IValueProvider _valueProvider;
public Functions(IValueProvider valueProvider)
{
_valueProvider = valueProvider;
}
public async Task ProcessQueueMessage([ServiceBusTrigger(Constants.Constants.Queues.ImportQueue)] BrokeredMessage message,
TextWriter logger)
{
var queueMessage = message.GetBody<string>();
using (var client = new HttpClient())
{
client.BaseAddress = new Uri(_valueProvider.Get("ServiceBaseUri"));
var stringContent = new StringContent(queueMessage, Encoding.UTF8, "application/json");
var result = await client.PostAsync(RestfulUrls.ImportMenu.ProcessUrl, stringContent);
if (result.IsSuccessStatusCode)
{
await message.CompleteAsync();
}
else
{
await message.AbandonAsync();
}
}
}
}

As far as I know, azure webjobs sdk enable concurrent processing on a single instance(the default is 16).
If you run your webjobs, it will read 16 queue messages(peeklock and calls Complete on the message if the function finishes successfully, or calls Abandon) and create 16 processes to execute the trigger function at same time. So you feel the queue dequeues very frequently.
If you want to disable concurrent processing on a single instance.
I suggest you could set ServiceBusConfiguration's MessageOptions.MaxConcurrentCalls to 1.
More details, you could refer to below codes:
In the program.cs:
JobHostConfiguration config = new JobHostConfiguration();
ServiceBusConfiguration serviceBusConfig = new ServiceBusConfiguration();
serviceBusConfig.MessageOptions.MaxConcurrentCalls = 1;
config.UseServiceBus(serviceBusConfig);
JobHost host = new JobHost(config);
host.RunAndBlock();
If you want to create a delay between 2 message reads, I suggest you could create a custom ServiceBusConfiguration.MessagingProvider.
It contains CompleteProcessingMessageAsync method, this method completes processing of the specified message, after the job function has been invoked.
I suggest you could add thread.sleep method in CompleteProcessingMessageAsync to achieve delay read.
More detail, you could refer to below code sample:
CustomMessagingProvider.cs:
Notice: I override the CompleteProcessingMessageAsync method codes.
public class CustomMessagingProvider : MessagingProvider
{
private readonly ServiceBusConfiguration _config;
public CustomMessagingProvider(ServiceBusConfiguration config)
: base(config)
{
_config = config;
}
public override NamespaceManager CreateNamespaceManager(string connectionStringName = null)
{
// you could return your own NamespaceManager here, which would be used
// globally
return base.CreateNamespaceManager(connectionStringName);
}
public override MessagingFactory CreateMessagingFactory(string entityPath, string connectionStringName = null)
{
// you could return a customized (or new) MessagingFactory here per entity
return base.CreateMessagingFactory(entityPath, connectionStringName);
}
public override MessageProcessor CreateMessageProcessor(string entityPath)
{
// demonstrates how to plug in a custom MessageProcessor
// you could use the global MessageOptions, or use different
// options per entity
return new CustomMessageProcessor(_config.MessageOptions);
}
private class CustomMessageProcessor : MessageProcessor
{
public CustomMessageProcessor(OnMessageOptions messageOptions)
: base(messageOptions)
{
}
public override Task<bool> BeginProcessingMessageAsync(BrokeredMessage message, CancellationToken cancellationToken)
{
// intercept messages before the job function is invoked
return base.BeginProcessingMessageAsync(message, cancellationToken);
}
public override async Task CompleteProcessingMessageAsync(BrokeredMessage message, FunctionResult result, CancellationToken cancellationToken)
{
if (result.Succeeded)
{
if (!MessageOptions.AutoComplete)
{
// AutoComplete is true by default, but if set to false
// we need to complete the message
cancellationToken.ThrowIfCancellationRequested();
await message.CompleteAsync();
Console.WriteLine("Begin sleep");
//Sleep 5 seconds
Thread.Sleep(5000);
Console.WriteLine("Sleep 5 seconds");
}
}
else
{
cancellationToken.ThrowIfCancellationRequested();
await message.AbandonAsync();
}
}
}
}
Program.cs main method:
static void Main()
{
var config = new JobHostConfiguration();
if (config.IsDevelopment)
{
config.UseDevelopmentSettings();
}
var sbConfig = new ServiceBusConfiguration
{
MessageOptions = new OnMessageOptions
{
AutoComplete = false,
MaxConcurrentCalls = 1
}
};
sbConfig.MessagingProvider = new CustomMessagingProvider(sbConfig);
config.UseServiceBus(sbConfig);
var host = new JobHost(config);
// The following code ensures that the WebJob will be running continuously
host.RunAndBlock();
}
Result:

Queue of async tasks with throttling which supports muti-threading

I need to implement a library to request vk.com API. The problem is that API supports only 3 requests per second. I would like to have API asynchronous.
Important: API should support safe accessing from multiple threads.
My idea is implement some class called throttler which allow no more than 3 request/second and delay other request.
The interface is next:
public interface IThrottler : IDisposable
{
Task<TResult> Throttle<TResult>(Func<Task<TResult>> task);
}
The usage is like
var audio = await throttler.Throttle(() => api.MyAudio());
var messages = await throttler.Throttle(() => api.ReadMessages());
var audioLyrics = await throttler.Throttle(() => api.AudioLyrics(audioId));
/// Here should be delay because 3 requests executed
var photo = await throttler.Throttle(() => api.MyPhoto());
How to implement throttler?
Currently I implemented it as queue which is processed by background thread.
public Task<TResult> Throttle<TResult>(Func<Task<TResult>> task)
{
/// TaskRequest has method Run() to run task
/// TaskRequest uses TaskCompletionSource to provide new task
/// which is resolved when queue processed til this element.
var request = new TaskRequest<TResult>(task);
requestQueue.Enqueue(request);
return request.ResultTask;
}
This is shorten code of background thread loop which process the queue:
private void ProcessQueue(object state)
{
while (true)
{
IRequest request;
while (requestQueue.TryDequeue(out request))
{
/// Delay method calculates actual delay value and calls Thread.Sleep()
Delay();
request.Run();
}
}
}
Is it possible to implement this without background thread?

So we'll start out with a solution to a simpler problem, that of creating a queue that process up to N tasks concurrently, rather than throttling to N tasks started per second, and build on that:
public class TaskQueue
{
private SemaphoreSlim semaphore;
public TaskQueue()
{
semaphore = new SemaphoreSlim(1);
}
public TaskQueue(int concurrentRequests)
{
semaphore = new SemaphoreSlim(concurrentRequests);
}
public async Task<T> Enqueue<T>(Func<Task<T>> taskGenerator)
{
await semaphore.WaitAsync();
try
{
return await taskGenerator();
}
finally
{
semaphore.Release();
}
}
public async Task Enqueue(Func<Task> taskGenerator)
{
await semaphore.WaitAsync();
try
{
await taskGenerator();
}
finally
{
semaphore.Release();
}
}
}
We'll also use the following helper methods to match the result of a TaskCompletionSource to a `Task:
public static void Match<T>(this TaskCompletionSource<T> tcs, Task<T> task)
{
task.ContinueWith(t =>
{
switch (t.Status)
{
case TaskStatus.Canceled:
tcs.SetCanceled();
break;
case TaskStatus.Faulted:
tcs.SetException(t.Exception.InnerExceptions);
break;
case TaskStatus.RanToCompletion:
tcs.SetResult(t.Result);
break;
}
});
}
public static void Match<T>(this TaskCompletionSource<T> tcs, Task task)
{
Match(tcs, task.ContinueWith(t => default(T)));
}
Now for our actual solution what we can do is each time we need to perform a throttled operation we create a TaskCompletionSource, and then go into our TaskQueue and add an item that starts the task, matches the TCS to its result, doesn't await it, and then delays the task queue for 1 second. The task queue will then not allow a task to start until there are no longer N tasks started in the past second, while the result of the operation itself is the same as the create Task:
public class Throttler
{
private TaskQueue queue;
public Throttler(int requestsPerSecond)
{
queue = new TaskQueue(requestsPerSecond);
}
public Task<T> Enqueue<T>(Func<Task<T>> taskGenerator)
{
TaskCompletionSource<T> tcs = new TaskCompletionSource<T>();
var unused = queue.Enqueue(() =>
{
tcs.Match(taskGenerator());
return Task.Delay(TimeSpan.FromSeconds(1));
});
return tcs.Task;
}
public Task Enqueue<T>(Func<Task> taskGenerator)
{
TaskCompletionSource<bool> tcs = new TaskCompletionSource<bool>();
var unused = queue.Enqueue(() =>
{
tcs.Match(taskGenerator());
return Task.Delay(TimeSpan.FromSeconds(1));
});
return tcs.Task;
}
}

I solved a similar problem using a wrapper around SemaphoreSlim. In my scenario, I had some other throttling mechanisms as well, and I needed to make sure that requests didn't hit the external API too often even if request number 1 took longer to reach the API than request number 3. My solution was to use a wrapper around SemaphoreSlim that had to be released by the caller, but the actual SemaphoreSlim would not be released until a set time had passed.
public class TimeGatedSemaphore
{
private readonly SemaphoreSlim semaphore;
public TimeGatedSemaphore(int maxRequest, TimeSpan minimumHoldTime)
{
semaphore = new SemaphoreSlim(maxRequest);
MinimumHoldTime = minimumHoldTime;
}
public TimeSpan MinimumHoldTime { get; }
public async Task<IDisposable> WaitAsync()
{
await semaphore.WaitAsync();
return new InternalReleaser(semaphore, Task.Delay(MinimumHoldTime));
}
private class InternalReleaser : IDisposable
{
private readonly SemaphoreSlim semaphoreToRelease;
private readonly Task notBeforeTask;
public InternalReleaser(SemaphoreSlim semaphoreSlim, Task dependantTask)
{
semaphoreToRelease = semaphoreSlim;
notBeforeTask = dependantTask;
}
public void Dispose()
{
notBeforeTask.ContinueWith(_ => semaphoreToRelease.Release());
}
}
}
Example usage:
private TimeGatedSemaphore requestThrottler = new TimeGatedSemaphore(3, TimeSpan.FromSeconds(1));
public async Task<T> MyRequestSenderHelper(string endpoint)
{
using (await requestThrottler.WaitAsync())
return await SendRequestToAPI(endpoint);
}

Here is one solution that uses a Stopwatch:
public class Throttler : IThrottler
{
private readonly Stopwatch m_Stopwatch;
private int m_NumberOfRequestsInLastSecond;
private readonly int m_MaxNumberOfRequestsPerSecond;
public Throttler(int max_number_of_requests_per_second)
{
m_MaxNumberOfRequestsPerSecond = max_number_of_requests_per_second;
m_Stopwatch = Stopwatch.StartNew();
}
public async Task<TResult> Throttle<TResult>(Func<Task<TResult>> task)
{
var elapsed = m_Stopwatch.Elapsed;
if (elapsed > TimeSpan.FromSeconds(1))
{
m_NumberOfRequestsInLastSecond = 1;
m_Stopwatch.Restart();
return await task();
}
if (m_NumberOfRequestsInLastSecond >= m_MaxNumberOfRequestsPerSecond)
{
TimeSpan time_to_wait = TimeSpan.FromSeconds(1) - elapsed;
await Task.Delay(time_to_wait);
m_NumberOfRequestsInLastSecond = 1;
m_Stopwatch.Restart();
return await task();
}
m_NumberOfRequestsInLastSecond++;
return await task();
}
}
Here is how this code can be tested:
class Program
{
static void Main(string[] args)
{
DoIt();
Console.ReadLine();
}
static async Task DoIt()
{
Func<Task<int>> func = async () =>
{
await Task.Delay(100);
return 1;
};
Throttler throttler = new Throttler(3);
for (int i = 0; i < 10; i++)
{
var result = await throttler.Throttle(func);
Console.WriteLine(DateTime.Now);
}
}
}

You can use this as Generic
public TaskThrottle(int maxTasksToRunInParallel)
{
_semaphore = new SemaphoreSlim(maxTasksToRunInParallel);
}
public void TaskThrottler<T>(IEnumerable<Task<T>> tasks, int timeoutInMilliseconds, CancellationToken cancellationToken = default(CancellationToken)) where T : class
{
// Get Tasks as List
var taskList = tasks as IList<Task<T>> ?? tasks.ToList();
var postTasks = new List<Task<int>>();
// When the first task completed, it will flag
taskList.ForEach(x =>
{
postTasks.Add(x.ContinueWith(y => _semaphore.Release(), cancellationToken));
});
taskList.ForEach(x =>
{
// Wait for open slot
_semaphore.Wait(timeoutInMilliseconds, cancellationToken);
cancellationToken.ThrowIfCancellationRequested();
x.Start();
});
Task.WaitAll(taskList.ToArray(), cancellationToken);
}

Edit: this solution works but use it only if it is ok to process all request in serial (in one thread). Otherwise use solution accepted as answer.
Well, thanks to Best way in .NET to manage queue of tasks on a separate (single) thread
My question is almost duplicate except adding delay before execution, which is actually simple.
The main helper here is SemaphoreSlim class which allows to restrict degree of parallelism.
So, first create a semaphore:
// Semaphore allows run 1 thread concurrently.
private readonly SemaphoreSlim semaphore = new SemaphoreSlim(1, 1);
And, final version of throttle looks like
public async Task<TResult> Throttle<TResult>(Func<Task<TResult>> task)
{
await semaphore.WaitAsync();
try
{
await delaySource.Delay();
return await task();
}
finally
{
semaphore.Release();
}
}
Delay source is also pretty simple:
private class TaskDelaySource
{
private readonly int maxTasks;
private readonly TimeSpan inInterval;
private readonly Queue<long> ticks = new Queue<long>();
public TaskDelaySource(int maxTasks, TimeSpan inInterval)
{
this.maxTasks = maxTasks;
this.inInterval = inInterval;
}
public async Task Delay()
{
// We will measure time of last maxTasks tasks.
while (ticks.Count > maxTasks)
ticks.Dequeue();
if (ticks.Any())
{
var now = DateTime.UtcNow.Ticks;
var lastTick = ticks.First();
// Calculate interval between last maxTasks task and current time
var intervalSinceLastTask = TimeSpan.FromTicks(now - lastTick);
if (intervalSinceLastTask < inInterval)
await Task.Delay((int)(inInterval - intervalSinceLastTask).TotalMilliseconds);
}
ticks.Enqueue(DateTime.UtcNow.Ticks);
}
}