Develop Reference

Why is my loop ending after task cancellation?

I have a music queue manager that plays the next music if the queue is not empty. I also have a "skip" function that stops the current music by cancelling the current task. To stop the music, I create and forward a CancellationTokenSource.Token to PlayMusicAsync() function. When the task "PlayMusicAsync()" is cancelled, the CancellationTokenSource is reseted and the loop ends, instead of play the next song.
MusicQueueManager.cs:
public class MusicQueueManager
{
private CancellationTokenSource _cts = new();
private readonly MusicPlayer _musicPlayer;
public MusicQueueManager(MusicPlayer musicPlayer)
{
_musicPlayer = musicPlayer;
}
public List<YoutubeMediaInfo> MusicQueue { get; private set; } = new();
public bool IsPlaying { get; private set; } = false;
public bool HasNextSong => MusicQueue.Count > 0;
public async Task StartPlayingAsync(IAudioClient audioClient, SocketCommandContext context)
{
IsPlaying = true;
while (HasNextSong)
{
var nextSong = PopSong();
if (nextSong is null)
{
await context.Message.ReplyAsync("Error.");
continue;
}
try
{
await _musicPlayer.PlayMusicAsync(nextSong, audioClient, context, _cts)
}
finally
{
_cts = new CancellationTokenSource();
}
await Task.Delay(1000);
}
IsPlaying = false;
}
public void SkipSong()
{
_cts.Cancel();
}
}
MusicPlayer.cs:
public class MusicPlayer
{
public async Task PlayMusicAsync(YoutubeMediaInfo youtubeMediaInfo, IAudioClient audioClient, SocketCommandContext context, CancellationTokenSource cts)
{
var youtube = new YoutubeClient();
var streamInfo = youtubeMediaInfo.StreamManifest?.GetAudioOnlyStreams().GetWithHighestBitrate();
if (streamInfo is null)
{
return;
}
var stream = await youtube.Videos.Streams.GetAsync(streamInfo);
var memoryStream = new MemoryStream();
await Cli.Wrap("ffmpeg")
.WithArguments(" -hide_banner -loglevel panic -i pipe:0 -ac 2 -f s16le -ar 48000 pipe:1")
.WithStandardInputPipe(PipeSource.FromStream(stream))
.WithStandardOutputPipe(PipeTarget.ToStream(memoryStream))
.ExecuteAsync();
using var discord = audioClient.CreatePCMStream(AudioApplication.Mixed);
try
{
await discord.WriteAsync(memoryStream.ToArray().AsMemory(0, (int)memoryStream.Length), cts.Token);
}
finally
{
await discord.FlushAsync();
}
}
}
I tried a solution that adds .ContinueWith(x => { return; }) to PlayMusicAsync() and it worked well, but I don't know why.
Code Snippet:
public async Task StartPlayingAsync(IAudioClient audioClient, SocketCommandContext context)
{
IsPlaying = true;
while (HasNextSong)
{
var nextSong = PopSong();
if (nextSong is null)
{
await context.Message.ReplyAsync("Erro.");
continue;
}
try
{
await _musicPlayer.PlayMusicAsync(nextSong, audioClient, context, _cts).ContinueWith(x => { return; });
}
finally
{
_cts = new CancellationTokenSource();
}
await Task.Delay(1000);
}
IsPlaying = false;
}

Upon cancellation it's most likely that await _musicPlayer.PlayMusicAsync throws the whole while loop "stops" (StartPlayingAsync exits with an exception).
In order to keep the loop going you need to catch OperationCanceledException:
try
{
await _musicPlayer.PlayMusicAsync(nextSong, audioClient, context,
}
catch (OperationCanceledException ex)
{
// Reste cts here
}
BTW. There is no need for PlayMusicAsync to accept a token source, and normally a method like this would accept just a token.

Enabling Await and Exception Capture with TPL Dataflow

I have designed a simple JobProcessor using TPL Data flow (my first time using it). I want to be able to create jobs, and have them invoked and placed on a priority queue (the PriorityBufferBlock). My code structure is as follows
public interface IJob<TInput>
{
Task Execute(TInput input);
Priority Priority { get; }
}
where
public enum Priority
{
High,
Medium,
Low
}
and I have a custom version of the PriorityBufferBlock (taken from PriorityBufferBlock) with a custom refreshing cache implementation, to ensure clean up of the messages that are "reserved", this looks like
class PriorityBufferBlock<T> : ISourceBlock<T>, IReceivableSourceBlock<T>
{
private readonly BufferBlock<T> _highPriorityBuffer;
private readonly BufferBlock<T> _mediumPriorityBuffer;
private readonly BufferBlock<T> _lowPriorityBuffer;
private readonly RefreshingInMemoryCache<DataflowMessageHeader, ISourceBlock<T>> _messagesCache;
// ... More code here
}
The JobProcessor interface is
public interface IJobProcessor<TInput>
{
void RegisterHandler<TTask>(TInput input) where TTask : IJob<TInput>;
Task Enqueue(IJob<TInput> task);
}
with implementation as
public class JobProcessor<TInput> : IJobProcessor<TInput>
{
private readonly PriorityBufferBlock<IJob<TInput>> _priorityBufferBlock;
private readonly IOptions<AzureOptions> _options;
private readonly ILogger<IJobProcessor<TInput>> _logger;
private readonly CancellationToken _token;
public JobProcessor(
IClock clock,
IOptions<AzureOptions> options,
ILogger<IJobProcessor<TInput>> logger,
CancellationToken token)
{
_options = options ?? throw new ArgumentNullException(nameof(options));
_logger = logger;
_token = token;
var dataflowBlockOptions = new DataflowBlockOptions { CancellationToken = token };
_priorityBufferBlock = new PriorityBufferBlock<IJob<TInput>>(dataflowBlockOptions, clock, options, logger);
_logger?.LogInformation($"{nameof(JobProcessor<TInput>)} initialized and configured successfully");
}
public void RegisterHandler<TJob>(TInput input) where TJob : IJob<TInput>
{
var actionBlock = new ActionBlock<IJob<TInput>>(
(job) => job.Execute(input),
new ExecutionDataflowBlockOptions
{
CancellationToken = _token,
MaxDegreeOfParallelism = _options.Value.TaskProcessorMaxDegreeOfParallelism
});
_priorityBufferBlock.LinkTo(
actionBlock,
new DataflowLinkOptions
{
PropagateCompletion = true
},
(task) => task is IJob<TInput>);
_logger?.LogInformation($"Handler for {typeof(TJob).Name} registered successfully");
}
public async Task Enqueue(IJob<TInput> task)
{
await _priorityBufferBlock.SendAsync(task, task.Priority);
_logger?.LogInformation($"Successfully enqueued {task.GetType().Name} for processing");
}
}
and this is proving to work well for everything except Exception handling. Because I am not awaiting the actionBlock, any exception from inside the running IJob are swallowed, as you would expect (here I am essentially doing Task.Run(() => throw new Exception());.
My question is, what is the best way to change the above so I can await and bubble the exceptions upwards to allow error responses from this API?
I have tried
public async Task RegisterHandler<TJob>(TInput input) where TJob : IJob<TInput>
{
var actionBlock = new ActionBlock<IJob<TInput>>(
(job) => job.Execute(input),
new ExecutionDataflowBlockOptions
{
CancellationToken = _token,
MaxDegreeOfParallelism = _options.Value.TaskProcessorMaxDegreeOfParallelism
});
await actionBlock.Completion;
_priorityBufferBlock.LinkTo(
actionBlock,
new DataflowLinkOptions
{
PropagateCompletion = true
},
(job) => job is IJob<TInput>);
_logger?.LogInformation($"Handler for {typeof(TJob).Name} registered successfully");
}
but this is not working and also makes no sense, I am a bit lost. Any help hugely appreciated. Thanks for your time.
NOTE:
An example of the working code via unit test is
[TestCase(1)]
[TestCase(3)]
[TestCase(6)]
public async Task EnsureConcurrencyLimitsAreNotExceeded(int maxDegreeOfParallelism)
{
var waitHandle = new ManualResetEventSlim(false);
var priorityBuffer = new ConcurrentQueue<Priority>();
_mockGeneralOptions.SetReturnsDefault(new AzureOptions()
{
PriorityBufferBlockExpiryMilliseconds = 10,
TaskProcessorMaxDegreeOfParallelism = maxDegreeOfParallelism
});
var jobProcessor = new JobProcessor<OptionalPair<ViewformTour, ViewformTour>>(
_mockClock.Object,
_mockGeneralOptions.Object,
_mockLogger.Object,
CancellationToken.None
);
var tour = new ViewformTour
{
Id = "_id"
};
jobProcessor.RegisterHandler<HighPriorityJob>(
new OptionalPair<ViewformTour, ViewformTour>(
Optional.From(tour),
Optional.None<ViewformTour>()
));
jobProcessor.RegisterHandler<LowPriorityJob>(
new OptionalPair<ViewformTour, ViewformTour>(
Optional.From(tour),
Optional.None<ViewformTour>()
));
var tasks = new List<Task>();
tasks.Add(jobProcessor.Enqueue(new LowPriorityJob(waitHandle, priorityBuffer)));
tasks.Add(jobProcessor.Enqueue(new LowPriorityJob(waitHandle, priorityBuffer)));
tasks.Add(jobProcessor.Enqueue(new LowPriorityJob(waitHandle, priorityBuffer)));
tasks.Add(jobProcessor.Enqueue(new LowPriorityJob(waitHandle, priorityBuffer)));
tasks.Add(jobProcessor.Enqueue(new LowPriorityJob(waitHandle, priorityBuffer)));
tasks.Add(jobProcessor.Enqueue(new LowPriorityJob(waitHandle, priorityBuffer)));
await Task.WhenAll(tasks.ToArray());
Thread.Sleep(150);
waitHandle.Set();
Assert.That(priorityBuffer, Is.Not.Null);
Assert.That(priorityBuffer.Count, Is.EqualTo(maxDegreeOfParallelism));
}
with
private class HighPriorityJob : IJob<OptionalPair<ViewformTour, ViewformTour>>
{
private ConcurrentQueue<Priority> _priorityBuffer;
public HighPriorityJob(ConcurrentQueue<Priority> priorityBuffer)
{
_priorityBuffer = priorityBuffer;
}
public Task Execute(OptionalPair<ViewformTour, ViewformTour> input)
{
_priorityBuffer.Enqueue(Priority);
return Task.CompletedTask;
}
public Priority Priority => Priority.High;
}
private class LowPriorityJob : IJob<OptionalPair<ViewformTour, ViewformTour>>
{
private ManualResetEventSlim _waitHandle;
private ConcurrentQueue<Priority> _priorityBuffer;
private int _delayMilliseconds;
private bool _setWaitHandle = false;
public LowPriorityJob(
ManualResetEventSlim waitHandle,
ConcurrentQueue<Priority> priorityBuffer,
bool setWaitHandle = false,
int delayMilliseconds = 100)
{
_waitHandle = waitHandle;
_priorityBuffer = priorityBuffer;
_delayMilliseconds = delayMilliseconds;
_setWaitHandle = setWaitHandle;
}
public async Task Execute(OptionalPair<ViewformTour, ViewformTour> input)
{
await Task.Delay(_delayMilliseconds);
_priorityBuffer.Enqueue(Priority);
if (_setWaitHandle)
{
_waitHandle.Set();
}
}
public Priority Priority => Priority.Low;
}
EDIT II:
Okay, so I have now tried the following - the RegisterHandler now returns the ActionBlock on registration, I then set up a continuation
public ActionBlock<IJob<TInput>> RegisterHandler<TJob>(TInput input) where TJob : IJob<TInput>
{
var actionBlock = new ActionBlock<IJob<TInput>>(
(job) => job.Execute(input),
new ExecutionDataflowBlockOptions
{
CancellationToken = _token,
MaxDegreeOfParallelism = _options.Value.TaskProcessorMaxDegreeOfParallelism
});
_priorityBufferBlock.LinkTo(
actionBlock,
new DataflowLinkOptions
{
PropagateCompletion = true
},
(job) => job is IJob<TInput>);
_logger?.LogInformation($"Handler for {typeof(TJob).Name} registered successfully");
return actionBlock;
}
Then in my test
[Test]
public async Task DoesHandleExceptionGracefully()
{
var priorityBuffer = new ConcurrentQueue<Priority>();
var jobProcessor = new JobProcessor<OptionalPair<ViewformTour, ViewformTour>>(
_mockClock.Object,
_mockGeneralOptions.Object,
_mockLogger.Object,
CancellationToken.None
);
var tour = new ViewformTour
{
Id = "_id"
};
try
{
var handler = jobProcessor.RegisterHandler<ThrowingHighPriorityJob>(
new OptionalPair<ViewformTour, ViewformTour>(
Optional.From(tour),
Optional.None<ViewformTour>()
));
await jobProcessor.Enqueue(new ThrowingHighPriorityJob());
await handler.Completion.ContinueWith(ant =>
{
throw new Exception("... From Continuation");
}, TaskContinuationOptions.OnlyOnFaulted);
}
catch (Exception ex)
{
Console.WriteLine($"External capture{ex.Message}");
}
}
This outputs "External capture... From continuation", we have externalized the exception handling. HOWEVER, the continuation setup is now blocking. In production, I want to enqueue jobs dynamically, and this prevents that. :'[

In the end, there was no way to do this cleanly without breaking the control flow and go against the "design principles" of the library. So, to do this, I merely embraced the pipeline/dataflow mantra and created a IPoisonQueue
public class PoisonQueue<TInput> : IPoisonQueue<TInput>
{
private readonly IPriorityBufferBlock<IJob<TInput>> _priorityBufferBlock;
public PoisonQueue(IPriorityBufferBlock<IJob<TInput>> priorityBufferBlock)
{
_priorityBufferBlock = priorityBufferBlock ?? throw new ArgumentNullException(nameof(priorityBufferBlock));
}
public async Task Enqueue(IJob<TInput> job)
{
// Push the PoisonJob onto the buffer block...
}
}
Then I have gone further with the DI for the JobProcessor and unit tested this to high heaven, the final implementation is
public class JobProcessor<TInput> : IJobProcessor<TInput>
{
private readonly IPriorityBufferBlock<IJob<TInput>> _priorityBufferBlock;
private readonly IPoisonQueue<TInput> _poisonQueue;
private readonly IOptions<GeneralOptions> _options;
private readonly ILogger<IJobProcessor<TInput>> _logger;
public JobProcessor(
IPriorityBufferBlock<IJob<TInput>> priorityBufferBlock,
IPoisonQueue<TInput> poisonQueue,
IClock clock,
IOptions<GeneralOptions> options,
ILogger<IJobProcessor<TInput>> logger)
{
_options = options ?? throw new ArgumentNullException(nameof(options));
_logger = logger;
_priorityBufferBlock = priorityBufferBlock ?? throw new ArgumentNullException(nameof(priorityBufferBlock));
_priorityBufferBlock.Initialize();
_poisonQueue = poisonQueue ?? throw new ArgumentNullException(nameof(poisonQueue));
_logger?.LogInformation($"{nameof(JobProcessor<TInput>)} initialized and configured successfully");
}
public void RegisterHandler<TJob>(TInput input) where TJob : IJob<TInput>
{
var actionBlock = new ActionBlock<IJob<TInput>>(
async (job) =>
{
var jobType = job.GetType().Name;
var retryCount = _options.Value.JobProcessorRetryCount;
var retryIntervalMilliseconds = _options.Value.JobProcessorRetryIntervalMilliseconds;
var retryPolicy = Policy
.Handle<Exception>()
.WaitAndRetryAsync(
retryCount,
retryAttempt => TimeSpan.FromMilliseconds(Math.Pow(retryIntervalMilliseconds, retryAttempt)));
try
{
await retryPolicy.ExecuteAsync(async () =>
{
_logger?.LogInformation("Starting execution of job {JobType}...", jobType);
await job.Execute(input);
});
}
catch (Exception ex)
{
_logger?.LogError(ex, $"{jobType} failed");
_logger?.LogWarning("Adding job {JobType} to poison queue...", jobType);
await _poisonQueue.Enqueue(job);
}
},
new ExecutionDataflowBlockOptions
{
MaxDegreeOfParallelism = _options.Value.JobProcessorMaxDegreeOfParallelism
});
_priorityBufferBlock.LinkTo(
actionBlock,
new DataflowLinkOptions
{
PropagateCompletion = true
},
(job) => job is IJob<TInput>);
_logger?.LogInformation($"Handler for {typeof(TJob).Name} registered successfully");
}
public async Task Enqueue(IJob<TInput> job)
{
await _priorityBufferBlock.SendAsync(job, job.Priority);
_logger?.LogInformation($"Successfully enqueued {job.GetType().Name} for processing");
}
}
Here I have use "Polly" for the retry and backoff, and once the retries are complete (and we still have failure), I push a PoisonJob on to the PriorityBufferBlock and the errors are handled in the same way other jobs are. It seems to work well, fingers crossed my test coverage is complete and correct.

JSON-RPC - Handling exceptions with TaskCompletionSource (SetException)

There is a JSON-RPC API, which I'm currently implementing. It can be tested here.
The problem is that if an incorrect DTO model is passed to SendAsync<TResponse>, JsonSerializer.Deserialize is going to throw a JsonException, which is not handled by my code. I know I've got to use SetException in some way, but I don't know how to do it, so here is the question. The exception message should be printed in the console as well.
public sealed class Client : IDisposable
{
...
private readonly ConcurrentDictionary<long, IResponseHandler> _handlers = new();
...
public Task StartAsync(CancellationToken cancellationToken)
{
_ = Task.Run(async () =>
{
await foreach (var message in _client.Start(cancellationToken))
{
using var response = JsonDocument.Parse(message);
try
{
var requestId = response.RootElement.GetProperty("id").GetInt32();
// TODO: Handle JsonException errors via `SetException`?
// TODO: Show error when incorrect input parameters are filled
if (_handlers.ContainsKey(requestId))
{
_handlers[requestId].SetResult(message);
_handlers.TryRemove(requestId, out _);
}
}
catch (KeyNotFoundException)
{
// My point is that a message should be processed only if it doesn't include `id`,
// because that means that the message is an actual web socket subscription.
_messageReceivedSubject.OnNext(message);
}
}
}, cancellationToken);
...
return Task.CompletedTask;
}
public Task<TResponse> SendAsync<TResponse>(string method, object #params)
{
var request = new JsonRpcRequest<object>
{
JsonRpc = "2.0",
Id = NextId(),
Method = method,
Params = #params
};
//var tcs = new TaskCompletionSource<TResponse>();
//_requestManager.Add(request.Id, request, tcs);
var handler = new ResponseHandlerBase<TResponse>();
_handlers[request.Id] = handler;
var message = JsonSerializer.Serialize(request);
_ = _client.SendAsync(message);
return handler.Task;
//return tcs.Task;
}
public async Task<AuthResponse?> AuthenticateAsync(string clientId, string clientSecret)
{
var #params = new Dictionary<string, string>
{
{"grant_type", "client_credentials"},
{"client_id", clientId},
{"client_secret", clientSecret}
};
var response = await SendAsync<SocketResponse<AuthResponse>>("public/auth", #params).ConfigureAwait(false);
return response.Result;
}
...
private interface IResponseHandler
{
void SetResult(string payload);
}
private class ResponseHandlerBase<TRes> : IResponseHandler
{
private readonly TaskCompletionSource<TRes> _tcs = new();
public Task<TRes> Task => _tcs.Task;
public void SetResult(string payload)
{
var result = JsonSerializer.Deserialize(payload, typeof(TRes));
_tcs.SetResult((TRes) result);
}
}
}

Coincidentally, I did something very similar while live-coding a TCP/IP chat application last week.
Since in this case you already have an IAsyncEnumerable<string> - and since you can get messages other than responses - I recommend also exposing that IAsyncEnumerable<string>:
public sealed class Client : IDisposable
{
public async IAsyncEnumerable<string> Start(CancellationToken cancellationToken)
{
await foreach (var message in _client.Start(cancellationToken))
{
// TODO: parse and handle responses for our requests
yield return message;
}
}
}
You can change this to be Rx-based if you want (_messageReceivedSubject.OnNext), but I figure if you already have IAsyncEnumerable<T>, then you may as well keep the same abstraction.
Then, you can parse and detect responses, passing along all other messages:
public sealed class Client : IDisposable
{
public async IAsyncEnumerable<string> Start(CancellationToken cancellationToken)
{
await foreach (var message in _client.Start(cancellationToken))
{
var (requestId, response) = TryParseResponse(message);
if (requestId != null)
{
// TODO: handle
}
else
{
yield return message;
}
}
(long? RequestId, JsonDocument? Response) TryParseResponse(string message)
{
try
{
var document = JsonDocument.Parse(message);
var requestId = response.RootElement.GetProperty("id").GetInt32();
return (document, requestId);
}
catch
{
return (null, null);
}
}
}
}
Then, you can define your collection of outstanding requests and handle messages that are for those requests:
public sealed class Client : IDisposable
{
private readonly ConcurrentDictionary<int, TaskCompletionSource<JsonDocument>> _requests = new();
public async IAsyncEnumerable<string> Start(CancellationToken cancellationToken)
{
await foreach (var message in _client.Start(cancellationToken))
{
var (requestId, response) = TryParseResponse(message);
if (requestId != null && _requests.TryRemove(requestId.Value, out var tcs))
{
tcs.TrySetResult(response);
}
else
{
yield return message;
}
}
(long? RequestId, JsonDocument? Response) TryParseResponse(string message)
{
try
{
var document = JsonDocument.Parse(message);
var requestId = response.RootElement.GetProperty("id").GetInt32();
return (document, requestId);
}
catch
{
return (null, null);
}
}
}
}
Note the usage of ConcurrentDictionary.TryRemove, which is safer than accessing the value and then removing it.
Now you can write your general SendAsync. As I note in my video, I prefer to split up the code that runs synchronously in SendAsync and the code that awaits the response:
public sealed class Client : IDisposable
{
...
public Task<TResponse> SendAsync<TResponse>(string method, object #params)
{
var request = new JsonRpcRequest<object>
{
JsonRpc = "2.0",
Id = NextId(),
Method = method,
Params = #params,
};
var tcs = new TaskCompletionSource<JsonDocument>(TaskCreationOptions.RunContinuationsAsynchronously);
_requests.TryAdd(request.Id, tcs);
return SendRequestAndWaitForResponseAsync();
async Task<TResponse> SendRequestAndWaitForResponseAsync()
{
var message = JsonSerializer.Serialize(request);
await _client.SendAsync(message);
var response = await tcs.Task;
return JsonSerializer.Deserialize(response, typeof(TResponse));
}
}
}
I've removed the "handler" concept completely, since it was just providing the type for JsonSerializer.Deserialize. Also, by using a local async method, I can use the async state machine to propagate exceptions naturally.
Then, your higher-level methods can be built on this:
public sealed class Client : IDisposable
{
...
public async Task<AuthResponse?> AuthenticateAsync(string clientId, string clientSecret)
{
var #params = new Dictionary<string, string>
{
{"grant_type", "client_credentials"},
{"client_id", clientId},
{"client_secret", clientSecret}
};
var response = await SendAsync<SocketResponse<AuthResponse>>("public/auth", #params);
return response.Result;
}
}
So the final code ends up being:
public sealed class Client : IDisposable
{
private readonly ConcurrentDictionary<int, TaskCompletionSource<JsonDocument>> _requests = new();
public async IAsyncEnumerable<string> Start(CancellationToken cancellationToken)
{
await foreach (var message in _client.Start(cancellationToken))
{
var (requestId, response) = TryParseResponse(message);
if (requestId != null && _requests.TryRemove(requestId.Value, out var tcs))
{
tcs.TrySetResult(response);
}
else
{
yield return message;
}
}
(long? RequestId, JsonDocument? Response) TryParseResponse(string message)
{
try
{
var document = JsonDocument.Parse(message);
var requestId = response.RootElement.GetProperty("id").GetInt32();
return (document, requestId);
}
catch
{
return (null, null);
}
}
}
public Task<TResponse> SendAsync<TResponse>(string method, object #params)
{
var request = new JsonRpcRequest<object>
{
JsonRpc = "2.0",
Id = NextId(),
Method = method,
Params = #params,
};
var tcs = new TaskCompletionSource<JsonDocument>(TaskCreationOptions.RunContinuationsAsynchronously);
_requests.TryAdd(request.Id, tcs);
return SendRequestAndWaitForResponseAsync();
async Task<TResponse> SendRequestAndWaitForResponseAsync()
{
var message = JsonSerializer.Serialize(request);
await _client.SendAsync(message);
var response = await tcs.Task;
return JsonSerializer.Deserialize(response, typeof(TResponse));
}
}
public async Task<AuthResponse?> AuthenticateAsync(string clientId, string clientSecret)
{
var #params = new Dictionary<string, string>
{
{"grant_type", "client_credentials"},
{"client_id", clientId},
{"client_secret", clientSecret}
};
var response = await SendAsync<SocketResponse<AuthResponse>>("public/auth", #params);
return response.Result;
}
}
You may also want to check out David Fowler's Project Bedrock, which may simplify this code quite a bit.

what's wrong with the implementation of this async chain?

I have the code below in a console app. The LookUpUser method gets called and PostAsJsonAsync gets called but breakpoints in the response checking don't get hit afterwards. What am I doing incorrectly in this implementation?
static void Main(string[] args)
{
TestUseCase().GetAwaiter().GetResult();
}
private static async Task TestUseCase()
{
await GetUserGuids();
}
private static async Task GetUserGuids()
{
var userGuids = new List<Guid>();
userGuids.Add(Guid.Parse("7b5cf09c-196c-4e0b-a0e2-0683e4f11213"));
userGuids.Add(Guid.Parse("3a636154-b7fc-4d96-9cd1-d806119ff79f"));
userGuids.ForEach(async x => await LookUpUser(x));
}
private static async Task LookUpUser(Guid adUserGuid)
{
var client = new HttpClientManager().GetHttpClient();
var response = await client.PostAsJsonAsync("api/v1/users/search", new { ADUserGuid = adUserGuid });
if (response.IsSuccessStatusCode)
{
var groups = await response.Content.ReadAsAsync<List<User>>();
}
else //not 200
{
var message = await response.Content.ReadAsStringAsync();
}
}

userGuids.ForEach(async x => await LookUpUser(x));
The delegate in the ForEach is basically a async void (fire and forget)
Consider selecting a collection of Task and then use Task.WhenAll
private static async Task GetUserGuids() {
var userGuids = new List<Guid>();
userGuids.Add(Guid.Parse("7b5cf09c-196c-4e0b-a0e2-0683e4f11213"));
userGuids.Add(Guid.Parse("3a636154-b7fc-4d96-9cd1-d806119ff79f"));
var tasks = userGuids.Select(x => LookUpUser(x)).ToArray();
await Task.WhenAll(tasks);
}
Also assuming HttpClientManager.GetHttpClient() returns a HttpClient there is no need to create multiple instances. on static client should do
static HttpClient client = new HttpClientManager().GetHttpClient();
private static async Task LookUpUser(Guid adUserGuid) {
var response = await client.PostAsJsonAsync("api/v1/users/search", new { ADUserGuid = adUserGuid });
if (response.IsSuccessStatusCode) {
var groups = await response.Content.ReadAsAsync<List<User>>();
} else {
//not 200
var message = await response.Content.ReadAsStringAsync();
}
}

I got it to work by changing the ForEach to:
foreach (var guid in userGuids)
{
await LookUpUserInSecurityApi(guid);
}

Queue of async tasks with throttling which supports muti-threading

I need to implement a library to request vk.com API. The problem is that API supports only 3 requests per second. I would like to have API asynchronous.
Important: API should support safe accessing from multiple threads.
My idea is implement some class called throttler which allow no more than 3 request/second and delay other request.
The interface is next:
public interface IThrottler : IDisposable
{
Task<TResult> Throttle<TResult>(Func<Task<TResult>> task);
}
The usage is like
var audio = await throttler.Throttle(() => api.MyAudio());
var messages = await throttler.Throttle(() => api.ReadMessages());
var audioLyrics = await throttler.Throttle(() => api.AudioLyrics(audioId));
/// Here should be delay because 3 requests executed
var photo = await throttler.Throttle(() => api.MyPhoto());
How to implement throttler?
Currently I implemented it as queue which is processed by background thread.
public Task<TResult> Throttle<TResult>(Func<Task<TResult>> task)
{
/// TaskRequest has method Run() to run task
/// TaskRequest uses TaskCompletionSource to provide new task
/// which is resolved when queue processed til this element.
var request = new TaskRequest<TResult>(task);
requestQueue.Enqueue(request);
return request.ResultTask;
}
This is shorten code of background thread loop which process the queue:
private void ProcessQueue(object state)
{
while (true)
{
IRequest request;
while (requestQueue.TryDequeue(out request))
{
/// Delay method calculates actual delay value and calls Thread.Sleep()
Delay();
request.Run();
}
}
}
Is it possible to implement this without background thread?

So we'll start out with a solution to a simpler problem, that of creating a queue that process up to N tasks concurrently, rather than throttling to N tasks started per second, and build on that:
public class TaskQueue
{
private SemaphoreSlim semaphore;
public TaskQueue()
{
semaphore = new SemaphoreSlim(1);
}
public TaskQueue(int concurrentRequests)
{
semaphore = new SemaphoreSlim(concurrentRequests);
}
public async Task<T> Enqueue<T>(Func<Task<T>> taskGenerator)
{
await semaphore.WaitAsync();
try
{
return await taskGenerator();
}
finally
{
semaphore.Release();
}
}
public async Task Enqueue(Func<Task> taskGenerator)
{
await semaphore.WaitAsync();
try
{
await taskGenerator();
}
finally
{
semaphore.Release();
}
}
}
We'll also use the following helper methods to match the result of a TaskCompletionSource to a `Task:
public static void Match<T>(this TaskCompletionSource<T> tcs, Task<T> task)
{
task.ContinueWith(t =>
{
switch (t.Status)
{
case TaskStatus.Canceled:
tcs.SetCanceled();
break;
case TaskStatus.Faulted:
tcs.SetException(t.Exception.InnerExceptions);
break;
case TaskStatus.RanToCompletion:
tcs.SetResult(t.Result);
break;
}
});
}
public static void Match<T>(this TaskCompletionSource<T> tcs, Task task)
{
Match(tcs, task.ContinueWith(t => default(T)));
}
Now for our actual solution what we can do is each time we need to perform a throttled operation we create a TaskCompletionSource, and then go into our TaskQueue and add an item that starts the task, matches the TCS to its result, doesn't await it, and then delays the task queue for 1 second. The task queue will then not allow a task to start until there are no longer N tasks started in the past second, while the result of the operation itself is the same as the create Task:
public class Throttler
{
private TaskQueue queue;
public Throttler(int requestsPerSecond)
{
queue = new TaskQueue(requestsPerSecond);
}
public Task<T> Enqueue<T>(Func<Task<T>> taskGenerator)
{
TaskCompletionSource<T> tcs = new TaskCompletionSource<T>();
var unused = queue.Enqueue(() =>
{
tcs.Match(taskGenerator());
return Task.Delay(TimeSpan.FromSeconds(1));
});
return tcs.Task;
}
public Task Enqueue<T>(Func<Task> taskGenerator)
{
TaskCompletionSource<bool> tcs = new TaskCompletionSource<bool>();
var unused = queue.Enqueue(() =>
{
tcs.Match(taskGenerator());
return Task.Delay(TimeSpan.FromSeconds(1));
});
return tcs.Task;
}
}

I solved a similar problem using a wrapper around SemaphoreSlim. In my scenario, I had some other throttling mechanisms as well, and I needed to make sure that requests didn't hit the external API too often even if request number 1 took longer to reach the API than request number 3. My solution was to use a wrapper around SemaphoreSlim that had to be released by the caller, but the actual SemaphoreSlim would not be released until a set time had passed.
public class TimeGatedSemaphore
{
private readonly SemaphoreSlim semaphore;
public TimeGatedSemaphore(int maxRequest, TimeSpan minimumHoldTime)
{
semaphore = new SemaphoreSlim(maxRequest);
MinimumHoldTime = minimumHoldTime;
}
public TimeSpan MinimumHoldTime { get; }
public async Task<IDisposable> WaitAsync()
{
await semaphore.WaitAsync();
return new InternalReleaser(semaphore, Task.Delay(MinimumHoldTime));
}
private class InternalReleaser : IDisposable
{
private readonly SemaphoreSlim semaphoreToRelease;
private readonly Task notBeforeTask;
public InternalReleaser(SemaphoreSlim semaphoreSlim, Task dependantTask)
{
semaphoreToRelease = semaphoreSlim;
notBeforeTask = dependantTask;
}
public void Dispose()
{
notBeforeTask.ContinueWith(_ => semaphoreToRelease.Release());
}
}
}
Example usage:
private TimeGatedSemaphore requestThrottler = new TimeGatedSemaphore(3, TimeSpan.FromSeconds(1));
public async Task<T> MyRequestSenderHelper(string endpoint)
{
using (await requestThrottler.WaitAsync())
return await SendRequestToAPI(endpoint);
}

Here is one solution that uses a Stopwatch:
public class Throttler : IThrottler
{
private readonly Stopwatch m_Stopwatch;
private int m_NumberOfRequestsInLastSecond;
private readonly int m_MaxNumberOfRequestsPerSecond;
public Throttler(int max_number_of_requests_per_second)
{
m_MaxNumberOfRequestsPerSecond = max_number_of_requests_per_second;
m_Stopwatch = Stopwatch.StartNew();
}
public async Task<TResult> Throttle<TResult>(Func<Task<TResult>> task)
{
var elapsed = m_Stopwatch.Elapsed;
if (elapsed > TimeSpan.FromSeconds(1))
{
m_NumberOfRequestsInLastSecond = 1;
m_Stopwatch.Restart();
return await task();
}
if (m_NumberOfRequestsInLastSecond >= m_MaxNumberOfRequestsPerSecond)
{
TimeSpan time_to_wait = TimeSpan.FromSeconds(1) - elapsed;
await Task.Delay(time_to_wait);
m_NumberOfRequestsInLastSecond = 1;
m_Stopwatch.Restart();
return await task();
}
m_NumberOfRequestsInLastSecond++;
return await task();
}
}
Here is how this code can be tested:
class Program
{
static void Main(string[] args)
{
DoIt();
Console.ReadLine();
}
static async Task DoIt()
{
Func<Task<int>> func = async () =>
{
await Task.Delay(100);
return 1;
};
Throttler throttler = new Throttler(3);
for (int i = 0; i < 10; i++)
{
var result = await throttler.Throttle(func);
Console.WriteLine(DateTime.Now);
}
}
}

You can use this as Generic
public TaskThrottle(int maxTasksToRunInParallel)
{
_semaphore = new SemaphoreSlim(maxTasksToRunInParallel);
}
public void TaskThrottler<T>(IEnumerable<Task<T>> tasks, int timeoutInMilliseconds, CancellationToken cancellationToken = default(CancellationToken)) where T : class
{
// Get Tasks as List
var taskList = tasks as IList<Task<T>> ?? tasks.ToList();
var postTasks = new List<Task<int>>();
// When the first task completed, it will flag
taskList.ForEach(x =>
{
postTasks.Add(x.ContinueWith(y => _semaphore.Release(), cancellationToken));
});
taskList.ForEach(x =>
{
// Wait for open slot
_semaphore.Wait(timeoutInMilliseconds, cancellationToken);
cancellationToken.ThrowIfCancellationRequested();
x.Start();
});
Task.WaitAll(taskList.ToArray(), cancellationToken);
}

Edit: this solution works but use it only if it is ok to process all request in serial (in one thread). Otherwise use solution accepted as answer.
Well, thanks to Best way in .NET to manage queue of tasks on a separate (single) thread
My question is almost duplicate except adding delay before execution, which is actually simple.
The main helper here is SemaphoreSlim class which allows to restrict degree of parallelism.
So, first create a semaphore:
// Semaphore allows run 1 thread concurrently.
private readonly SemaphoreSlim semaphore = new SemaphoreSlim(1, 1);
And, final version of throttle looks like
public async Task<TResult> Throttle<TResult>(Func<Task<TResult>> task)
{
await semaphore.WaitAsync();
try
{
await delaySource.Delay();
return await task();
}
finally
{
semaphore.Release();
}
}
Delay source is also pretty simple:
private class TaskDelaySource
{
private readonly int maxTasks;
private readonly TimeSpan inInterval;
private readonly Queue<long> ticks = new Queue<long>();
public TaskDelaySource(int maxTasks, TimeSpan inInterval)
{
this.maxTasks = maxTasks;
this.inInterval = inInterval;
}
public async Task Delay()
{
// We will measure time of last maxTasks tasks.
while (ticks.Count > maxTasks)
ticks.Dequeue();
if (ticks.Any())
{
var now = DateTime.UtcNow.Ticks;
var lastTick = ticks.First();
// Calculate interval between last maxTasks task and current time
var intervalSinceLastTask = TimeSpan.FromTicks(now - lastTick);
if (intervalSinceLastTask < inInterval)
await Task.Delay((int)(inInterval - intervalSinceLastTask).TotalMilliseconds);
}
ticks.Enqueue(DateTime.UtcNow.Ticks);
}
}