Develop Reference

Message is published from API #1, how do I create a listener in another API?

I've two APIs.
When one of the endpoints are called in API #1, message is sent to queue in Azure Service Bus.
API #2 should listen and make some changes in DB after this message appears in queue.
Message is sent to queue successfully.
Listener part doesn't work because the listener method is never called (and I do not understand how to call it).
Listener in API #2 :
public class MessageConsumer : IMessageConsumer
{
const string connectionString = "stringTakenFromAzure";
private static IQueueClient queueClient;
private CartingDbContext context;
public MessageConsumer(CartingDbContext context)
{
this.context = context;
}
public async Task Consume()
{
queueClient = new QueueClient(connectionString, "cartqueue");
var options = new MessageHandlerOptions(ExceptionReceivedHandler)
{
MaxConcurrentCalls = 1,
AutoComplete = false
};
queueClient.RegisterMessageHandler(ProcessMessageAsync, options);
await queueClient.CloseAsync();
}
private async Task ProcessMessageAsync(Microsoft.Azure.ServiceBus.Message message, CancellationToken cancellationToken)
{
var jsonBody = Encoding.UTF8.GetString(message.Body);
var categoryItem = JsonSerializer.Deserialize<CategoryItem>(jsonBody);
//update item in DB.
var categoryItemInDb = context.CategoryItems.Where(x => x.Id == categoryItem.Id).FirstOrDefault();
if (categoryItemInDb == null)
{
context.CategoryItems.Add(categoryItem);
}
else
{
context.CategoryItems.Update(categoryItem);
}
context.SaveChanges();
await queueClient.CompleteAsync(message.SystemProperties.LockToken);
}
private static Task ExceptionReceivedHandler(ExceptionReceivedEventArgs args)
{
return Task.CompletedTask;
}
}
Program.cs
builder.Services.AddTransient<IMessageConsumer, MessageConsumer>();

Connect with SingalR client with IHubContext provided in diffrent thread

I am using ASP.NET Core, and I am adding some users to a collection via SingalR hub endpoint:
public class MatchMakingHub : Hub
{
//....
// called by client
public async Task EnlistMatchMaking(int timeControlMs)
{
Guid currentId = Guid.Parse(this.Context.User.GetSubjectId());
GetPlayerByIdQuery getPlayerByIdQuery = new GetPlayerByIdQuery(currentId);
Player currentPlayer = await requestSender.Send<Player>(getPlayerByIdQuery);
var waitingPlayer = new WaitingPlayer(currentPlayer, timeControlMs);
this.matchMakePool.Add(waitingPlayer);
}
}
matchMakePool being a singleton collection.
Later, I have an ASP.NET Core background service fetch the users from the collection, and notify them about being fetched:
public class MatchMakingBackgroundService : BackgroundService
{
private readonly MatchMakePoolSingleton matchMakePoolSingleton;
private readonly IServiceProvider serviceProvider;
private const int RefreshTimeMs = 1000;
public MatchMakingBackgroundService(MatchMakePoolSingleton matchMakePoolSingleton, IServiceProvider serviceProvider)
{
this.matchMakePoolSingleton = matchMakePoolSingleton;
this.serviceProvider = serviceProvider;
}
protected override async Task ExecuteAsync(CancellationToken stoppingToken)
{
while(!stoppingToken.IsCancellationRequested)
{
var result = matchMakePoolSingleton.RefreshMatches();
var tasks = new List<Task>();
foreach(var match in result)
{
tasks.Add(StartGameAsync(match));
}
await Task.WhenAll(tasks);
await Task.Delay(RefreshTimeMs, stoppingToken);
}
}
private async Task StartGameAsync(MatchMakeResult match)
{
using var scope = serviceProvider.CreateScope();
var sender = scope.ServiceProvider.GetRequiredService<ISender>();
var hubContext = serviceProvider.GetRequiredService<IHubContext<MatchMakingHub>>();
CreateNewGameCommand newGameCommand = new CreateNewGameCommand(match.WhitePlayer.Id, match.BlackPlayer.Id, TimeSpan.FromMilliseconds(match.TimeControlMs));
Guid gameGuid = await sender.Send(newGameCommand);
await hubContext.Clients.User(match.WhitePlayer.Id.ToString()).SendAsync("NotifyGameFound", gameGuid);
await hubContext.Clients.User(match.BlackPlayer.Id.ToString()).SendAsync("NotifyGameFound", gameGuid);
}
}
My problem is that NotifyGameFound is not being called in the client side. When I notified them straight from the hub itself it was received, but for some reason it doesn't when I call it through the provided IHubContext<MatchMakingHub>. I suspect that this is because it runs on another thread.
Here is the client code:
// in blazor
protected override async Task OnInitializedAsync()
{
var tokenResult = await TokenProvider.RequestAccessToken();
if(tokenResult.TryGetToken(out var token))
{
hubConnection
= new HubConnectionBuilder().WithUrl(NavigationManager.ToAbsoluteUri("/hubs/MatchMaker"), options =>
{
options.AccessTokenProvider = () => Task.FromResult(token.Value);
}).Build();
await hubConnection.StartAsync();
hubConnection.On<Guid>("NotifyGameFound", id =>
{
//do stuff
});
await MatchMakeRequast();
}
}
async Task MatchMakeRequast() =>
await hubConnection.SendAsync("EnlistMatchMaking", Secs * 1000);

I use injection to achieve this.
In my servers Startup.cs ConfigureServices mothod I have:
services.AddScoped<INotificationsBroker, NotificationsBroker>();
In your case I am assuming you are injecting MatchMakingBackgroundService
Something like:
services.AddScoped<MatchMakingBackgroundService>();
In my NotificationsBroker constructor I inject the context:
private readonly IHubContext<NotificationsHub> hub;
public NotificationsBroker(IHubContext<NotificationsHub> hub)
=> this.hub = hub;
I then inject the broker into any service I require it and the service can call the hubs methods I expose via the interface.
You don't have to go the extra step, I do this for testing, you could inject the context directly into your MatchMakingBackgroundService.

gRPC keeping response streams open for subscriptions

I've tried to define a gRPC service where client can subscribe to receive broadcasted messages and they can also send them.
syntax = "proto3";
package Messenger;
service MessengerService {
rpc SubscribeForMessages(User) returns (stream Message) {}
rpc SendMessage(Message) returns (Close) {}
}
message User {
string displayName = 1;
}
message Message {
User from = 1;
string message = 2;
}
message Close {}
My idea was that when a client requests to subscribe to the messages, the response stream would be added to a collection of response streams, and when a message is sent, the message is sent through all the response streams.
However, when my server attempts to write to the response streams, I get an exception System.InvalidOperationException: 'Response stream has already been completed.'
Is there any way to tell the server to keep the streams open so that new messages can be sent through them? Or is this not something that gRPC was designed for and a different technology should be used?
The end goal service would be allows multiple types of subscriptions (could be to new messages, weather updates, etc...) through different clients written in different languages (C#, Java, etc...). The different languages part is mainly the reason I chose gRPC to try this, although I intend on writing the server in C#.
Implementation example
using System;
using System.Collections.Concurrent;
using System.Collections.Generic;
using System.Linq;
using System.Threading;
using System.Threading.Tasks;
using Grpc.Core;
using Messenger;
namespace SimpleGrpcTestStream
{
/*
Dependencies
Install-Package Google.Protobuf
Install-Package Grpc
Install-Package Grpc.Tools
Install-Package System.Interactive.Async
Install-Package System.Linq.Async
*/
internal static class Program
{
private static void Main()
{
var messengerServer = new MessengerServer();
messengerServer.Start();
var channel = Common.GetNewInsecureChannel();
var client = new MessengerService.MessengerServiceClient(channel);
var clientUser = Common.GetUser("Client");
var otherUser = Common.GetUser("Other");
var cancelClientSubscription = AddCancellableMessageSubscription(client, clientUser);
var cancelOtherSubscription = AddCancellableMessageSubscription(client, otherUser);
client.SendMessage(new Message { From = clientUser, Message_ = "Hello" });
client.SendMessage(new Message { From = otherUser, Message_ = "World" });
client.SendMessage(new Message { From = clientUser, Message_ = "Whoop" });
cancelClientSubscription.Cancel();
cancelOtherSubscription.Cancel();
channel.ShutdownAsync().Wait();
messengerServer.ShutDown().Wait();
}
private static CancellationTokenSource AddCancellableMessageSubscription(
MessengerService.MessengerServiceClient client,
User user)
{
var cancelMessageSubscription = new CancellationTokenSource();
var messages = client.SubscribeForMessages(user);
var messageSubscription = messages
.ResponseStream
.ToAsyncEnumerable()
.Finally(() => messages.Dispose());
messageSubscription.ForEachAsync(
message => Console.WriteLine($"New Message: {message.Message_}"),
cancelMessageSubscription.Token);
return cancelMessageSubscription;
}
}
public static class Common
{
private const int Port = 50051;
private const string Host = "localhost";
private static readonly string ChannelAddress = $"{Host}:{Port}";
public static User GetUser(string name) => new User { DisplayName = name };
public static readonly User ServerUser = GetUser("Server");
public static readonly Close EmptyClose = new Close();
public static Channel GetNewInsecureChannel() => new Channel(ChannelAddress, ChannelCredentials.Insecure);
public static ServerPort GetNewInsecureServerPort() => new ServerPort(Host, Port, ServerCredentials.Insecure);
}
public sealed class MessengerServer : MessengerService.MessengerServiceBase
{
private readonly Server _server;
public MessengerServer()
{
_server = new Server
{
Ports = { Common.GetNewInsecureServerPort() },
Services = { MessengerService.BindService(this) },
};
}
public void Start()
{
_server.Start();
}
public async Task ShutDown()
{
await _server.ShutdownAsync().ConfigureAwait(false);
}
private readonly ConcurrentDictionary<User, IServerStreamWriter<Message>> _messageSubscriptions = new ConcurrentDictionary<User, IServerStreamWriter<Message>>();
public override async Task<Close> SendMessage(Message request, ServerCallContext context)
{
await Task.Run(() =>
{
foreach (var (_, messageStream) in _messageSubscriptions)
{
messageStream.WriteAsync(request);
}
}).ConfigureAwait(false);
return await Task.FromResult(Common.EmptyClose).ConfigureAwait(false);
}
public override async Task SubscribeForMessages(User request, IServerStreamWriter<Message> responseStream, ServerCallContext context)
{
await Task.Run(() =>
{
responseStream.WriteAsync(new Message
{
From = Common.ServerUser,
Message_ = $"{request.DisplayName} is listening for messages!",
});
_messageSubscriptions.TryAdd(request, responseStream);
}).ConfigureAwait(false);
}
}
public static class AsyncStreamReaderExtensions
{
public static IAsyncEnumerable<T> ToAsyncEnumerable<T>(this IAsyncStreamReader<T> asyncStreamReader)
{
if (asyncStreamReader is null) { throw new ArgumentNullException(nameof(asyncStreamReader)); }
return new ToAsyncEnumerableEnumerable<T>(asyncStreamReader);
}
private sealed class ToAsyncEnumerableEnumerable<T> : IAsyncEnumerable<T>
{
public IAsyncEnumerator<T> GetAsyncEnumerator(CancellationToken cancellationToken = default)
=> new ToAsyncEnumerator<T>(_asyncStreamReader, cancellationToken);
private readonly IAsyncStreamReader<T> _asyncStreamReader;
public ToAsyncEnumerableEnumerable(IAsyncStreamReader<T> asyncStreamReader)
{
_asyncStreamReader = asyncStreamReader;
}
private sealed class ToAsyncEnumerator<TEnumerator> : IAsyncEnumerator<TEnumerator>
{
public TEnumerator Current => _asyncStreamReader.Current;
public async ValueTask<bool> MoveNextAsync() => await _asyncStreamReader.MoveNext(_cancellationToken);
public ValueTask DisposeAsync() => default;
private readonly IAsyncStreamReader<TEnumerator> _asyncStreamReader;
private readonly CancellationToken _cancellationToken;
public ToAsyncEnumerator(IAsyncStreamReader<TEnumerator> asyncStreamReader, CancellationToken cancellationToken)
{
_asyncStreamReader = asyncStreamReader;
_cancellationToken = cancellationToken;
}
}
}
}
}

The problem you're experiencing is due to the fact that MessengerServer.SubscribeForMessages returns immediately. Once that method returns, the stream is closed.
You'll need an implementation similar to this to keep the stream alive:
public class MessengerService : MessengerServiceBase
{
private static readonly ConcurrentDictionary<User, IServerStreamWriter<Message>> MessageSubscriptions =
new Dictionary<User, IServerStreamWriter<Message>>();
public override async Task SubscribeForMessages(User request, IServerStreamWriter<ReferralAssignment> responseStream, ServerCallContext context)
{
if (!MessageSubscriptions.TryAdd(request))
{
// User is already subscribed
return;
}
// Keep the stream open so we can continue writing new Messages as they are pushed
while (!context.CancellationToken.IsCancellationRequested)
{
// Avoid pegging CPU
await Task.Delay(100);
}
// Cancellation was requested, remove the stream from stream map
MessageSubscriptions.TryRemove(request);
}
}
As far as unsubscribing / cancellation goes, there are two possible approaches:
The client can hold onto a CancellationToken and call Cancel() when it wants to disconnect
The server can hold onto a CancellationToken which you would then store along with the IServerStreamWriter in the MessageSubscriptions dictionary via a Tuple or similar. Then, you could introduce an Unsubscribe method on the server which looks up the CancellationToken by User and calls Cancel on it server-side

Similar to Jon Halliday's answer, an indefinately long Task.Delay(-1) could be used and passed the context's cancellation token.
A try catch can be used to remove end the server's response stream when the task is cancelled.
public override async Task SubscribeForMessages(User request, IServerStreamWriter<Message> responseStream, ServerCallContext context)
{
if (_messageSubscriptions.ContainsKey(request))
{
return;
}
await responseStream.WriteAsync(new Message
{
From = Common.ServerUser,
Message_ = $"{request.DisplayName} is listening for messages!",
}).ConfigureAwait(false);
_messageSubscriptions.TryAdd(request, responseStream);
try
{
await Task.Delay(-1, context.CancellationToken);
}
catch (TaskCanceledException)
{
_messageSubscriptions.TryRemove(request, out _);
}
}

TPL BufferBlock Consume Method Not Being Called

I want to implement the consumer/producer pattern using the BufferBlock that runs continuously similar to the question here and the code here.
I tried to use an ActionBlock like the OP, but if the bufferblock is full and new messages are in it's queue then the new messages never get added to the ConcurrentDictionary _queue.
In the code below the ConsumeAsync method never gets called when a new message is added to the bufferblock with this call:_messageBufferBlock.SendAsync(message)
How can I correct the code below so that the ConsumeAsync method is called every time a new message is added using _messageBufferBlock.SendAsync(message)?
public class PriorityMessageQueue
{
private volatile ConcurrentDictionary<int,MyMessage> _queue = new ConcurrentDictionary<int,MyMessage>();
private volatile BufferBlock<MyMessage> _messageBufferBlock;
private readonly Task<bool> _initializingTask; // not used but allows for calling async method from constructor
private int _dictionaryKey;
public PriorityMessageQueue()
{
_initializingTask = Init();
}
public async Task<bool> EnqueueAsync(MyMessage message)
{
return await _messageBufferBlock.SendAsync(message);
}
private async Task<bool> ConsumeAsync()
{
try
{
// This code does not fire when a new message is added to the buffereblock
while (await _messageBufferBlock.OutputAvailableAsync())
{
// A message object is never received from the bufferblock
var message = await _messageBufferBlock.ReceiveAsync();
}
return true;
}
catch (Exception ex)
{
return false;
}
}
private async Task<bool> Init()
{
var executionDataflowBlockOptions = new ExecutionDataflowBlockOptions
{
MaxDegreeOfParallelism = Environment.ProcessorCount,
BoundedCapacity = 50
};
var prioritizeMessageBlock = new ActionBlock<MyMessage>(msg =>
{
SetMessagePriority(msg);
}, executionDataflowBlockOptions);
_messageBufferBlock = new BufferBlock<MyMessage>();
_messageBufferBlock.LinkTo(prioritizeMessageBlock, new DataflowLinkOptions { PropagateCompletion = true, MaxMessages = 50});
return await ConsumeAsync();
}
}
EDIT
I have removed all the extra code and added comments.

I'm still not completely certain what you're trying to accomplish but I'll try to point you in the right direction. Most of the code in the example isn't strictly necessary.
I need to know when a new message arrives
If this is your only requirement then I'll assume you just need to run some arbitrary code whenever a new message is passed in. The easiest way to do that in dataflow is to use a TransformBlock and set that block as the initial receiver in your pipeline. Each block has it's own buffer so unless you have need for another buffer you can leave it out.
public class PriorityMessageQueue {
private TransformBlock<MyMessage, MyMessage> _messageReciever;
public PriorityMessageQueue() {
var executionDataflowBlockOptions = new ExecutionDataflowBlockOptions {
MaxDegreeOfParallelism = Environment.ProcessorCount,
BoundedCapacity = 50
};
var prioritizeMessageBlock = new ActionBlock<MyMessage>(msg => {
SetMessagePriority(msg);
}, executionDataflowBlockOptions);
_messageReciever = new TransformBlock<MyMessage, MyMessage>(msg => NewMessageRecieved(msg), executionDataflowBlockOptions);
_messageReciever.LinkTo(prioritizeMessageBlock, new DataflowLinkOptions { PropagateCompletion = true });
}
public async Task<bool> EnqueueAsync(MyMessage message) {
return await _messageReciever.SendAsync(message);
}
private MyMessage NewMessageRecieved(MyMessage message) {
//do something when a new message arrives
//pass the message along in the pipeline
return message;
}
private void SetMessagePriority(MyMessage message) {
//Handle a message
}
}
Of course the other option you have would be to do whatever it is you need to immediately within EnqueAsync before returning the task from SendAsync but the TransformBlock gives you extra flexibility.

Capture all data posted by client to server

I am developing an application in C# using HttpClient. My code is flowing through a lot of functions and then finally it does a PostAsyc.
What I want to do is that I want to have a EventHandler which is called when PostAsyc is done. and in that event handler I want to capture and print everything which the client has sent to the server .
Is this possible in .NET HTTPClient?

public void PostData(string data, Action<string> callback)
{
var client = new HttpClient();
var task = client.PostAsync("uri", new StringContent(data));
task.ContinueWith((t) =>
{
t.Result.Content.ReadAsStringAsync().ContinueWith((trep) =>
{
string response = trep.Result;
callback(response);
});
});
}
Instead of using Action<string> callback you can define an event delegate and use that also, this gives you more flexibility of attaching multiple receivers.
public class PostEventArgs : EventArgs { public string Data { get; set; } }
public event EventHandler<PostEventArgs> postDone;
public void PostData(string data)
{
var client = new HttpClient();
var task = client.PostAsync("uri", new StringContent(data));
task.ContinueWith((t) =>
{
t.Result.Content.ReadAsStringAsync().ContinueWith((trep) =>
{
string response = trep.Result;
if (postDone != null)
postDone(this, new PostEventArgs() { Data = response });
});
});
}
Usage:
First Case
serviceObj.PostData("some data", (response)=> { Console.WriteLine(response); });
Second case
serviceObj.postDone += (obj,response)=>{ Console.WriteLine(response); }; // register only once
serviceObj.PostData("some data");
Updated with Task.ContinueWith.

Create a message handler class like this,
public class LoggingMessageHandler : DelegatingHandler
{
protected override async Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
{
var response = await base.SendAsync(request, cancellationToken);
if (request.Method == HttpMethod.Post)
{
// Log whatever you want here
Console.WriteLine(request.ToString());
Console.WriteLine(response.ToString());
}
return response;
}
}
and then create your HttpClient with the message handler as part of the request/response pipeline
var client = new HttpClient(new LoggingMessageHandler() {InnerHandler = new HttpClientHandler()});
client.PostAsync(...) // Whatever
Any request you make from this point on will pass through the LoggingRequestHandler.
By taking this approach you do not need to wrap the HttpClient object and it is also easy to retrofit into existing code.