Develop Reference

Azure Event Hubs : QuotaExceededException : 4999 handles limit

I am using Azure Event Hub library for C# in WebApi App. And I got this exception on message sending:
"Message":"An error has occurred.",
"ExceptionMessage":"Cannot allocate more handles to the current session or connection. The maximum number of handles allowed is 4999. Please free up resources and try again., referenceId: d975f39c71a14ae5915c9adca322e110_G15"
"ExceptionType":"Microsoft.ServiceBus.Messaging.QuotaExceededException"
I thought that instantiantion of EventHubProducerClient one time and reusing it instead of creating its instance on each message sending (with IAsyncDisposable pattern) will help as mentioned here
EventHub Exception :Cannot allocate more handles to the current session or connection. But it did not.
I believe there might be some more global issue. Might be missing something.
I am using single event hub with 7 consumer groups each of which is used by separate web application (single); well actually there is additional consumer group ($Default) but it is not used;
For receiving messages I use EventProcessorClient;
I use Azure.Messaging.EventHubs 5.2.0 and Azure.Messaging.EventHubs.Processor 5.2.0 packages;
Here is the whole code (did everything according to quickstart):
public class EventHubService : SubscriberBase
{
private readonly Action<string> errorHandler;
private readonly BlobContainerClient blobContainerClient;
private readonly EventProcessorClient eventProcessorClient;
private readonly EventHubProducerClient eventHubProducerClient;
private readonly int eventsToCheckpoint;
private readonly Timer checkpointTimer;
private int eventsSinceLastCheckpoint;
private bool shouldUpdateCheckpoint;
public EventHubService(EventHubSettings settings, Action<string> errorHandler) : base()
{
this.errorHandler = errorHandler;
eventHubProducerClient = new EventHubProducerClient(settings.ConnectionString, settings.EventHubName);
if (!String.IsNullOrWhiteSpace(settings.GroupId))
{
eventManager = new EventManager();
blobContainerClient = new BlobContainerClient(settings.StorageConnectionString, settings.BlobContainerName);
eventProcessorClient = new EventProcessorClient(blobContainerClient, settings.GroupId, settings.ConnectionString, settings.EventHubName);
eventsToCheckpoint = settings.EventsToUpdateCheckpoint;
checkpointTimer = new Timer(settings.IntervalToUpdateCheckpoint.TotalMilliseconds);
checkpointTimer.Elapsed += (sender, eventArgs) => shouldUpdateCheckpoint = true;
}
}
public override void Start()
{
eventProcessorClient.ProcessErrorAsync += HandleError;
eventProcessorClient.ProcessEventAsync += ProcessEventData;
eventProcessorClient.StartProcessingAsync().Wait();
checkpointTimer.Start();
}
public override async Task Stop()
{
try
{
checkpointTimer.Stop();
await eventProcessorClient.StopProcessingAsync();
}
finally
{
eventProcessorClient.ProcessEventAsync -= ProcessEventData;
eventProcessorClient.ProcessErrorAsync -= HandleError;
}
}
public override async Task Publish(string topic, JObject message)
{
using (EventDataBatch eventBatch = await eventHubProducerClient.CreateBatchAsync())
{
var #event = new Event(topic, message);
string json = #event.ToString(Formatting.None);
byte[] bytes = Encoding.UTF8.GetBytes(json);
var eventData = new EventData(bytes);
eventBatch.TryAdd(eventData);
await eventHubProducerClient.SendAsync(eventBatch);
}
}
private async Task ProcessEventData(ProcessEventArgs eventArgs)
{
if (eventArgs.CancellationToken.IsCancellationRequested)
{
return;
}
if (++eventsSinceLastCheckpoint >= eventsToCheckpoint)
{
eventsSinceLastCheckpoint = 0;
shouldUpdateCheckpoint = true;
}
if (shouldUpdateCheckpoint)
{
await eventArgs.UpdateCheckpointAsync();
shouldUpdateCheckpoint = false;
}
string json = Encoding.UTF8.GetString(eventArgs.Data.Body.ToArray());
var #event = new Event(json);
eventManager.TryRaise(#event);
}
private Task HandleError(ProcessErrorEventArgs eventArgs)
{
if (!eventArgs.CancellationToken.IsCancellationRequested)
{
errorHandler.Invoke($"[P:{eventArgs.PartitionId}][O:{eventArgs.Operation}] {eventArgs.Exception.Message}");
}
return Task.CompletedTask;
}
}
I have found some info in Service Bus Quotas like:
Number of concurrent receive requests on a queue, topic, or subscription entity (5000).
Subsequent receive requests are rejected, and an exception is received
by the calling code. This quota applies to the combined number
of concurrent receive operations across all subscriptions on a topic.
But still can't figure how to deal with it.
Please help.
Thanks.

This is indeed the answer EventHub Exception :Cannot allocate more handles to the current session or connection.
I did similar "fix" for Azure Event Hub library for NET Core but I have forgotten that I am also using Azure Event Hub library for NET Framework!
So I have instantiated EventHubProducerClient one time and reusing it now.
Seems working fine.
My bad. Was not attentive enough.

In my case, except creating only one instance of Client, use only one instance of sender.
I used method CreateSender each time when send a messsage, it also generates an exception

Discord bot running, but will not connect to server

I've been trying to swap over my code from the 0.9.6 Discord.NET API to the new 1.0.1 API, and it's basically calling for a complete restructure to my code. But I've been having some trouble actually getting the bot up and running first of all.
I set up the code body according to the guide linked here
And while it runs without error, the bot itself is not appearing online in my server.
And before you ask, I had in fact replaced "Bot token here" with the actual bot token.
namespace DiscordBot{
public class Program
{
private CommandService commands;
private DiscordSocketClient client;
private IServiceProvider services;
static void Main(string[] args) => new Program().Start().GetAwaiter().GetResult();
public async Task Start()
{
client = new DiscordSocketClient();
commands = new CommandService();
string token = "<token>";
services = new ServiceCollection()
.BuildServiceProvider();
await InstallCommands();
await client.LoginAsync(TokenType.Bot, token);
await client.StartAsync();
await Task.Delay(-1);
}
public async Task InstallCommands()
{
// Hook the MessageReceived Event into our Command Handler
client.MessageReceived += HandleCommand;
// Discover all of the commands in this assembly and load them.
await commands.AddModulesAsync(Assembly.GetEntryAssembly());
}
public async Task HandleCommand(SocketMessage messageParam)
{
// Don't process the command if it was a System Message
var message = messageParam as SocketUserMessage;
if (message == null) return;
// Create a number to track where the prefix ends and the command begins
int argPos = 0;
// Determine if the message is a command, based on if it starts with '!' or a mention prefix
if (!(message.HasCharPrefix('!', ref argPos) || message.HasMentionPrefix(client.CurrentUser, ref argPos))) return;
// Create a Command Context
var context = new CommandContext(client, message);
// Execute the command. (result does not indicate a return value,
// rather an object stating if the command executed successfully)
var result = await commands.ExecuteAsync(context, argPos, services);
if (!result.IsSuccess)
await context.Channel.SendMessageAsync(result.ErrorReason);
}
}
}
And then for the MyBot.cs class
namespace DiscordBot
{
class MyBot : ModuleBase
{
private CommandService _service;
public MyBot(CommandService service)
{
_service = service;
}
}
}

The first thing you might want to do is add some logging to your bot.
As your code might be correct, but discord could be rejecting your connection for any amount of reason.
After await client.StartAsync(); add
client.Log += (msg) => {return Console.WriteLine(${msg.ToString()}");};`
This will output the message your receive from your client to the console.
Now you also need to configure which message should be send to this event. This can be done when creating your DiscordClient(). So instead of client = new DiscordSocketClient(); You could use
client = new DiscordSocketClient(
new DiscordSocketConfig()
{
LogLevel = LogSeverity.Verbose
}
);
Verbose should give you all the information you need. However you could also use LogSeverity.Debug instead, which is the highest available logging, and therefore would give you all messages.
Now that you have some feedback in your console, go and see what concrete errors you are having.
Also I would recommend first completing the your first bot part of the linked tutorial, instead of stepping into the commands directly. Once you got this working, you can continue onwards

Using JustSaying SNS and SQS all hookup in AWS but Handler not getting invoked

Using JustSaying 4.0.306 and configuring the publisher like so:
_publisher = CreateMeABus
.InRegion("eu-west-2").WithNamingStrategy(()=>new Naming<T>())
.WithSnsMessagePublisher<MessageHolder<T>>();
I'm overriding the naming strategy as when using generics is creates a very verbose name.
public class Naming<T> : INamingStrategy
{
public string GetTopicName(string topicName, string messageType)
{
return "topic_" + typeof(T).Name.ToLower();
}
public string GetQueueName(SqsReadConfiguration sqsConfig, string messageType)
{
return "queue_" + typeof(T).Name.ToLower();
}
}
I'm configuring the subscription end like so:
var queuename = "queue" + typeof(T).Name.ToLower();
var bus = CreateMeABus.InRegion("eu-west-2")
.WithNamingStrategy(() => new NamingStrategy<T>())
.WithSqsTopicSubscriber().IntoQueue(queuename)
.WithMessageHandler<MessageHolder<T>>(new HandleMessage<T>(_handler));
bus.StartListening();
In AWS the SNS Topic gets created along with the SQS queue's and all the subscriptions etc. When publishing I can see messages flowing into the queue and they are being read from the queue but the HandleMessage<T>.Handle(MessageHolder<T> message) never gets invoked.
I can see that the messages are taken from the queue and sometimes they are in flight so something in the JustSaying world is taking them out of the queue but it never gets to the handler.
Anyone failure with this framework have any idea why is there something I have missed or could it be serialization is failing before the handler is getting invoked?
g/

Synchronizing asynchronous events

I have a class that receives standard .Net events from an external class.
These events have an address property (in addition to a lot of other properties, of course) that I can use to synchronize my events, so that I should be able to create a method to Get something, wait for the correct event, then return the data from the event in the Get method.
However, I'm fairly new to synchronization in C# and was hoping any of you could help me out. Below is somewhat pseudo code for what I want to accomplish:
Someone calls DoAsynchronousToSynchronousCall
That method waits until an event have been received with the same address (or until it times out)
The event checks against all current requests. If it finds a request with the same address, let DoAsynchronousToSynchronousCall know the reply has arrived
DoAsynchronousCall gets (or retrieves) the reply and returns it to the caller
public class MyMessage
{
public string Address { get; set; }
public string Data { get; set; }
}
public Main
{
externalClass.MessageReceived += MessageReceived;
}
public void MessageReceived(MyMessage message)
{
MyMessage request = _requestQueue.FirstOrDefault(m => m.Address = message.Address);
if (request != null)
{
// Do something to let DoAsynchronousToSynchronousCall() know the reply has arrived
}
}
private List<MyMessage> _requestQueue = new List<MyMessage>();
public MyMessage DoAsynchronousToSynchronousCall(MyMessage message)
{
_requestQueue.Add(message);
externalClass.Send(message);
// Do something to wait for a reply (as checked for above)
MyMessage reply = WaitForCorrectReply(timeout: 10000);
return reply;
}
I feel like I'm missing an opportunity to use async and await (yet I don't know how), and I hope you're able to understand what I'm trying to accomplish based on the information above.

You really can't have multiple calls on the fly and have synchronous responses. If you want synchronous responses for multiple calls then you need to do the calls synchronously too.
I would look at using Microsoft's Reactive Extensions (NuGet "Rx-Main") to make what you're doing as simple as possible. Rx lets you turn events into streams of values that you can query against.
Here's what I would do.
I would first define a stream of the received messages as IObservable<MyMessage> receivedMessages like this:
receivedMessages =
Observable
.FromEvent<MessageReceivedHandler, MyMessage>(
h => externalClass.MessageReceived += h,
h => externalClass.MessageReceived -= h);
(You didn't provide a class def so I've called the event delegate MessageReceivedHandler.)
Now you can redefine DoAsynchronousToSynchronousCall as:
public IObservable<MyMessage> DoAsynchronousCall(MyMessage message)
{
return Observable.Create<MyMessage>(o =>
{
IObservable<MyMessage> result =
receivedMessages
.Where(m => m.Address == message.Address)
.Take(1);
IObservable<MyMessage> timeout =
Observable
.Timer(TimeSpan.FromSeconds(10.0))
.Select(x => (MyMessage)null);
IDisposable subscription =
Observable
.Amb(result, timeout)
.Subscribe(o);
externalClass.Send(message);
return subscription;
});
}
The result observable is the receivedMessages filtered for the current message.Address.
The timeout observable is a default value to return if the call takes longer than TimeSpan.FromSeconds(10.0) to complete.
Finally the subscription uses Observable.Amb(...) to determine which of result or timeout produces a value first and subscribes to that result.
So now to call this you can do this:
DoAsynchronousCall(new MyMessage() { Address = "Foo", Data = "Bar" })
.Subscribe(response => Console.WriteLine(response.Data));
So, if I make a simple definition of ExternalClass like this:
public class ExternalClass
{
public event MessageReceivedHandler MessageReceived;
public void Send(MyMessage message)
{
this.MessageReceived(new MyMessage()
{
Address = message.Address,
Data = message.Data + "!"
});
}
}
...I get the result Bar! printed on the console.
If you have a whole bunch of messages that you want to process you can do this:
var messagesToSend = new List<MyMessage>();
/* populate `messagesToSend` */
var query =
from message in messagesToSend.ToObservable()
from response in DoAsynchronousCall(message)
select new
{
message,
response
};
query
.Subscribe(x =>
{
/* Do something with each correctly paired
`x.message` & `x.response`
*/
});

You're probably looking for ManualResetEvent which functions as a "toggle" of sorts to switch between thread-blocking and non-blocking behavior. The DoAsynchronousToSynchronousCall would Reset and then WaitOne(int timeoutMilliseconds) the event to block the thread, and the thing checking for the correct reply arrived would do the Set call to let the thread continue on its way if the correct thing arrived.

Access objects within a Consumer

I am working on a project that consumes messages using MassTransit and RabbitMQ in C#
I'm building a prototype and right now the consumer project is a console application. Within the Main program.cs class, I connect to the bus and subscribe to accept messages from the publisher like this:
var bus = BusInitializer.CreateBus("Subscriber", x =>
{
x.Subscribe(subs =>
{
subs.Consumer<UpdatedInformation>().Permanent();
});
});
For reference, here's the CreateBus() method:
public static IServiceBus CreateBus(string queueName, Action<ServiceBusConfigurator> moreInitialization)
{
var bus = ServiceBusFactory.New(x =>
{
x.UseRabbitMq();
x.ReceiveFrom("rabbitmq://localhost/test" + queueName);
moreInitialization(x);
});
return bus;
}
And here is an example of one of the consumer classes:
class UpdatedInformationConsumer : Consumes<UpdatedInformation>.Context
{
public void Consume(IConsumeContext<UpdatedInformation> message)
{
Console.Write("\nMessage: " + message.Message);
Console.Write("\nPROCESSED: " + DateTime.Now.ToString());
}
}
In the main class of the subscriber, I also initialize services and other configurations. What I need to do is be able to access those services/objects in my consumer classes as I don't want to initialize new ones every time a message is received in consumed.
This should be relatively easy, but I'm stumped since I'm not actually creating instances of my consumer classes.

Rather than put a bunch of singletons in your service, which leads to a bunch of static methods and so forth, you can specify a consumer factory with your subscription that passes those dependencies to your consumer. You can also use any of the supported containers (or even your own) to resolve those dependencies.
x.Subscribe<UpdatedInformationConsumer>(() =>
new UpdatedInformationConsumer(dependency1, dependency2, etc));
A new instance of the consumer will be created for each message receiving, so object lifecycles can also be managed in this way.