Azure Event Hubs : QuotaExceededException : 4999 handles limit - c#

I am using Azure Event Hub library for C# in WebApi App. And I got this exception on message sending:
"Message":"An error has occurred.",
"ExceptionMessage":"Cannot allocate more handles to the current session or connection. The maximum number of handles allowed is 4999. Please free up resources and try again., referenceId: d975f39c71a14ae5915c9adca322e110_G15"
"ExceptionType":"Microsoft.ServiceBus.Messaging.QuotaExceededException"
I thought that instantiantion of EventHubProducerClient one time and reusing it instead of creating its instance on each message sending (with IAsyncDisposable pattern) will help as mentioned here
EventHub Exception :Cannot allocate more handles to the current session or connection. But it did not.
I believe there might be some more global issue. Might be missing something.
I am using single event hub with 7 consumer groups each of which is used by separate web application (single); well actually there is additional consumer group ($Default) but it is not used;
For receiving messages I use EventProcessorClient;
I use Azure.Messaging.EventHubs 5.2.0 and Azure.Messaging.EventHubs.Processor 5.2.0 packages;
Here is the whole code (did everything according to quickstart):
public class EventHubService : SubscriberBase
{
private readonly Action<string> errorHandler;
private readonly BlobContainerClient blobContainerClient;
private readonly EventProcessorClient eventProcessorClient;
private readonly EventHubProducerClient eventHubProducerClient;
private readonly int eventsToCheckpoint;
private readonly Timer checkpointTimer;
private int eventsSinceLastCheckpoint;
private bool shouldUpdateCheckpoint;
public EventHubService(EventHubSettings settings, Action<string> errorHandler) : base()
{
this.errorHandler = errorHandler;
eventHubProducerClient = new EventHubProducerClient(settings.ConnectionString, settings.EventHubName);
if (!String.IsNullOrWhiteSpace(settings.GroupId))
{
eventManager = new EventManager();
blobContainerClient = new BlobContainerClient(settings.StorageConnectionString, settings.BlobContainerName);
eventProcessorClient = new EventProcessorClient(blobContainerClient, settings.GroupId, settings.ConnectionString, settings.EventHubName);
eventsToCheckpoint = settings.EventsToUpdateCheckpoint;
checkpointTimer = new Timer(settings.IntervalToUpdateCheckpoint.TotalMilliseconds);
checkpointTimer.Elapsed += (sender, eventArgs) => shouldUpdateCheckpoint = true;
}
}
public override void Start()
{
eventProcessorClient.ProcessErrorAsync += HandleError;
eventProcessorClient.ProcessEventAsync += ProcessEventData;
eventProcessorClient.StartProcessingAsync().Wait();
checkpointTimer.Start();
}
public override async Task Stop()
{
try
{
checkpointTimer.Stop();
await eventProcessorClient.StopProcessingAsync();
}
finally
{
eventProcessorClient.ProcessEventAsync -= ProcessEventData;
eventProcessorClient.ProcessErrorAsync -= HandleError;
}
}
public override async Task Publish(string topic, JObject message)
{
using (EventDataBatch eventBatch = await eventHubProducerClient.CreateBatchAsync())
{
var #event = new Event(topic, message);
string json = #event.ToString(Formatting.None);
byte[] bytes = Encoding.UTF8.GetBytes(json);
var eventData = new EventData(bytes);
eventBatch.TryAdd(eventData);
await eventHubProducerClient.SendAsync(eventBatch);
}
}
private async Task ProcessEventData(ProcessEventArgs eventArgs)
{
if (eventArgs.CancellationToken.IsCancellationRequested)
{
return;
}
if (++eventsSinceLastCheckpoint >= eventsToCheckpoint)
{
eventsSinceLastCheckpoint = 0;
shouldUpdateCheckpoint = true;
}
if (shouldUpdateCheckpoint)
{
await eventArgs.UpdateCheckpointAsync();
shouldUpdateCheckpoint = false;
}
string json = Encoding.UTF8.GetString(eventArgs.Data.Body.ToArray());
var #event = new Event(json);
eventManager.TryRaise(#event);
}
private Task HandleError(ProcessErrorEventArgs eventArgs)
{
if (!eventArgs.CancellationToken.IsCancellationRequested)
{
errorHandler.Invoke($"[P:{eventArgs.PartitionId}][O:{eventArgs.Operation}] {eventArgs.Exception.Message}");
}
return Task.CompletedTask;
}
}
I have found some info in Service Bus Quotas like:
Number of concurrent receive requests on a queue, topic, or subscription entity (5000).
Subsequent receive requests are rejected, and an exception is received
by the calling code. This quota applies to the combined number
of concurrent receive operations across all subscriptions on a topic.
But still can't figure how to deal with it.
Please help.
Thanks.

This is indeed the answer EventHub Exception :Cannot allocate more handles to the current session or connection.
I did similar "fix" for Azure Event Hub library for NET Core but I have forgotten that I am also using Azure Event Hub library for NET Framework!
So I have instantiated EventHubProducerClient one time and reusing it now.
Seems working fine.
My bad. Was not attentive enough.

In my case, except creating only one instance of Client, use only one instance of sender.
I used method CreateSender each time when send a messsage, it also generates an exception

Related

What is tripping me up on SSE in C#?

I have successfully implemented the trivial example documented here, which demonstrates how to transmit server-sent events to a client, but I'm having difficulty going beyond the example.
The server I wrote (IIS/ASP.Net/C#/Web API 2) can handle multiple instances of a game being played at once. Each game has a unique handle.
The client provides the UI for the game. It makes an API request to start a game and is given a handle in the response. It then calls the Subscribe method, passing the handle. As in the example, it uses PushStreamContent to do so. My implementation looks like this;
private Game mGame;
private static readonly ConcurrentBag<Tuple<Game, StreamWriter>> mscolListenerStreams;
// GET: api/v1.0/subscription/{gameId:int}
[HttpGet]
[Route("{gameId:int}")]
public HttpResponseMessage Subscribe(int gameId) {
mGame = Game.GetGame(gameId);
if (mGame is null) {
return this.Request.CreateErrorResponse(HttpStatusCode.NotFound, "Unknown GameId");
}
mGame.GameEvent += game_Event; // Assign an event handler to follow game events.
HttpResponseMessage response = this.Request.CreateResponse(HttpStatusCode.OK);
response.Content = new PushStreamContent(
(a, b, c) => { this.OnStreamAvailable(a, b, c); },
"text/event-stream");
return response;
}
Instead of just saving off the stream as in the example, I need to tie a stream to a game. I do that like this:
private void OnStreamAvailable(Stream stream, HttpContent content, TransportContext context) {
mscolListenerStreams.Add(new Tuple<Game, StreamWriter>(mGame, new StreamWriter(stream)));
}
Instead of the timer event in the example, I use the game event handler to send messages to the relevant streams:
private async static void game_Event(Game game, EventId eventId, int index) {
foreach (Tuple<Game, StreamWriter> tuple in mscolListenerStreams) {
if (!object.ReferenceEquals(tuple.Item1, game)){
continue;
}
try {
await tuple.Item2.WriteAsync($"event:Game\ndata:{game.Id}/{eventId}/{index}\n\n");
await tuple.Item2.FlushAsync();
}
catch {
Tuple<Game, StreamWriter> ignore;
if (mscolListenerStreams.TryTake(out ignore)) {
ignore.Item2.Dispose();
}
}
}
}
As soon as the game fires an event (instead of the timer that's in the example), the clients locks up and stop receiving server messages or they get CORS errors whenever they try to make subsequent API calls. (The client domain is allowed.) The timer still works.
What am I missing here?

Discord bot running, but will not connect to server

I've been trying to swap over my code from the 0.9.6 Discord.NET API to the new 1.0.1 API, and it's basically calling for a complete restructure to my code. But I've been having some trouble actually getting the bot up and running first of all.
I set up the code body according to the guide linked here
And while it runs without error, the bot itself is not appearing online in my server.
And before you ask, I had in fact replaced "Bot token here" with the actual bot token.
namespace DiscordBot{
public class Program
{
private CommandService commands;
private DiscordSocketClient client;
private IServiceProvider services;
static void Main(string[] args) => new Program().Start().GetAwaiter().GetResult();
public async Task Start()
{
client = new DiscordSocketClient();
commands = new CommandService();
string token = "<token>";
services = new ServiceCollection()
.BuildServiceProvider();
await InstallCommands();
await client.LoginAsync(TokenType.Bot, token);
await client.StartAsync();
await Task.Delay(-1);
}
public async Task InstallCommands()
{
// Hook the MessageReceived Event into our Command Handler
client.MessageReceived += HandleCommand;
// Discover all of the commands in this assembly and load them.
await commands.AddModulesAsync(Assembly.GetEntryAssembly());
}
public async Task HandleCommand(SocketMessage messageParam)
{
// Don't process the command if it was a System Message
var message = messageParam as SocketUserMessage;
if (message == null) return;
// Create a number to track where the prefix ends and the command begins
int argPos = 0;
// Determine if the message is a command, based on if it starts with '!' or a mention prefix
if (!(message.HasCharPrefix('!', ref argPos) || message.HasMentionPrefix(client.CurrentUser, ref argPos))) return;
// Create a Command Context
var context = new CommandContext(client, message);
// Execute the command. (result does not indicate a return value,
// rather an object stating if the command executed successfully)
var result = await commands.ExecuteAsync(context, argPos, services);
if (!result.IsSuccess)
await context.Channel.SendMessageAsync(result.ErrorReason);
}
}
}
And then for the MyBot.cs class
namespace DiscordBot
{
class MyBot : ModuleBase
{
private CommandService _service;
public MyBot(CommandService service)
{
_service = service;
}
}
}
The first thing you might want to do is add some logging to your bot.
As your code might be correct, but discord could be rejecting your connection for any amount of reason.
After await client.StartAsync(); add
client.Log += (msg) => {return Console.WriteLine(${msg.ToString()}");};`
This will output the message your receive from your client to the console.
Now you also need to configure which message should be send to this event. This can be done when creating your DiscordClient(). So instead of client = new DiscordSocketClient(); You could use
client = new DiscordSocketClient(
new DiscordSocketConfig()
{
LogLevel = LogSeverity.Verbose
}
);
Verbose should give you all the information you need. However you could also use LogSeverity.Debug instead, which is the highest available logging, and therefore would give you all messages.
Now that you have some feedback in your console, go and see what concrete errors you are having.
Also I would recommend first completing the your first bot part of the linked tutorial, instead of stepping into the commands directly. Once you got this working, you can continue onwards

Rebus send from Bus2 in Bus1's messagehandler

I have to Rebus buses that use Azure ServiceBus, but when I try to send to from Bus2 in one of Bus1's messagehandlers this does not work. The message is not sent.
Any thoughts ?
EDIT
Bus 1
string padesQueueAddress = "padesworker";
int numberOfWorkes = Settings.NumberOfWorkers>0 Settings.NumberOfWorkers:10;
string errorQueueAddress = string.Format("{0}-error", queueAddress);
var adapter = new AutofacContainerAdapter(Container);
Bus = Configure.With(adapter)
.Logging(l => l.Use(UseRaygunRebusLoggingFactory(rayclient,Settings.Debug ? RaygunLoggerLevel.DEBUG : RaygunLoggerLevel.WARN)))
.Transport(t => t.UseAzureServiceBus( Settings.AzureQueueConnectionString, queueAddress,AzureServiceBusMode.Standard))
.Sagas(s => s.StoreInSqlServer(string.IsNullOrWhiteSpace(Settings.RebusSagaSqlConnectionString) ? Settings.AzureSqlConnectionString : Settings.RebusSagaSqlConnectionString, "Saga", "SagaIndex") )
.Routing(r => r.TypeBased().MapAssemblyOf<SendSmsCommand>(queueAddress).MapAssemblyOf<Unipluss.Sign.Pades.Commands.CreatePadesCommand>(padesQueueAddress))
.Options(o =>
{
o.SimpleRetryStrategy(secondLevelRetriesEnabled: true, maxDeliveryAttempts:5,errorQueueAddress: errorQueueAddress);
o.SetNumberOfWorkers(numberOfWorkes);
o.SetMaxParallelism(numberOfWorkes);
})
.Start();
await Bus.SendLocal(new HeartBeatCommand());
Bus 2
private IBus CreateExternalEventBus()
{
var eventBus = Configure.With(new BuiltinHandlerActivator())
.Transport(t => t.UseAzureServiceBus(Settings.EventServiceBusConnectionString, queueAddress+"_event", AzureServiceBusMode.Basic))
.Logging(x=>x.ColoredConsole(LogLevel.Debug))
.Options(o =>
{
o.LogPipeline(true);
o.EnableCompression();
o.EnableEncryption(Settings.RebusEncryptionExternalEvents);
})
.Start();
eventBus.Advanced.Routing.Send("1dd0f6f9422146048516a30f00aef4e5",new Unipluss.Sign.Events.Entities.DocumentCancledEvent() {CancledMessage = "test",DocumentId = Guid.NewGuid()});
return eventBus;
}
The eventBus send with the hardcoded send is working, but when this i sent from within one of Bus1's message handler the message is not sent (With logging on the logger says that the message is sent but it does appear in the queue).
Bus 2 is wrapped in a wrapper class an then injected in to Autofac to avoid having to IBus interfaces in Autofac.
builder.Register(c => new ExternalEventsBus(CreateExternalEventBus()))
.As<IExternalEventsBus>().SingleInstance();
public class ExternalEventsBus:IExternalEventsBus
{
private IBus Bus;
public ExternalEventsBus(IBus bus)
{
Bus = bus;
}
public async Task Send(object message, Guid documentProviderId)
{
await Bus.Advanced.Routing.Send(documentProviderId.ToString("n"), message);
}
public Task Send(object message, DocumentProvider documentProvider)
{
if (!string.IsNullOrWhiteSpace(documentProvider.RebusQueueConnectionString))
return Send(message, documentProvider.Id);
return Task.FromResult(true);
}
public void Dispose()
{
if(Bus!=null)
Bus.Dispose();
}
}
IExternalEventsBus is then used in multiple messagehandlers in Bus1.
Ok... let me see if I understand this (please correct me if I am wrong):
You have two bus instances in your process:
One that uses an input queue with the value specified by queueAddress, which you did not include here.
Another one (the "external event bus") that has an input queue with the value specified by queueAddress+"_event", which you also did not include here.
It seems like the purpose of your bus instances is that the first one is used to coordinate stuff within the application, whereas the second one is used to route events to listeners on the outside - this is what you would call a "content-based router", since it will route messages depending on some value of the content of the message (in this case documentProviderId).
Now you are experiencing an error: When bus (1) uses bus (2) from within one of its own message handlers, the routed message does not seem to be sent.
It is not clear from the code you posted from ExternalEventsBus which Send method you are calling - but I can tell you that the method with the signature public Task Send(object message, DocumentProvider documentProvider) will only send a message if documentProvider.RebusQueueConnectionString is not null.
Did you verify that the connection string does in fact carry a value?
Why is there a _connectionString field in ExternalEventsBus? Should you have used that when creating the bus?
Do you remember to await bus.Send(...) (i.e. await the result of the asynchronous operation) each time you call the bus?

using EWS Managed API with ASP.NET MVC

I'm trying to create a web app which does many things but the one that I'm currently focused in is the inbox count. I want to use EWS StreamSubscription so that I can get notification for each event and returns the total count of items in the inbox. How can I use this in terms of MVC? I did find some code from Microsoft tutorial that I was gonna test, but I just couldn't figure how I could use it in MVC world i.e. What's the model going to be, if model is the count then how does it get notified every time an event occurs in Exchange Server, etc.
Here's the code I downloaded from Microsoft, but just couldn't understand how I can convert the count to json and push it to client as soon as a new change event occurs. NOTE: This code is unchanged, so it doesn't return count, yet.
using System;
using System.Linq;
using System.Net;
using System.Threading;
using Microsoft.Exchange.WebServices.Data;
namespace StreamingNotificationsSample
{
internal class Program
{
private static AutoResetEvent _Signal;
private static ExchangeService _ExchangeService;
private static string _SynchronizationState;
private static Thread _BackroundSyncThread;
private static StreamingSubscriptionConnection CreateStreamingSubscription(ExchangeService service,
StreamingSubscription subscription)
{
var connection = new StreamingSubscriptionConnection(service, 30);
connection.AddSubscription(subscription);
connection.OnNotificationEvent += OnNotificationEvent;
connection.OnSubscriptionError += OnSubscriptionError;
connection.OnDisconnect += OnDisconnect;
connection.Open();
return connection;
}
private static void SynchronizeChangesPeriodically()
{
while (true)
{
try
{
// Get all changes from the server and process them according to the business
// rules.
SynchronizeChanges(new FolderId(WellKnownFolderName.Inbox));
}
catch (Exception ex)
{
Console.WriteLine("Failed to synchronize items. Error: {0}", ex);
}
// Since the SyncFolderItems operation is a
// rather expensive operation, only do this every 10 minutes
Thread.Sleep(TimeSpan.FromMinutes(10));
}
}
public static void SynchronizeChanges(FolderId folderId)
{
bool moreChangesAvailable;
do
{
Console.WriteLine("Synchronizing changes...");
// Get all changes since the last call. The synchronization cookie is stored in the _SynchronizationState field.
// Only the the ids are requested. Additional properties should be fetched via GetItem calls.
var changes = _ExchangeService.SyncFolderItems(folderId, PropertySet.IdOnly, null, 512,
SyncFolderItemsScope.NormalItems, _SynchronizationState);
// Update the synchronization cookie
_SynchronizationState = changes.SyncState;
// Process all changes
foreach (var itemChange in changes)
{
// This example just prints the ChangeType and ItemId to the console
// LOB application would apply business rules to each item.
Console.Out.WriteLine("ChangeType = {0}", itemChange.ChangeType);
Console.Out.WriteLine("ChangeType = {0}", itemChange.ItemId);
}
// If more changes are available, issue additional SyncFolderItems requests.
moreChangesAvailable = changes.MoreChangesAvailable;
} while (moreChangesAvailable);
}
public static void Main(string[] args)
{
// Create new exchange service binding
// Important point: Specify Exchange 2010 with SP1 as the requested version.
_ExchangeService = new ExchangeService(ExchangeVersion.Exchange2010_SP1)
{
Credentials = new NetworkCredential("user", "password"),
Url = new Uri("URL to the Exchange Web Services")
};
// Process all items in the folder on a background-thread.
// A real-world LOB application would retrieve the last synchronization state first
// and write it to the _SynchronizationState field.
_BackroundSyncThread = new Thread(SynchronizeChangesPeriodically);
_BackroundSyncThread.Start();
// Create a new subscription
var subscription = _ExchangeService.SubscribeToStreamingNotifications(new FolderId[] {WellKnownFolderName.Inbox},
EventType.NewMail);
// Create new streaming notification conection
var connection = CreateStreamingSubscription(_ExchangeService, subscription);
Console.Out.WriteLine("Subscription created.");
_Signal = new AutoResetEvent(false);
// Wait for the application to exit
_Signal.WaitOne();
// Finally, unsubscribe from the Exchange server
subscription.Unsubscribe();
// Close the connection
connection.Close();
}
private static void OnDisconnect(object sender, SubscriptionErrorEventArgs args)
{
// Cast the sender as a StreamingSubscriptionConnection object.
var connection = (StreamingSubscriptionConnection) sender;
// Ask the user if they want to reconnect or close the subscription.
Console.WriteLine("The connection has been aborted; probably because it timed out.");
Console.WriteLine("Do you want to reconnect to the subscription? Y/N");
while (true)
{
var keyInfo = Console.ReadKey(true);
{
switch (keyInfo.Key)
{
case ConsoleKey.Y:
// Reconnect the connection
connection.Open();
Console.WriteLine("Connection has been reopened.");
break;
case ConsoleKey.N:
// Signal the main thread to exit.
Console.WriteLine("Terminating.");
_Signal.Set();
break;
}
}
}
}
private static void OnNotificationEvent(object sender, NotificationEventArgs args)
{
// Extract the item ids for all NewMail Events in the list.
var newMails = from e in args.Events.OfType<ItemEvent>()
where e.EventType == EventType.NewMail
select e.ItemId;
// Note: For the sake of simplicity, error handling is ommited here.
// Just assume everything went fine
var response = _ExchangeService.BindToItems(newMails,
new PropertySet(BasePropertySet.IdOnly, ItemSchema.DateTimeReceived,
ItemSchema.Subject));
var items = response.Select(itemResponse => itemResponse.Item);
foreach (var item in items)
{
Console.Out.WriteLine("A new mail has been created. Received on {0}", item.DateTimeReceived);
Console.Out.WriteLine("Subject: {0}", item.Subject);
}
}
private static void OnSubscriptionError(object sender, SubscriptionErrorEventArgs args)
{
// Handle error conditions.
var e = args.Exception;
Console.Out.WriteLine("The following error occured:");
Console.Out.WriteLine(e.ToString());
Console.Out.WriteLine();
}
}
}
I just want to understand the basic concept as in what can be model, and where can I use other functions.
Your problem is that you are confusing a service (EWS) with your applications model. They are two different things. Your model is entirely in your control, and you can do whatever you want with it. EWS is outside of your control, and is merely a service you call to get data.
In your controller, you call the EWS service and get the count. Then you populate your model with that count, then in your view, you render that model property. It's really that simple.
A web page has no state. It doesn't get notified when things change. You just reload the page and get whatever the current state is (ie, whatever the current count is).
In more advanced applications, like Single Page Apps, with Ajax, you might periodically query the service in the background. Or, you might have a special notification service that uses something like SignalR to notify your SPA of a change, but these concepts are far more advanced than you currently are. You should probably develop your app as a simple stateless app first, then improve it to add ajax functionality or what not once you have a better grasp of things.
That's a very broad question without a clear-cut answer. Your model could certainly have a "Count" property that you could update. The sample code you found would likely be used by your controller.

Cannot open signalr connection in silverlight client

I am trying to implement SignalR in Silverlight based on this blog: SignalR and Silverlight.
When I try to .Invoke() I get a runtime error "System.InvalidOperationException: The Start method must be called before data can be sent. at Microsoft.AspNet.SignalR.Client.Connection.Send(String data) at Microsoft.AspNet.SignalR.Client.Hubs.HubProxy.Invoke[T](String method, Object[] args)..."
I have _conn.Start() on my connection. If I try to Start() it a second time, right before invoke, it throws an exception. Here is my code:
private IHubProxy _hub;
private HubConnection _conn;
public AddProductView()
{
InitializeComponent();
var url = Application.Current.Host.Source.GetComponents(UriComponents.Scheme | UriComponents.HostAndPort,
UriFormat.Unescaped);
_conn = new HubConnection(url);
_hub = _conn.CreateHubProxy("SilverlightPrism.Mvc.Services.ProductHub");
_hub.On<string>("NewMessage", message => Deployment.Current.Dispatcher.BeginInvoke(() => DoAddItem(message) ));
_conn.Start();
}
private void DoAddItem(string item)
{
var product = DeserializeToProduct(item);
ProductData.Products.Add(product);
}
private void buttonAdd_Click(object sender, RoutedEventArgs e)
{
Random random = new Random();
var id = Guid.NewGuid();
var product = new Product
{
Price = random.Next(1000,100000),
ProdId = id,
ProdName = "New prod."
};
var jsonMessage = SerializeToJsonString(product);
_hub.Invoke("SendMessage", jsonMessage);
}
It is throwing the exception on _hub.Invoke();
How do I correctly get the hub connect and send a message?
HubConnection.Start is asynchronous. You cannot call Start right before Invoke because your HubConnection is probably still in the Connecting state.
You need to wait for the Task returned from Start to complete before you can call IHubProxy.Invoke. You can use await (or Task.ContinueWith if you aren't running .NET 4.5) to ensure Start finishes before you enable buttonAdd.
You could also create your AddProductView object asynchronously in a factory method. Instead of calling HubConnection.Start in the constructor, you could do it in a static Task<AddProductView> CreateAddProductView() method.
Alternatively, if you don't care about the creation of your AddProductView being asynchronous, you can just call Start synchronously:
_conn.Start().Wait();

Categories

Resources