Develop Reference

Azure.Messaging.ServiceBus.ServiceBusException - Creation of ReceivingAmqpLink did not complete in 30000 milliseconds & 'receiver31' is closed

I am new to Azure Service Bus and appreciate any help I can get.
In my current project, using c# and Azure.Messaging.ServiceBus we use an On-Premise server running a "Task Engine" windows service that listens to various queues (including MSMQ) to receive and process messages. We are migrating to Azure Service Bus Queue now.
I implemented ReceiveMessageAsync() method to read and process messages. The connection is persistent because the base class of the Task Engine service is already running in loop. While the below code works fine from my local pc (connected to VPN), it fails with the following error as soon as it's deployed to the on-premise server. The server also uses up all memory and shuts down causing other queues to terminate.
Error messages:
Azure.Messaging.ServiceBus.ServiceBusException: Creation of ReceivingAmqpLink did not complete in 30000 milliseconds. (ServiceTimeout)
Azure.Messaging.ServiceBus.ServiceBusException: 'receiver31' is closed (GeneralError)
Note:
Private Endpoint is enabled on Azure Service Bus and we use token and client credentials to connect to Azure.
All code below works fine locally when run for more than 2 hours and processes messages as soon as they are manually sent to queue using Azure Portal.
Code:
public **override **void StartUp(ContextBase context)
{
// Save the thread context
base.StartUp(context);
//Get values from Config
_tenantId = System.Configuration.ConfigurationManager.AppSettings["tenant-id"];
_clientId = System.Configuration.ConfigurationManager.AppSettings["client-id"];
_clientSecret = System.Configuration.ConfigurationManager.AppSettings["client-secret"];
_servicebusNamespace = System.Configuration.ConfigurationManager.AppSettings["servicebus-namespace"];
_messageQueueName = System.Configuration.ConfigurationManager.AppSettings["servicebus-inbound-queue"];
getAzureServiceBusAccess();
// Set the running flag
_isRunning = true;
}
//Called when service is initialized and then when Reset Connection happens due to error
private static void getAzureServiceBusAccess()
{
var _token = new ClientSecretCredential(_tenantId, _clientId, _clientSecret);
var clientOptions = new ServiceBusClientOptions()
{
TransportType = ServiceBusTransportType.AmqpWebSockets
};
_serviceBusClient = new ServiceBusClient(_servicebusNamespace, _token, clientOptions);
_serviceBusReceiver = _serviceBusClient.CreateReceiver(_messageQueueName, new ServiceBusReceiverOptions());
}
public **override **void DoAction()
{
// Make sure we haven't shut down
if (_isRunning)
{
// Wait next message
tryReceiveMessages();
}
}
private async void tryReceiveMessages()
{
try
{
ServiceBusReceivedMessage message = null;
message = await _serviceBusReceiver.ReceiveMessageAsync();
if (message != null && _isRunning)
{
try
{
string _messageBody = message.Body.ToString();
// <<Send message body to Task Adapter that adds it to the database and processes the job>>
await _serviceBusReceiver.CompleteMessageAsync(message);
}
catch (ServiceBusException s)
{
Tracer.RaiseError(Source.AzureSB, "Azure Service Bus Queue resulted in exception when processing message.", s);
throw;
}
catch (Exception ex)
{
Tracer.RaiseError(Source.AzureSB, "Unexpected error occurred moving task from Azure Service Bus to database; attempting to re-queue message.", ex);
if (message != null)
await _serviceBusReceiver.AbandonMessageAsync(message);
}
}
}
catch (ServiceBusException s)
{
tryResetConnections(s);
}
catch(Exception ex)
{
Tracer.RaiseError(Source.AzureSB, "Azure Service Bus Queue reset connection error.", ex);
throw;
}
}
private void tryResetConnections(Exception exception)
{
try
{
if (DateTime.Now.Subtract(LastQueueReset).TotalSeconds > 1800)
{
LastQueueReset = DateTime.Now;
getAzureServiceBusAccess();
}
else
{
//Send notification email to dev group
}
}
catch (Exception ex)
{
throw;
}
}
private async void closeAndDisposeConnectionAsync()
{
try
{
await _serviceBusReceiver.DisposeAsync();
}
catch(Exception ex)
{
//Do not throw and eat exception - Receiver may have been already disposed
}
try
{
await _serviceBusClient.DisposeAsync();
}
catch (Exception ex)
{
//Do not throw and eat exception - Client may have been already disposed
}
}
We tried to open the network settings on Azure Service Bus to public but that didn't resolve the issue.
I have requested the DevOps team to open ports 443, 5671 and 5672 for AMQPWebSockets and still waiting to hear back to test.

C# How to force the wait for the connection to a WCF service

I have a service running as local SYSTEM that launches another application with the user credentials.
That second app is only a tray icon that shows balloon tips to the user with the string received using the callback method. This second application connects to the WCF in duplex mode.
My problem is that for some reason the connection to the WCF is finalized at the end of the method Main. So I cannot send a callback message to the app right after the execution, included in the last line "kiosk.MyStart(args);". there the callback is still pointing to null.
Any idea how could I solve this issue?
static void Main(string []args)
{
if (Environment.UserInteractive)
{
// Start the WCf service
var host = new ServiceHost(typeof(WcfService));
host.Open();
//Launch the Kiosk Agent which connects to the WCF
bool ret = ProcessAsUser.Launch("C:\\Program Files (x86)\\KIOSK\\KioskAgent.exe");
WinService kiosk = new WinService(args);
// some checks and a welcome message is sent to the user.
kiosk.MyStart(args);
//...
//...
}
}
Edit: to clarify a bit more, inside kiosk.MyStart method is where I try to execute the callback to show a welcome message, but the callback is still NULL.
As a result I assume that the client was not properly started for any reason and I launch it once again...
if (WcfService.Callback != null)
WcfService.Callback.UIMessageOnCallback(UIMessage);
else
ProcessAsUser.Launch("C:\\Program Files (x86)\\KIOSK\\KioskAgent.exe");

Add a try catch block over the callback method, if the client not reachable it falls in the catch you can unsubscribe it. Is also good practice send a keepalive message to your client, to check if it available.
private void InformClient(ClientInfo clientInfo)
{
var subscribers = this._subscriberRepository.GetAll();
foreach (var subscriber in subscribers)
{
try
{
if (subscriber.Callback.FireInformClient(clientInfo));
{
//If subscriber not reachable, unsubscribe it
this._subscriberRepository.Unsubscribe(subscriber.ClientId);
}
}
catch (Exception exception)
{
//If subscriber not reachable, unsubscribe it
this._subscriberRepository.Unsubscribe(subscriber.ClientId);
Log.Error(nameof(InformClient), exception);
}
}
}
IClientCallback
public interface IClientCallback
{
[OperationContract]
bool FireInformClient(ClientInfo clientInfo);
}
If you have more subscribers for example a terminal, server create a subscriberRepository to manage all subscribers.
var callback = OperationContext.Current.GetCallbackChannel<IClientCallback>();
if (this._subscriberRepository.Subscribe(clientId, callback))
{
return true;
}

Self-Hosted SignalR service won't work (start)

I have SignalR server as Class Library Project and i referenced it in Console application (to simulate Windows service)
Here is code for SignalR
public void Start()
{
try
{
string url = #"http://*:8081";
using (WebApp.Start<Startup>(url))
{
Logger.Info(string.Format("Server running at {0}", url));
}
}
catch (Exception ex)
{
Logger.Exception(ex, "Signalr start");
}
Run = true;
Logger.Info("Starting Worker");
workerThread = new Thread(() =>
{
Worker();
});
workerThread.Start();
}
And here is Startup class
public class Startup
{
Microsoft.AspNet.SignalR.HubConfiguration hubconfiguration = null;
public void Configuration(IAppBuilder app)
{
app.UseCors(CorsOptions.AllowAll);
hubconfiguration = new HubConfiguration();
hubconfiguration.EnableDetailedErrors = true;
app.MapSignalR(hubconfiguration);
}
}
So, it is in one thread, and worker is in another. That seems fine since i did it in other project where it works. Worker thread isn't problem, it's just empty loop, not related to server in any way.
Problem is that server seems to "stop" - when i look with Netstat, nobody is listening on port 8081. There is no exception, it just silently fails.
I referenced Owin.Cors (and Owin.Host.HttpListener) in console project that actually runs this server but as I said, server just stops.
When I try to connect, client says "connection actively refused" and Putty (telnet) also says "can't connect".
Where is the problem? In a nutshell, i have Class Library with SignalR server that is referenced in Console project that runs it but server just wont work.
[edit]
And there is code of Console app that starts service
static void Main(string[] args)
{
ServiceEngine Engine = new ServiceEngine();
Engine.Start();
Console.ReadKey();
Engine.Stop();
}
P.S. Sorry for my bad English.

Well, i solved it. Here was a problem:
public static void Start()
{
try
{
string url = #"http://127.0.0.1:8081";
WebApp.Start<Startup>(url);
Logger.Info(string.Format("Server running at {0}", url));
}
catch (Exception ex)
{
Logger.Exception(ex, "signalr start");
}
Run = true;
Logger.Info("Starting Worker");
workerThread = new Thread(() =>
{
Worker();
});
workerThread.Start();
}
As you can see, using statement was removed and now it works fine! Interesting note - you can also make Singleton implementation of this "Engine", and it will also work.

I got Solution after doing lot of R & D. And Its a Simple change related to Account and Access Rights.
Use LocalSystem Account instead of LocalService Account in Service Installer.
You can do this either from doing below change in design view of your service installer:
Properties of Service Process Installer -> Set Account to LocalSystem.
or by doing below change in in designer.cs file of your service installer:
this.serviceProcessInstaller1.Account = System.ServiceProcess.ServiceAccount.LocalSystem;

C# custom service does not start on 2008 Server

I've build a little service (c# ,net 4). After a few tests and tuning on dev environment i've tried it on my own pc. All is working fine.
When I install the service on a windows 2008 R2 server (production environment) it won't start giving me a 1053 error (Service does not start in a timely fashion time)
tTe weird is that I get this error suddenly, as I start the service, just after a second or two.
I've added the option this.RequireMoreTime(120000) to my service, and I've edited the registry key of ServicePipelineMode.
Some more info:
I'm using Log4net to log events in the Application log.
I use a custom source (created using a powershell command)
The service is set to run under Network Service Account, Starting it with an administrative account get same results
I've put every part under Try - catch using log.error(exception.tostring()) but nothing is logged in event log.
I've read lot of posts on the web bbut nothing helps me to fix.
Dev and server has the same framework:
Any help would be appreciated.
Many thanks in advance.
Edit:
I've tried to add a try-catch on program.cs, the start point of my project:
static class Program
{
/// <summary>
/// The main entry point for the application.
/// </summary>
static void Main()
{
try
{
if (!EventLog.SourceExists("CheckMail"))
{
EventLog.CreateEventSource("CheckMail", "Application");
EventLog.WriteEntry("EventSystem", "EventSource created");
}
ServiceBase[] ServicesToRun;
ServicesToRun = new ServiceBase[] { new CheckMailSvc() };
ServiceBase.Run(ServicesToRun);
}
catch (Exception ex)
{
string message = string.Format("Error starting Service {0}", ex.ToString());
EventLog.WriteEntry("CheckMail", message, EventLogEntryType.Warning, 666);
}
}
}
nothing is trapped (neither rhe EventSource create event or an exception)
All other code is in try-catch, using log4net to log events (and in dev environment it works fine).

Thanks for your suggestions.
My OnStart is this:
protected override void OnStart(string[] args)
{
try
{
this.RequestAdditionalTime(30000);
ServiceStart = DateTime.Now;
XmlConfigurator.Configure();
log4net.ThreadContext.Properties["EventID"] = 666;
log.Info("Service Started");
StringBuilder sb = new StringBuilder();
sb.AppendLine("Service Configuration:");
log.Info(sb.ToString());
Timer t = new Timer(new TimerCallback(SendMail));
t.Change(TimeSpan.Zero, TimeSpan.FromMinutes(every));
}
catch (Exception ex)
{
log.ErrorFormat("Error starting service: {0}", ex.ToString());
}
}
As you can see nothing strange happens, and I should see a starting message in Application log.

WCF Windows service with an API to a windows form client, API return value tells me it could not connect to device. Whats the preffered way?

I have a WCF windows service which exposes an API to a windows form application.
API connection to client:
var serviceType = typeof(Mail2SmsServerApi);
var uri = new Uri("http://localhost:8000/");
host = new ServiceHost(serviceType, new[] { uri });
var behaviour = new ServiceMetadataBehavior() { HttpGetEnabled = true };
host.Description.Behaviors.Add(behaviour);
host.AddServiceEndpoint(serviceType, new BasicHttpBinding(), "Hello");
host.AddServiceEndpoint(typeof(IMetadataExchange), new BasicHttpBinding(), "mex");
host.Open();
My ServiceContract:
[ServiceContract]
public class Mail2SmsServerApi
{
[OperationContract]
public string Imei()
{
try
{
GSMHandler gsm = new GSMHandler();
return gsm.GetImei();
}
catch (Exception ex)
{
LogText.Error("API GetImei(), exception: " + ex.ToString());
return null;
}
}
}
My GSMHandler class and method:
public bool OpenConnection()
{
modem = new GsmPhone(_comport, _baudrate, _timeout);
if (!comm.IsConnected())
{
try
{
modem.Open();
return true;
}
catch (Exception ex)
{
LogText.Debug("OpenConnection(), exception" + ex.ToString());
return false;
}
}
else
{
try
{
modem.Close();
modem.Open();
return true;
}
catch (Exception ex)
{
LogText.Debug("OpenConnection(), exception" + ex.ToString());
return false;
}
}
}
public string GetImei()
{
string imei = "";
try
{
imei = modem.RequestSerialNumber();
LogText.Debug("IMEI:" + _IMEI);
return imei;
}
catch (Exception ex)
{
LogText.Error("Error caught in GetImei(), exception: " + ex.ToString());
return imei;
}
}
When the OnStart method in my service are called, I'm opening a connection to the modem with :
gsm = new GSMHandler();
gsm.OpenConnection();
When OnStop are called, I'm stopping it with:
gsm = new GSMHandler();
gsm.OpenConnection();
My idea was that, with this design the service would handle the communication and the client and service could interact with the modem without getting a com port not open or com port busy problem. This is obviously wrong, since I'm not able to return values from the modem this way. It has to be a design failure from my side.
What I'm trying to achieve is that a client can talk to the modem through the service, and that not both of them make a direct connection to the modem. But that the service can handle the opening and closing of connection to modem, and pass commands in to the modem from the client...
So my question is, what's the appropriate way to design such a scenario? I'm not asking for the code, just how It's usual to design it...
I'm appreciating all answers :) Thanks in advance!

I think you might have much more success if you performed the interaction with the modem as a single unit of work.
For example, design your service such that the caller calls a single method to send a text message, providing all of the necessary detail in the interface call.
The service method then performs all of the tasks necessary to open the modem, send the text message, and close the modem, in a single unit of work.
This design will allow you to ensure that the modem is always opened and closed correctly and completely within the unit of work instead of waiting for additional commands through the service that may never arrive.
Also, this design will allow you to eventually correctly support multiple modems, which your current design will not. You could have a modem pool and when a new request arrives, you could obtain an available modem from the pool, perform the unit of work, then return the modem to the pool on completion, even in a failure situation.