I am new to Azure Service Bus and appreciate any help I can get.
In my current project, using c# and Azure.Messaging.ServiceBus we use an On-Premise server running a "Task Engine" windows service that listens to various queues (including MSMQ) to receive and process messages. We are migrating to Azure Service Bus Queue now.
I implemented ReceiveMessageAsync() method to read and process messages. The connection is persistent because the base class of the Task Engine service is already running in loop. While the below code works fine from my local pc (connected to VPN), it fails with the following error as soon as it's deployed to the on-premise server. The server also uses up all memory and shuts down causing other queues to terminate.
Error messages:
Azure.Messaging.ServiceBus.ServiceBusException: Creation of ReceivingAmqpLink did not complete in 30000 milliseconds. (ServiceTimeout)
Azure.Messaging.ServiceBus.ServiceBusException: 'receiver31' is closed (GeneralError)
Note:
Private Endpoint is enabled on Azure Service Bus and we use token and client credentials to connect to Azure.
All code below works fine locally when run for more than 2 hours and processes messages as soon as they are manually sent to queue using Azure Portal.
Code:
public **override **void StartUp(ContextBase context)
{
// Save the thread context
base.StartUp(context);
//Get values from Config
_tenantId = System.Configuration.ConfigurationManager.AppSettings["tenant-id"];
_clientId = System.Configuration.ConfigurationManager.AppSettings["client-id"];
_clientSecret = System.Configuration.ConfigurationManager.AppSettings["client-secret"];
_servicebusNamespace = System.Configuration.ConfigurationManager.AppSettings["servicebus-namespace"];
_messageQueueName = System.Configuration.ConfigurationManager.AppSettings["servicebus-inbound-queue"];
getAzureServiceBusAccess();
// Set the running flag
_isRunning = true;
}
//Called when service is initialized and then when Reset Connection happens due to error
private static void getAzureServiceBusAccess()
{
var _token = new ClientSecretCredential(_tenantId, _clientId, _clientSecret);
var clientOptions = new ServiceBusClientOptions()
{
TransportType = ServiceBusTransportType.AmqpWebSockets
};
_serviceBusClient = new ServiceBusClient(_servicebusNamespace, _token, clientOptions);
_serviceBusReceiver = _serviceBusClient.CreateReceiver(_messageQueueName, new ServiceBusReceiverOptions());
}
public **override **void DoAction()
{
// Make sure we haven't shut down
if (_isRunning)
{
// Wait next message
tryReceiveMessages();
}
}
private async void tryReceiveMessages()
{
try
{
ServiceBusReceivedMessage message = null;
message = await _serviceBusReceiver.ReceiveMessageAsync();
if (message != null && _isRunning)
{
try
{
string _messageBody = message.Body.ToString();
// <<Send message body to Task Adapter that adds it to the database and processes the job>>
await _serviceBusReceiver.CompleteMessageAsync(message);
}
catch (ServiceBusException s)
{
Tracer.RaiseError(Source.AzureSB, "Azure Service Bus Queue resulted in exception when processing message.", s);
throw;
}
catch (Exception ex)
{
Tracer.RaiseError(Source.AzureSB, "Unexpected error occurred moving task from Azure Service Bus to database; attempting to re-queue message.", ex);
if (message != null)
await _serviceBusReceiver.AbandonMessageAsync(message);
}
}
}
catch (ServiceBusException s)
{
tryResetConnections(s);
}
catch(Exception ex)
{
Tracer.RaiseError(Source.AzureSB, "Azure Service Bus Queue reset connection error.", ex);
throw;
}
}
private void tryResetConnections(Exception exception)
{
try
{
if (DateTime.Now.Subtract(LastQueueReset).TotalSeconds > 1800)
{
LastQueueReset = DateTime.Now;
getAzureServiceBusAccess();
}
else
{
//Send notification email to dev group
}
}
catch (Exception ex)
{
throw;
}
}
private async void closeAndDisposeConnectionAsync()
{
try
{
await _serviceBusReceiver.DisposeAsync();
}
catch(Exception ex)
{
//Do not throw and eat exception - Receiver may have been already disposed
}
try
{
await _serviceBusClient.DisposeAsync();
}
catch (Exception ex)
{
//Do not throw and eat exception - Client may have been already disposed
}
}
We tried to open the network settings on Azure Service Bus to public but that didn't resolve the issue.
I have requested the DevOps team to open ports 443, 5671 and 5672 for AMQPWebSockets and still waiting to hear back to test.
I have implemented a rabbitMQ listener, which essentially just sits in a loop waiting for messages to arrive. When the message arrives I wish to fire an event and run some code.
However I don't always seem to receive this event, and I am wondering if this is because I am running the queue polling part of the code on a different thread.
It does seem to work initially though, so it is possible that the threading is not the problem. Can anyone give me an opinion on this?
QueueListener:
public void CreateQueueListener<T>() where T : IPubSubEvent
{
var mqServer = new RabbitMqServer(m_RabbitMqAddress);
var mqClient = (RabbitMqQueueClient)mqServer.MessageFactory.CreateMessageQueueClient();
var channel = mqClient.Channel;
string queueName = mqClient.GetTempQueueName();
channel.QueueBind(queueName, m_EventExchange, routingKey: QueueNames<T>.In);
var consumer = new RabbitMqBasicConsumer(channel);
channel.BasicConsume(queue: queueName, autoAck: true, consumer: consumer);
Task.Run(() =>
{
while (true)
{
BasicGetResult basicGetResult;
try
{
basicGetResult = consumer.Queue.Dequeue();
}
catch (Exception ex)
{
throw;
}
var message = basicGetResult.ToMessage<T>();
PublishEvent?.Invoke(this, new PubSubEventArgs { EventData = message.GetBody().EventName });
}
});
}
Consuming Class
public class MyClass
{
public MyClass(IEventClient eventClient)
{
eventClient.CreateQueueListener<AuthoriseEvent>();
eventClient.PublishEvent += OnPublishEvent;
}
private async void OnPublishEvent(object sender, PubSubEventArgs e)
{
if (e.EventData == "AuthoriseEvent")
//dostuff
}
}
I am running the queue polling part of the code on a different thread
As far as I know, this isn't supported by the .NET client.
NOTE: the RabbitMQ team monitors the rabbitmq-users mailing list and only sometimes answers questions on StackOverflow.
I have some code which rolls through a list of ports and sets up a stream containing the UDP Buffer contents which can be subscribed to.
I'm pretty new to RX and this was built with some help in response to another stackoverflow question.
This code seemed to be OK in early tests and handled all the packets we could send it but now it's in soak test the subscription seems to fail after a while and it stops sending events to the event hub.
I'm particularly worried that I have an exception somewhere in the subscription code which causes me to lose the information but isn't caught.
Here is the code, any comments or suggestions for simplification or improvement would be greatly appreciated:
var receiveStream =
_deviceTypeProvider.GetDeviceTypes().ToObservable().Trace("GetDeviceTypes")
.SelectMany(devicePortMapping => Observable
.Using(() => UdpListener(devicePortMapping),
client =>
Observable
.FromAsync(client.ReceiveAsync)
.Repeat()).Trace("UdpListener"),
(devicePortMapping, stream) =>
{
Log
.ForContext("Raw", stream.Buffer.ToPrintByteArray())
.Verbose("Received Incoming {DeviceType} Message from {Device} on Port {Port}",
devicePortMapping.DeviceType, stream.RemoteEndPoint.Address, devicePortMapping.Port);
try
{
var timeZoneOffset =
_deviceTimeZoneOffsetProvider.GetDeviceTimeZoneOffset(devicePortMapping.Port);
var tenant = _deviceTenantProvider.GetDeviceTenant(devicePortMapping.Port);
if (tenant == null || timeZoneOffset == null)
{
Log
.Error(
"Tenant or TimeOffset Missing for Port: {Port}, cannot continue processing this message",
devicePortMapping.Port);
return null;
}
var message =
new DeviceMessage(new Device(stream.RemoteEndPoint.Address.ToIPAddressNum(),
stream.RemoteEndPoint.Port, devicePortMapping.DeviceType, tenant.TenantId,
timeZoneOffset.Offset))
{
MessageUid = Guid.NewGuid(),
Timestamp = DateTime.UtcNow,
Raw = stream.Buffer,
};
message.Information("Received Incoming Message");
return message;
}
catch (Exception ex)
{
Log.Error(ex, "Exception whilst receiving incoming message");
throw;
}
}).Trace("SelectMany").Select(Task.FromResult).Trace("Select");
if (_settings.TestModeEnabled)
{
Log
.Warning("Test Mode is Enabled");
receiveStream = receiveStream
.Select(async message =>
await _testModeProvider.InjectTestModeAsync(await message)).Trace("TestMode");
}
_listener = receiveStream.Subscribe(async messageTask =>
{
var message = await messageTask;
if (message == null)
{
Log
.Warning("Message is null, returning");
return;
}
Log
.ForContext("Raw", message.Raw.ToPrintByteArray(), true)
.ForContext("Device", message.Device, true)
.Verbose("Publishing Message {MessageUid} from {#Device}", message.MessageUid, message.Device);
await _messagePublisher.Publish(message).ConfigureAwait(false);
}, error => { Log.Error(error, "Exception whilst publishing message"); });
Here is the InjectTestMode method:
public async Task<DeviceMessage> InjectTestModeAsync(DeviceMessage deviceMessage)
{
try
{
var imei = GetImei(deviceMessage.Raw);
if (string.IsNullOrEmpty(imei))
{
Log
.ForContext("DeviceMessage",deviceMessage,true)
.Error("Error while getting IMEI value from message raw data in Test Mode");
return null;
}
//var dummyIpAddress = DummyIPfromIMEI(imei).ToIPAddressNum();
var mapping = await _mappingService.GetIPMappingAsync(deviceMessage.Device.IPAddress);
if (mapping == null)
{
Log
.ForContext("DeviceMessage", deviceMessage, true)
.Warning("Test Mode updated IP Address mapping with IPAddress: {IPAddress} for IMEI: {IMEI}", deviceMessage.Device.IPAddress.ToIPAddressString(), imei);
await _mappingService.UpdateIPMappingAsync(deviceMessage.Device.IPAddress, imei);
}
// deviceMessage.Device.IPAddress = dummyIpAddress;
return deviceMessage;
}
catch (Exception ex)
{
Log
.ForContext("DeviceMessage",deviceMessage,true)
.Error("Exception raised whilst injecting Test Mode", ex);
return null;
}
}
Here is the UdpListener method:
private UdpClient UdpListener(DeviceTypeMap deviceTypeMap)
{
Log.Information("Listening for Device Type: {DeviceType} messages on Port: {Port}", deviceTypeMap.DeviceType,
deviceTypeMap.Port);
var udpClient = new UdpClient();
var serverEndpoint = new IPEndPoint(IPAddress.Any, deviceTypeMap.Port);
udpClient.Client.SetSocketOption(SocketOptionLevel.Socket, SocketOptionName.ReuseAddress, true);
udpClient.Client.Bind(serverEndpoint);
return udpClient;
}
[Update] - 16/11/2015
So I've done some research and it would seem I have more than one code smell in this so I've updated the code and it's now running but I thought I'd share the intent of this code and well as the revised code to see if someone can suggest a more elegant solution.
The Intent
Listen to several UDP Ports and send the traffic to Azure Event Hubs along with some meta based on the port it was received on.
This meta would depend on whether the system was in a 'Test Mode' or not.
The Implementation
For each Port and Device Type create a UDPListener and Observable Collection from the ReceiveAsync event, combine these Observable Collections into a single collection which could then be subscribed from component which would publish the data to EventHubs.
The Problems
The InjectMode was async, and this could return a null if there was a problem, this seemed to be killing the sequence. I think that this probably should have been some sort of Observable Extension Method which allow as to modify or remove the device message from the sequence but I couldn't figure this out.
Originally the publish to EventHub was in the subscription until I read that that you shouldn't use async inside a subscription as it generates a async void which is bad. All the research seemed to point to pushing this into a SelectMany which did'nt make any sense since this was the destination of the observed sequence not part of the process but I went with this. The subscription in effect became redundant.
I not sure that all of the try catch blocks were required but I was convinced that I had a problem which was disrupted the sequence. As noted by Enigmativity I made these all Exception catches and logged then and re-throw, nothing ever appeared from these log entries.
Retry().Do() doesn't feel right, I could get SelectMany() as suggested in many other posts to work so I had no choice.
Here's the code which is running now :
public void Start()
{
Log.Information("InboundUdpListener is starting");
var receiveStream =
_deviceTypeProvider.GetDeviceTypes().ToObservable().Trace("GetDeviceTypes")
.SelectMany(devicePortMapping => Observable
.Using(() => UdpListener(devicePortMapping),
client =>
Observable
.FromAsync(client.ReceiveAsync)
.Repeat()).Trace("UdpListener"),
(devicePortMapping, stream) =>
{
Log
.ForContext("Raw", stream.Buffer.ToPrintByteArray())
.Verbose("Received Incoming {DeviceType} Message from {Device} on Port {Port}",
devicePortMapping.DeviceType, stream.RemoteEndPoint.Address, devicePortMapping.Port);
try
{
var timeZoneOffset =
_deviceTimeZoneOffsetProvider.GetDeviceTimeZoneOffset(devicePortMapping.Port);
var tenant = _deviceTenantProvider.GetDeviceTenant(devicePortMapping.Port);
if (tenant == null || timeZoneOffset == null)
{
Log
.Error(
"Tenant or TimeOffset Missing for Port: {Port}, cannot continue processing this message",
devicePortMapping.Port);
return null;
}
var message =
new DeviceMessage(new Device(stream.RemoteEndPoint.Address.ToIPAddressNum(),
stream.RemoteEndPoint.Port, devicePortMapping.DeviceType, tenant.TenantId,
timeZoneOffset.Offset))
{
MessageUid = Guid.NewGuid(),
Timestamp = DateTime.UtcNow,
Raw = stream.Buffer,
};
message.Information("Received Incoming Message");
return message;
}
catch (Exception ex)
{
Log.Error(ex, "Exception whilst receiving incoming message");
throw;
}
}).Trace("SelectMany");
receiveStream = receiveStream.Retry().Do(async message =>
{
try
{
if (_testModeEnabled && message != null)
{
message = await _testModeProvider.InjectTestModeAsync(message);
}
if (message != null)
{
await _messagePublisher.Publish(message);
}
}
catch (Exception ex)
{
Log.Error(ex, "Exception whilst publishing incoming message");
throw;
}
}).Trace("Publish");
_listener = receiveStream.Retry().Subscribe(OnMessageReceive, OnError, OnComplete);
Log.Information("InboundUdpListener is started");
}
Can anyone see any issues with this code or suggest any improvements. I really would appreciate some help with this.
[Update Following Lee's Comment]
I totally agree that it was a mess and to show I'm willing to learn take on board people's help this is my next attempt
public void Start()
{
_listener = _deviceTypeProvider.GetDeviceTypes().ToObservable()
.SelectMany(CreateUdpListener, CreateMessage)
.SelectMany(InjectTestMode)
.SelectMany(PublishMessage)
.Retry()
.Subscribe(OnMessageReceive, OnError, OnComplete);
}
private IObservable<UdpReceiveResult> CreateUdpListener(DeviceTypeMap deviceType)
{
return Observable.Using(() => UdpListener(deviceType),
client => Observable.FromAsync(client.ReceiveAsync).Repeat());
}
private DeviceMessage CreateMessage(DeviceTypeMap deviceTypeMap, UdpReceiveResult receiveResult)
{
var timeZoneOffset =
_deviceTimeZoneOffsetProvider.GetDeviceTimeZoneOffset(deviceTypeMap.Port);
var tenant = _deviceTenantProvider.GetDeviceTenant(deviceTypeMap.Port);
if (tenant == null || timeZoneOffset == null)
{
Log
.Error(
"Tenant or TimeOffset Missing for Port: {Port}, cannot continue processing this message",
deviceTypeMap.Port);
return null;
}
var message =
new DeviceMessage(new Device(receiveResult.RemoteEndPoint.Address.ToIPAddressNum(),
receiveResult.RemoteEndPoint.Port, deviceTypeMap.DeviceType, tenant.TenantId,
timeZoneOffset.Offset))
{
MessageUid = Guid.NewGuid(),
Timestamp = DateTime.UtcNow,
Raw = receiveResult.Buffer,
};
message.Information("Received Incoming Message");
return message;
}
private async Task<DeviceMessage> InjectTestMode(DeviceMessage message)
{
if (_testModeEnabled && message != null)
{
message = await _testModeProvider.InjectTestModeAsync(message);
}
return message;
}
private async Task<DeviceMessage> PublishMessage(DeviceMessage message)
{
await _messagePublisher.Publish(message);
return message;
}
private void OnComplete()
{
throw new NotImplementedException();
}
private void OnError(Exception ex)
{
throw new NotImplementedException();
}
private void OnMessageReceive(object o)
{
throw new NotImplementedException();
}
[Final Update]
This is what we finally ended up with; an IObservable>
var listeners = Observable.Defer(() => _deviceTypeProvider.GetDeviceTypes()
.ToObservable()
.Select(UdpListener)
.SelectMany(listener =>
{
return Observable.Defer(() => Observable
.FromAsync(listener.UdpClient.ReceiveAsync)
.Where(x => x.Buffer.Length > 0)
.Repeat()
.Select(result => CreateMessage(listener.DeviceType, result))
.SelectMany(InjectTestMode)
.OfType<DeviceMessage>()
.Do(async message => await PublishMessage(message)))
.Retry();
})).Retry();
_listener = listeners.Subscribe(OnMessageReceive, OnError, OnComplete);
So I can post some code and be constructive, this is what I think the query in your Start method should look like:
_listener = _deviceTypeProvider.GetDeviceTypes().ToObservable()
.SelectMany(CreateUdpListener, CreateMessage)
.Retry()
.Subscribe(OnMessageReceive, OnError, OnComplete);
Now the poor guy to come along and support this code has a chance of understanding it. :-)
I think you're missing the fact that RX sequences do terminate on exceptions. You need to use Observable.Catch if you want to continue sequence when exception occur.
See this http://www.introtorx.com/content/v1.0.10621.0/11_AdvancedErrorHandling.html
I have a WCF windows service which exposes an API to a windows form application.
API connection to client:
var serviceType = typeof(Mail2SmsServerApi);
var uri = new Uri("http://localhost:8000/");
host = new ServiceHost(serviceType, new[] { uri });
var behaviour = new ServiceMetadataBehavior() { HttpGetEnabled = true };
host.Description.Behaviors.Add(behaviour);
host.AddServiceEndpoint(serviceType, new BasicHttpBinding(), "Hello");
host.AddServiceEndpoint(typeof(IMetadataExchange), new BasicHttpBinding(), "mex");
host.Open();
My ServiceContract:
[ServiceContract]
public class Mail2SmsServerApi
{
[OperationContract]
public string Imei()
{
try
{
GSMHandler gsm = new GSMHandler();
return gsm.GetImei();
}
catch (Exception ex)
{
LogText.Error("API GetImei(), exception: " + ex.ToString());
return null;
}
}
}
My GSMHandler class and method:
public bool OpenConnection()
{
modem = new GsmPhone(_comport, _baudrate, _timeout);
if (!comm.IsConnected())
{
try
{
modem.Open();
return true;
}
catch (Exception ex)
{
LogText.Debug("OpenConnection(), exception" + ex.ToString());
return false;
}
}
else
{
try
{
modem.Close();
modem.Open();
return true;
}
catch (Exception ex)
{
LogText.Debug("OpenConnection(), exception" + ex.ToString());
return false;
}
}
}
public string GetImei()
{
string imei = "";
try
{
imei = modem.RequestSerialNumber();
LogText.Debug("IMEI:" + _IMEI);
return imei;
}
catch (Exception ex)
{
LogText.Error("Error caught in GetImei(), exception: " + ex.ToString());
return imei;
}
}
When the OnStart method in my service are called, I'm opening a connection to the modem with :
gsm = new GSMHandler();
gsm.OpenConnection();
When OnStop are called, I'm stopping it with:
gsm = new GSMHandler();
gsm.OpenConnection();
My idea was that, with this design the service would handle the communication and the client and service could interact with the modem without getting a com port not open or com port busy problem. This is obviously wrong, since I'm not able to return values from the modem this way. It has to be a design failure from my side.
What I'm trying to achieve is that a client can talk to the modem through the service, and that not both of them make a direct connection to the modem. But that the service can handle the opening and closing of connection to modem, and pass commands in to the modem from the client...
So my question is, what's the appropriate way to design such a scenario? I'm not asking for the code, just how It's usual to design it...
I'm appreciating all answers :) Thanks in advance!
I think you might have much more success if you performed the interaction with the modem as a single unit of work.
For example, design your service such that the caller calls a single method to send a text message, providing all of the necessary detail in the interface call.
The service method then performs all of the tasks necessary to open the modem, send the text message, and close the modem, in a single unit of work.
This design will allow you to ensure that the modem is always opened and closed correctly and completely within the unit of work instead of waiting for additional commands through the service that may never arrive.
Also, this design will allow you to eventually correctly support multiple modems, which your current design will not. You could have a modem pool and when a new request arrives, you could obtain an available modem from the pool, perform the unit of work, then return the modem to the pool on completion, even in a failure situation.
I make an application in android can send character code to server C# when user input on android keyboard.
This is send method in android side:
public Boolean writeMessage(String message) {
try {
printWriter = new PrintWriter(socket.getOutputStream(), true);
printWriter.println(message);
return true;
} catch (Exception ex) {
Log.e(TAG,"write error: " +ex.getMessage());
return false;
}
}
And server listen the messages (C#):
Thread myThread = new Thread(new ThreadStart(ServeClient));
myThread.Start();
void ServeClient(){
StreamReader reader = new StreamReader(tcpClient.GetStream());
while (true)
{
try
{
String message = reader.ReadLine();
if (message != null)
{
Console.WriteLine(message);
}
}
catch (Exception ex)
{
Console.WriteLine(ex.Message);
}
}
}
My code works, but when user input faster, the server delay read message in 1,2 or 3 seconds (I have test the method get character when user input, it works fine, not delay). Where is my problem? Thanks in advance!
Your problem may be caused by Nagle's algorithm reducing TCP traffic. Consider setting TCP_NODELAY on your client (and server - TcpClient.Client.NoDelay - if communication is two-way).
Edit: For java it is Socket.setTcpNoDelay method.