Develop Reference

How to debug connection to MQTT server using Xamarin System.Net.Mqtt on Android?

I'm trying to build a simple MQTT application using Xamarin, and testing it on both an Android emulator and a phone. Unfortunately, I'm struggling to make a connection with CreateAsync, and at a loss how to debug it.
I've checked I can connect to my RabbitMQ server as follows:
using System.Net.Mqtt;
Console.WriteLine("Trying to connect...");
var configuration = new MqttConfiguration();
var client = MqttClient.CreateAsync("127.0.0.1", configuration).Result;
var sessionState = await client.ConnectAsync(new MqttClientCredentials(clientId: "test", userName:"mqtt", password:"mqtt"));
Console.WriteLine("...it worked.");
Console.Read();
As the code tells me... it worked. :o) RabbitMQ shows the connection. I tried it with "localhost", the hostname and IP of my PC to check they all work, and an incorrect host name to see what exception gets thrown ("Socketexception: No such host is known").
My troubles start when I try to do this in the actual app. The connection code is fundamentally the same, but run in a separate task as I read you shouldn't do it in the GUI thread:
private async Task<SessionState> Connect(string BrokerHostName, Action<MqttApplicationMessage> publishEventHandler)
{
MqttConfiguration config = new MqttConfiguration();
_client = MqttClient.CreateAsync(BrokerHostName, config).Result;
SessionState sessionState = await _client.ConnectAsync(
new MqttClientCredentials(clientId: Id, userName: "mqtt", password: "mqtt")
);
await _client.SubscribeAsync("common", MqttQualityOfService.AtMostOnce);
_client.MessageStream.Subscribe(publishEventHandler);
return sessionState;
}
Called by:
var task = Connect(BrokerHostName, publishEventHandler);
But nothing happens - the code reaches this line and just hangs. If I set a break, continuing just continues to do nothing. I've made sure the INTERNET and ACCESS_NETWORK_STATE permissions are ticked in the Android manifest (though it makes no apparent difference).
This is what I've tried after some hours of Googling:
Using the hostname or IP address of my PC with the Android device, running with and without debug, and also unplugged from PC and run on its own.
Using 10.0.2.2 and running on the emulator, as I understand this is the equivalent of localhost or 127.0.0.1.
Setting the proxy address on the emulator to the same as my PC and port 1883. Even though the 'Apply' button teases with a "Proxy status: success", it still doesn't connect.
It feels like a networking problem since I can put any old rubbish as the host address and it behaves the same, but I've totally run out of ideas for what to try next or how to see what's going on. Any advice very gratefully received!

I now have this working. Here's the steps I took, in case it helps someone else:
I wrote some test apps to check TCP communication. First a client and server in Windows to check they work, then a Xamarin client app. This worked and proved the network connections were OK.
Installed an MQTT Tester on the Android emulator to prove it was possible to connect to RabbitMQ.
Tried a different MQTT framework: MQTTnet.
Similar problem but different symptoms: the code would get stuck on the .Wait() rather than inside the task itself. Then I removed all the asynchronous code and then it connected.
My conclusion is that the problem may be my lack of understanding of asynchronous programming. System.Net.Mqtt seems to require it while MQTTnet does not, so all's well that ends well!

PubSub with 'cloud-builds' topic often produces unack'ed messages

So we've been using PubSub for receiving GCB events for a while.
We have 4 subscribers to our subscription, so they can split the workload.
The subscribers are identical and written using the official C# client
The subscribers use the default settings, we configure that only 1 thread should be pulling.
They are running as a HostedService in AspNetCore inside Kubernetes.
The subscriber application has only that one responsibility
This application is deployed a couple of times every week since it's bundle with a more heavy use api.
The issue we are facing is this:
When looking at our Kibana logs we sometimes see what appears to a delayed of the pubs message of 1 or more minutes (notice that QUEUED has a later timestamp than WORKING).
However looking at the publishTime it is clear that problem is not that the event is published later, but rather that it is handled by our code later.
Now if we look at the PubSub graphs we get:
Which confirms that there indeed WAS an incident where message where not acked.
This explains why we are seeing the delayed handling of the message :).
But it does not explain WHY we appear to exceed the deadline of 60 seconds.
There are no errors / exceptions anywhere to be found
We are using the C# client in a standard way (defaults)
Now here is where it gets interesting, I discovered that if I do a PURGE messages using the google UI, everything seems to run smoothly for a while (1-3 days). But then I happens again.
Now if we look at the metrics across all the instances when the issue occurs (this is from another incident) we are at no point in time over 200ms of computation time:
Thoughts:
We are misunderstanding something basic about the pubsub ack configuration
Maybe the deploys we do somehow leads the subscription to think that there are still active subscribers and therefore it awaits them to fail before trying the next subscriber? This is indicated by the PURGE reaction, however I have no way of inspecting how many subscribers currently are registered with the subscription and I can't see a bug in the code that could imply this.
Looking at the metrics the problem is not with our code. However there might be something with the official client default config / bug.
Im really puzzled and im missing insights into what is going on inside the pubsub clusters and the official client. Some tracing from the client would be nice or query tools for pubsub like the ones we have with our Kafka clusters.
The code:
public class GoogleCloudBuildHostedService : BackgroundService
{
...
private async Task<SubscriberClient> BuildSubscriberClient()
{
var subscriptionToUse = $"{_subscriptionName}";
var subscriptionName = new SubscriptionName(_projectId,subscriptionToUse);
var settings = new SubscriberServiceApiSettings();
var client = new SubscriberClient.ClientCreationSettings(1,
credentials: GoogleCredentials.Get().UnderlyingCredential.ToChannelCredentials(),
subscriberServiceApiSettings: settings);
return await SubscriberClient.CreateAsync(subscriptionName, client);
}
protected override async Task ExecuteAsync(CancellationToken cancellationToken)
{
await Task.Yield();
cancellationToken.Register(() => _log.Info("Consumer thread stopping."));
while (cancellationToken.IsCancellationRequested == false)
{
try
{
_log.Info($"Consumer starting...");
var client = await BuildSubscriberClient();
await client.StartAsync((msg, cancellationToken) =>
{
using (eventTimer.NewTimer())
{
try
{
...
}
catch (Exception e)
{
_log.Error(e);
}
}
return Task.FromResult(SubscriberClient.Reply.Ack);
});
await client.StopAsync(cancellationToken);
await Task.Delay(TimeSpan.FromSeconds(10), cancellationToken);
}
catch (Exception e)
{
_log.Info($"Consumer failed: {e.Message}");
}
}
_log.Info($"Consumer stopping...");
}
}
Hope someone out there in the great big void can enlighten me :).
Kind regards
Christian
UPDATE
So I looked into one of the cases again, and here below we see:
the same instance of the application handling messages from the same topic and subscription.
there's 1 client thread only configured
Notice that at 15:23:04 and 15:23:10 there's 2 messages handled at the same time of publication, now 2 minutes later a message that was published at 15:23:07 is handled. And in the mean time 2 other messages are being handled.
So why is a message published at 15:23:07 not handled until 15:25:25, when other messages arrive in the mean time?

This can be happening due to different reasons and it is not a trivial task to find and troubleshoot the root of the issue.
Possible latency reasons
Related to latency, it is normal for subscriptions to have backlogged messages if they are not consuming messages fast enough or have not finished working though the backlog.
I would start by reading the following documentation, where it mentions some reasons on why you might be exceeding the deadline in some cases.
Another reason for message latency might be due to an increase in message or payload size. Check if all of your messages are moreless the same size, or if those getting handled with delay are bigger in size.
Handling message failures
I would also like to suggest taking a read here, where it talks about how to handle message failures by setting a subscription retry policy or forwarding undelivered messages to a dead-letter topic (also known as a dead-letter queue).
Good practices
This article contains some good tips and tricks for understanding how latency and message stuckness can happen and some suggestions that can help to improve that.

xBurnsed offered some good advice links, so let me just supplement it with some other things:
This looks like a classic case of a subscriber holding on to a message and then upon its lease expiring, the message gets delivered to another subscriber. The original subscriber is perhaps one that went down after it received a message, but before it could process and ack the message. You could see if the backlog correlates with restarts of instances of your subscriber. If so, this is a likely culprit. You could check to see if your subscribers are shutting down cleanly, where the subscriber StopAsync call exits cleanly and you have acknowledged all messages received by your callback before actually stopping the subscriber application.
Make sure you are using the latest version of the client. Earlier versions were subject to issues with large backlogs of small messages, though in reality the issues were not limited to those cases. This is mostly relevant if your subscribers are running up against their flow control limits.
Do the machines on which the subscribe is running have any other tasks running on them that could be using up CPU or RAM? If so, it's possible that one of the subscriber applications is starved for resources and can't process messages quickly enough.
If you are still having issues, the next best step to take is to put in a request with Cloud Support, providing the name of your project, name of your subscription, and the message ID of a message that was delayed. Support will be able to track the lifetime of a message and determine if the delivery was delayed on the server or if it was delivered multiple times and not acked.

How to continue the dialogue without responding to the message with the buttons

I send a message using PromptCustomDialog. If a person cannot answer a question for some time, how can the next message be sent? I would be grateful for the examples.
await context.Forward(new PromptCustomDialog(message, answers), Complete, context.MakeMessage(), CancellationToken.None);
public async Task Complete(IDialogContext context, IAwaitable<string> result)
{
var res = await result;
string response = res;
await Choose(context, response);
}

This would require you to set some kind of timer that would trigger an event that would cause the bot to send out a proactive message to the user. You can read more about sending proactive messages here.
The only thing I would point out is that bots, like web services, are often running multiple instances across multiple servers (e.g. if you're deployed on Azure App Services), so you would need to use some kind of distributed, stateful timer service to help you with this to ensure that the timer fires and triggers the event no matter what server it originated from.

Getting data from 50 service bus queue for a real time dashboard in a azure web app

Using the code as shown here.. I was able to create a web app that every 30 seconds sent data to client using System.Threading.Timer.
I was able to add some code which received data from a service bus queue using Messaging factory and Messaging receiver and based on that sent data to signalR client instead of hard-coding as in the mentioned example.
Now my real application gets data from 50 such queue..
Theoretically, I could create 50 timer objects which would call 50 different methods which in turn would call service bus queue.
I would sincerely appreciate if someone could suggest the right way to achieve my goal..
Thanks

The message pump pattern seems like it would be a good fit for this application. You create a separate client for each queue and configure each one to automatically listen for messages in its queue and process them as they come in.
foreach (var queueName in queueNames){
var queueClient = QueueClient.CreateFromConnectionString(connectionString, queueName);
queueClient.OnMessage(message =>
{
// Do work here
Console.Out.WriteLine(string.Format("Recieved message {0} on queue {1}", message.MessageId, queueName));
});
}

Adding Ping Feature To A Windows Service

For some scheduled tasks i had developed a Windows Service and started at a server. But these tasks are important for us and we have to monitor service in case of any troubleshoot we have to restart or take action if required.
Currently we are using Argus for monitoring servers or systems; so i want to add my service in monitor list in Argus. In my opinion ping will be enough for it, but i don't know how to add ping feature my service?
I think, i need to open a port using socket. But then? How should i answer echo request? "Pong" or "OK"

Maybe my article 'Observing Applications via Heartbeat' on codeproject will help you.
But for your needs you have to change the behaviour a little bit. Your Service should wait for a request sent by UDP (or TCP, what you prefer) from Argus and send the current Status (the 'heartbeat') back to Argus. Argus will compare the response with the expected one and send an alarm if the status was wrong or nothing was received.
Ping (ICMP echo) would only check if the Server is available (reachable via IP).

please look at the ServiceController Class
Example:
using System.ServiceProcess;
ServiceController sc = new ServiceController(SERVICENAME);
switch (sc.Status)
{
case ServiceControllerStatus.Running:
return "Running";
case ServiceControllerStatus.Stopped:
return "Stopped";
case ServiceControllerStatus.Paused:
return "Paused";
default:
return "Status Changing";
}