Develop Reference

NetMQ/PyZMQ - Async poller sometimes doesn't recieve message until disposal

I have a DEALER/DEALER connection.
For some reason, there seems to be an importance to who runs first. If I run the C# one (NetMQ), followed by the python one, the message sent via C# will block until the python dealer is running (as expected - Send blocks until it can send). However...
sometimes, the response I send from the python dealer is not received by the C# poller (the method added to ReceiveReady isn't called). This seems to happen when I send a message from C#, then wait a few (tried ~10) seconds before I run the python client.
However the moment I either dispose of the poller on the C# end or the socket on the python end, I get the message (in the C# app, I seem to only get the first message. But if I dispose of the socket on the python end, I get all messages at once).
I'm not sure what's going on. I would have guessed some kind of timeout on the messages, if it weren't for the fact they end up being received.
C# code (about half the lines are just for disposing):
using UnityEngine;
using AsyncIO;
using NetMQ;
using NetMQ.Sockets;
using System;
using System.Threading.Tasks;
public class MsgClient_BASIC : MonoBehaviour
{
DealerSocket socket;
NetMQPoller poller;
private bool requesterIsStarted;
public static MsgClient_BASIC this_msgclient;
private void OnReceive(object s, NetMQ.NetMQSocketEventArgs a)
{
bool more;
string FS = a.Socket.ReceiveFrameString(out more);
Debug.Log(FS);
}
void Start()
{
this_msgclient = this;
ForceDotNet.Force();
socket = new DealerSocket();
socket.Bind("tcp://localhost:5558");
poller = new NetMQPoller { socket };
socket.ReceiveReady += OnReceive;
poller.RunAsync();
}
private void Update()
{
if (Input.GetKeyDown("m"))
{
Debug.Log("Sending S");
socket.SendFrame("S");
}
}
void OnDestroy()
{
Debug.Log("Stopping async poller");
try
{
poller.StopAsync();
}
catch
{
}
Debug.Log("Disposing Poller");
try
{
poller.Dispose();
}
catch
{
}
Debug.Log("Closing socket");
try
{
socket.Close();
}
catch
{
}
Debug.Log("Disposing of socket");
try
{
((IDisposable)socket).Dispose();
}
catch
{
}
Debug.Log("Cleaning up");
NetMQConfig.Cleanup(true);
}
}
Python code:
import zmq
import json
from datetime import datetime as dt
from os import path
context = zmq.Context()
class Client:
def __init__(self,port):
self.socket = context.socket(zmq.DEALER)
self.socket.setsockopt(zmq.LINGER,500)
self.socket.connect("tcp://localhost:%d" % port)
def check_msgs(self):
try:
response = self.socket.recv_string(flags=zmq.NOBLOCK)
except zmq.Again as e:
response = None
print(response)
return response
def send_dict(self,d):
jd = json.dumps(d)
self.socket.send_string(jd,flags=zmq.NOBLOCK)
def destroy(self):
self.socket.close()
context.destroy(5)
print("Destroyed")

How to read message off azure service bus

I'm trying to read a message off the queue (notified rather than polling) using C# azure function.
I'm currently pushing a message onto the queue using a console app for now, see below.
There seems to be two different products, azure service bus and a queue within a storage account. The following code uses the later, I'm not sure how to read the message of the queue?
Many thanks,
Code example
StorageCredentials creds = new StorageCredentials(accountname, accountkey);
CloudStorageAccount account = new CloudStorageAccount(creds, useHttps: true);
CloudQueueClient queueClient = account.CreateCloudQueueClient();
CloudQueue queue = queueClient.GetQueueReference("test-queue");
queue.CreateIfNotExists();
while (true)
{
CloudQueueMessage message = new CloudQueueMessage(JsonConvert.SerializeObject("my message"));
queue.AddMessage(message);
}
Update
After the comments and following links as suggested by experts trying to help I have tried the following code sample to push a message onto Azure Service Bus (i.e not a queue in a storage account)
namespace ConsoleApp1
{
using System;
using System.Text;
using System.Threading.Tasks;
using Microsoft.Azure.ServiceBus;
class Program
{
const string ServiceBusConnectionString = "Endpoint=sb://xxx.servicebus.windows.net/;SharedAccessKeyName=xxx;SharedAccessKey=xxx";
const string QueueName = "name_of_queue";
static IQueueClient queueClient;
public static async Task Main(string[] args)
{
const int numberOfMessages = 10;
queueClient = new QueueClient(ServiceBusConnectionString, QueueName);
await SendMessagesAsync(numberOfMessages);
await queueClient.CloseAsync();
}
static async Task SendMessagesAsync(int numberOfMessagesToSend)
{
for (var i = 0; i < numberOfMessagesToSend; i++)
{
string messageBody = $"Message {i}";
var message = new Message(Encoding.UTF8.GetBytes(messageBody));
Console.WriteLine($"Sending message: {messageBody}");
// Send the message to the queue
try
{
await queueClient.SendAsync(message); // this line
}
catch (Exception ex)
{
throw ex;
}
}
}
}
}
When I run the code above the console window doesn't do anything, no error message just nothing..! strange.. Looking in the azure service bus overview it active message count is zero.
I'm using this sample project but the queueClient.SendAsync never returns back. Do I need to set something up in azure, permissions perhaps?
https://github.com/Azure/azure-service-bus
Error eventually received
A connection attempt failed because the connected party did not
properly respond after a period of time, or established connection
failed because connected host has failed to respond
I can see requests within the portal service bus screen

Is these what you want?
1.If you want to get message from azure service bus queue:
using System;
using System.Text;
using System.Threading;
using System.Threading.Tasks;
using Microsoft.Azure.ServiceBus;
namespace ConsoleApp2
{
class Program
{
const string ServiceBusConnectionString = "xxxxxx";
const string QueueName = "xxx";
static IQueueClient queueClient;
public static async Task Main(string[] args)
{
queueClient = new QueueClient(ServiceBusConnectionString, QueueName);
RegisterOnMessageHandlerAndReceiveMessages();
Console.ReadKey();
await queueClient.CloseAsync();
}
static void RegisterOnMessageHandlerAndReceiveMessages()
{
var messageHandlerOptions = new MessageHandlerOptions(ExceptionReceivedHandler)
{
MaxConcurrentCalls = 10,
AutoComplete = false
};
queueClient.RegisterMessageHandler(ProcessMessagesAsync, messageHandlerOptions);
}
static async Task ProcessMessagesAsync(Message message, CancellationToken token)
{
Console.WriteLine($"Received message: SequenceNumber:{message.SystemProperties.SequenceNumber} Body:{Encoding.UTF8.GetString(message.Body)}");
await queueClient.CompleteAsync(message.SystemProperties.LockToken);
}
static Task ExceptionReceivedHandler(ExceptionReceivedEventArgs exceptionReceivedEventArgs)
{
Console.WriteLine($"Message handler encountered an exception {exceptionReceivedEventArgs.Exception}.");
var context = exceptionReceivedEventArgs.ExceptionReceivedContext;
Console.WriteLine("Exception context for troubleshooting:");
Console.WriteLine($"- Endpoint: {context.Endpoint}");
Console.WriteLine($"- Entity Path: {context.EntityPath}");
Console.WriteLine($"- Executing Action: {context.Action}");
return Task.CompletedTask;
}
}
}
Before receive from the service bus queue, I have put message in the queue.
This is the result:
2.If you want to get message from azure storage queue:
using System;
using Microsoft.Azure;
using Microsoft.Azure.Storage;
using Microsoft.Azure.Storage.Queue;
namespace ReceiveMessageFromAzureStorageQueue
{
class Program
{
static void Main(string[] args)
{
CloudStorageAccount storageAccount = CloudStorageAccount.Parse("xxxxxx");
CloudQueueClient queueClient = storageAccount.CreateCloudQueueClient();
CloudQueue queue = queueClient.GetQueueReference("xxx");
CloudQueueMessage peekedMessage = queue.PeekMessage();
queue.FetchAttributes();
int? cachedMessageCount = queue.ApproximateMessageCount;
Console.WriteLine("Number of messages in queue: " + cachedMessageCount);
for(int i=0; i<cachedMessageCount; i++) {
System.Threading.Thread.Sleep(10);
CloudQueueMessage retrievedMessage = queue.GetMessage(TimeSpan.FromMilliseconds(10));
Console.WriteLine(retrievedMessage.AsString);
queue.DeleteMessage(retrievedMessage);
}
Console.WriteLine("Already Read.");
}
}
}
Still put messages in the azure storage queue before receive them.
This is the result:
Please let me know if you have more doubts.

Azure IotHub Module's twin property change detection

I have an Azure IOThub that contains one edge device. Within this edge device I have several modules running and I can change the twin property of any individual module by connecting to it with it's connection string.
Now I would like the module to do something when it's twin property is changed but the module doesn't have access to it's connection string and it shouldn't because it shouldn't need to connect to itself.
How can a module detect it's twin property change without having a connection string?
I have followed this tutorial but this uses a connection string to detect changes: https://learn.microsoft.com/en-us/azure/iot-hub/iot-hub-csharp-csharp-module-twin-getstarted#create-a-module-identity
Here's the code for this module:
using System;
using Microsoft.Azure.Devices.Client;
using Microsoft.Azure.Devices.Shared;
using System.Threading.Tasks;
using Newtonsoft.Json;
namespace DataSyncService
{
class Program
{
private const string ModuleConnectionString = "CONNECTION STRING";
private static ModuleClient Client = null;
static void ConnectionStatusChangeHandler(ConnectionStatus status,
ConnectionStatusChangeReason reason)
{
Console.WriteLine("Connection Status Changed to {0}; the reason is {1}",
status, reason);
}
static void Main(string[] args)
{
Microsoft.Azure.Devices.Client.TransportType transport =
Microsoft.Azure.Devices.Client.TransportType.Amqp;
try
{
Client =
ModuleClient.CreateFromConnectionString(ModuleConnectionString, transport);
Client.SetConnectionStatusChangesHandler(ConnectionStatusChangeHandler);
// I want to set this callback but this requires a client and the client requires
// a connection string.
Client.SetDesiredPropertyUpdateCallbackAsync(OnDesiredPropertyChanged, null).Wait();
Console.WriteLine("Retrieving twin");
var twinTask = Client.GetTwinAsync();
twinTask.Wait();
var twin = twinTask.Result;
Console.WriteLine(JsonConvert.SerializeObject(twin.Properties));
Console.WriteLine("Sending app start time as reported property");
TwinCollection reportedProperties = new TwinCollection();
reportedProperties["DateTimeLastAppLaunch"] = DateTime.Now;
Client.UpdateReportedPropertiesAsync(reportedProperties);
}
catch (AggregateException ex)
{
Console.WriteLine("Error in sample: {0}", ex);
}
Console.WriteLine("Waiting for Events. Press enter to exit...");
Console.ReadLine();
Client.CloseAsync().Wait();
}
private static async Task OnDesiredPropertyChanged(TwinCollection desiredProperties,
object userContext)
{
Console.WriteLine("desired property change:");
Console.WriteLine(JsonConvert.SerializeObject(desiredProperties));
Console.WriteLine("Sending current time as reported property");
TwinCollection reportedProperties = new TwinCollection
{
["DateTimeLastDesiredPropertyChangeReceived"] = DateTime.Now
};
await Client.UpdateReportedPropertiesAsync(reportedProperties).ConfigureAwait(false);
}
}
}

When you create a new module in Visual Studio Code, it generates a template module for you that will show you how it's done. I'll include the important bit below:
static async Task Init()
{
MqttTransportSettings mqttSetting = new MqttTransportSettings(TransportType.Mqtt_Tcp_Only);
ITransportSettings[] settings = { mqttSetting };
// Open a connection to the Edge runtime
ModuleClient ioTHubModuleClient = await ModuleClient.CreateFromEnvironmentAsync(settings);
await ioTHubModuleClient.OpenAsync();
Console.WriteLine("IoT Hub module client initialized.");
// Register callback to be called when a message is received by the module
await ioTHubModuleClient.SetInputMessageHandlerAsync("input1", PipeMessage, ioTHubModuleClient);
}
The way this works is because the Azure IoT Edge runtime will create the module as a Docker container with the connection settings as environment variables. The module client uses those variables to connect to IoT Hub when you call
ModuleClient ioTHubModuleClient = await
ModuleClient.CreateFromEnvironmentAsync(settings);
There is a good sample on this Microsoft tutorial that also covers listening for Twin updates.

How to get IP addresses of hosts on local network running my program

I have built a peer to peer C# video conferencing application that uses a specific TCP port(17500) for audio communication. Currently, on my application interface, I enter the other IP address which has the program opened in order to communicate. What I want to do is to find the IP addresses automatically.
So, I though the best way to achieve this is to obtain the local IP addresses that are using the same TCP port number, 17500. How can I do that ? or is there any other methods getting IP addresses using the same application ?

As mentioned in comments, you need some kind of peer-discovery protocol.
As many multimedia devices, routers etc. use multicast based discovery protocols like SSDP, I created a similar discovery service sample .
Usage is simple. Just use
Discoverer.PeerJoined = ip => Console.WriteLine("JOINED:" + ip);
Discoverer.PeerLeft= ip => Console.WriteLine("LEFT:" + ip);
Discoverer.Start();
All your clients will use the same code.
using System;
using System.Net;
using System.Net.Sockets;
using System.Runtime.Caching; // add this library from the reference tab
using System.Text;
using System.Threading.Tasks;
namespace SO
{
public class Discoverer
{
static string MULTICAST_IP = "238.212.223.50"; //Random between 224.X.X.X - 239.X.X.X
static int MULTICAST_PORT = 2015; //Random
static UdpClient _UdpClient;
static MemoryCache _Peers = new MemoryCache("_PEERS_");
public static Action<string> PeerJoined = null;
public static Action<string> PeerLeft = null;
public static void Start()
{
_UdpClient = new UdpClient();
_UdpClient.Client.Bind(new IPEndPoint(IPAddress.Any, MULTICAST_PORT));
_UdpClient.JoinMulticastGroup(IPAddress.Parse(MULTICAST_IP));
Task.Run(() => Receiver());
Task.Run(() => Sender());
}
static void Sender()
{
var IamHere = Encoding.UTF8.GetBytes("I AM ALIVE");
IPEndPoint mcastEndPoint = new IPEndPoint(IPAddress.Parse(MULTICAST_IP), MULTICAST_PORT);
while (true)
{
_UdpClient.Send(IamHere, IamHere.Length, mcastEndPoint);
Task.Delay(1000).Wait();
}
}
static void Receiver()
{
var from = new IPEndPoint(0, 0);
while (true)
{
_UdpClient.Receive(ref from);
if (_Peers.Add(new CacheItem(from.Address.ToString(), from),
new CacheItemPolicy() {
SlidingExpiration = TimeSpan.FromSeconds(20),
RemovedCallback = (x) => { if (PeerLeft != null) PeerLeft(x.CacheItem.Key); }
}
)
)
{
if (PeerJoined != null) PeerJoined(from.Address.ToString());
}
Console.WriteLine(from.Address.ToString());
}
}
}
}
Now a little bit about the algorithm:
Every client multicasts a packet every seconds.
if the receiver(every client has it) gets a packet from an IP that isn't in its cache, it will fire PeerJoined method.
Cache will expire in 20 seconds. If a client doesn't receive a packet within that duration from another client in cache, it will fire PeerLeft method.

I believe if you are using a peer to peer application to exchange packets, when you need to know if someone "Is Online and Ready for connection", you need to send a broadcast. We can do it easily using an UDP connection.
I'll post an example where you use two methods: one to ask the entire network for ready clients in a broadcast message, and the other will start a listener to answer back broadcast asking message, or start a connection if a response of type "i am here" comes.
Hope it helps!
public sealed class UdpUtility
{
// Our UDP Port
private const int broadcastPort = 11000;
// Our message to ask if anyone is ready for connection
private const string askMessage = "ARE ANYONE OUT THERE?";
// Our answer message
private const string responseMessage = "I AM HERE!";
// We use this method to look for a client to connect with us.
// It will send a broadcast to the network, asking if any client is ready for connection.
public void SendBroadcastMessage()
{
var udp = new UdpClient(broadcastPort);
var endpoint = new IPEndPoint(IPAddress.Broadcast, broadcastPort);
try
{
var bytes = Encoding.ASCII.GetBytes(askMessage);
udp.Send(bytes, bytes.Length, endpoint);
}
catch (Exception ex)
{
// Treat your connection exceptions here!
}
}
// This method will start a listener on the port.
// The client will listen for the ask message and the ready message.
// It can then, answer back with a ready response, or start the TCP connection.
public void ListenBroadcastMessage()
{
var udp = new UdpClient(broadcastPort);
var endpoint = new IPEndPoint(IPAddress.Broadcast, broadcastPort);
bool received = false;
try
{
while (!received)
{
// We start listening broadcast messages on the broadcast IP Address interface.
// When a message comes, the endpoing IP Address will be updated with the sender IP Address.
// Then we can answer back the response telling that we are here, ready for connection.
var bytes = udp.Receive(ref endpoint);
var message = Encoding.ASCII.GetString(bytes);
if (message == askMessage)
{
// Our client received the ask message. We must answer back!
// When the client receives our response, his endpoint will be updated with our IP Address.
// The other client can, then, start the TCP connection and do the desired stuff.
var responseBytes = Encoding.ASCII.GetBytes(responseMessage);
udp.Send(responseBytes, responseBytes.Length, endpoint);
}
else if (message == responseMessage)
{
// We received a connection ready message! We can stop listening.
received = true;
// We received a response message!
// We can start our TCP connection here and do the desired stuff.
// Remember: The other client IP Address (the thing you want) will be on the
// endpoint object at this point. Just use it and start your TCP connection!
}
}
}
catch (Exception ex)
{
// Treat your connection exceptions here!
}
}
}

Invoke your command prompt to do "netstat -n" and extract the output.
Here is a piece of code taken from a program that I have wrote modified to fit your requirements. You will still need to further process the data to get the IP addresses
Process netP = new Process();
ProcessStartInfo netPI = new ProcessStartInfo();
netPI.FileName = "cmd";
netPI.UseShellExecute = false;
netPI.RedirectStandardOutput = true;
netPI.RedirectStandardInput = true;
netPI.RedirectStandardError = true;
netPI.CreateNoWindow = true;
netP.StartInfo = NetPI;
netP.Start();
while (!netP.Start())
Thread.Sleep(100);
StreamWriter sW = netP.StandardInput;
StreamReader sR = netP.StandardOutput;
sW.WriteLine("netstat -n")
sW.Close();
string data = sR.ReadToEnd();
sR.Close();
//Do some further processing to filter out the addresses and extract

WCF duplex P2P, avoid peer calling itself and connect to specific peer

I am working on a .NET application which would enable pairs of computers located anywhere on the Internet (and usually behind NATs) to exchange messages between each other in parallel. I did some research for possible solutions and came to the conlsuion that maybe WCF duplex P2P connectivity would be suitable for what I need. Since I am new to WCF, I created a small test application and it seems to work well in general with one exception - peer which sends particular message to the other side also gets a call on its own interface and receives it back. Here is the code
using System;
using System.Threading.Tasks;
using System.ServiceModel;
using System.ServiceModel.Channels;
using System.ServiceModel.Description;
[ServiceContract(CallbackContract = typeof(IDataChannel))]
public interface IDataChannel
{
[OperationContract(IsOneWay = true)]
void SendData(string data);
}
public class WCFP2PTransport : IDataChannel
{
private IDataChannel m_outChannel = null;
private DuplexChannelFactory<IDataChannel> m_factory = null;
private Task m_completion;
private string m_name;
public WCFP2PTransport(string service, string name)
{
m_name = name;
try
{
var binding = new NetPeerTcpBinding();
binding.Security.Mode = SecurityMode.None;
var endpoint = new ServiceEndpoint(
ContractDescription.GetContract(typeof(IDataChannel)),
binding,
new EndpointAddress("net.p2p://" + service));
m_factory = new DuplexChannelFactory<IDataChannel>(
new InstanceContext(this),
endpoint);
m_outChannel = m_factory.CreateChannel();
m_completion = Task.Factory.FromAsync(((ICommunicationObject)m_outChannel).BeginOpen, ((ICommunicationObject)m_outChannel).EndOpen, null);
}
catch (Exception ex)
{
Console.WriteLine(ex);
var tsk = new TaskCompletionSource<bool>();
tsk.SetException(ex);
m_completion = tsk.Task;
}
}
public Task ChannelOpened
{
get
{
return m_completion;
}
}
public void SendToPeer(string data)
{
m_outChannel.SendData(data);
}
// IDataChannel method(s), handle incoming traffic
public void SendData(string data)
{
Console.WriteLine("{0} Received data: {1}", m_name, data);
}
// cleanup code omitted for brevity
}
class Program
{
static void Main(string[] args)
{
var peer1 = new WCFP2PTransport("WCF_P2P_Test", "Peer1");
var peer2 = new WCFP2PTransport("WCF_P2P_Test", "Peer2");
Task.WaitAll(peer1.ChannelOpened, peer2.ChannelOpened);
peer1.SendToPeer("Packet1");
peer2.SendToPeer("Packet2");
Console.ReadKey();
}
}
Output of this application is
Peer1 Received data: Packet1
Peer2 Received data: Packet1
Peer1 Received data: Packet2
Peer2 Received data: Packet2
while I would like to see
Peer2 Received data: Packet1
Peer1 Received data: Packet2
Of course I can do some tricks in my application to filter out unwanted incoming traffic but before doing that I would like to know
Is it possible to configure WCF to not call the peer back in
response to its own call?
What is the best way to restrict peers to only connect in pairs and
each peer connecting to specific peer only? I can think of using
unique P2P service name for each pair that is why there is an
argument providing it to the peer class constructor?

You need the RemoteOnlyMessagePropagationFilter: when you create your Peer 2 Peer Channel:
var sourceContext = new InstanceContext(sourceObject);
var sourceFactory = new DuplexChannelFactory<TChannel>(sourceContext, endpointName);
TChannel sourceProxy = sourceFactory.CreateChannel();
var remoteOnlyFilter = new RemoteOnlyMessagePropagationFilter();
var peerNode = ((IClientChannel)sourceProxy).GetProperty<PeerNode>();
peerNode.MessagePropagationFilter = remoteOnlyFilter;
try
{
sourceProxy.Open();
}
catch (Exception)
{
sourceProxy.TryCloseOrAbort();
throw;
}

The most immediate answer I can think of would be to use an identification scheme and make sure you aren't sending to yourself. Once you've identified all the peers you are connected to (a managed list maybe?) just iterate over that and send the packets to each one.