Develop Reference

(C#/UWP) Replacing UdpClient code with DatagramSocket

I'm mostly a C++ programmer looking to port some code over to C# but unfortunately developing for the hololens has forced me to use UWP. The following code has been working just fine for receiving some very fast UDP broadcasts (~250-500 per second)
while ( listener.Available > 0 )
{
byte[] bytes = listener.Receive(ref groupEP);
position = BitConverter.ToSingle(bytes, 0);
}
and as far as the "getting it to work" stage it's done. But moving over to UWP I'm apparently no longer allowed to use UdpClient and now forced to use "DatagramSocket" which has an async callback rather than giving you control of when you check for/process your data. I'm not generally against a callback vs playing catchup each update loop but the DatagramSocket doing roughly the same thing is causing flickering in rendering and generally just not working at all. I've tried about a thousand different ways of re-organizing my callback, but currently it looks like this:
async void MessageReceived(DatagramSocket socket, DatagramSocketMessageReceivedEventArgs eventArguments)
{
try
{
lock (this)
{
DataReader broadcastReader = eventArguments.GetDataReader();
asyncPosition = broadcastReader.ReadSingle();
}
}
catch (Exception exception)
{
}
}
I then in my update loop have this at the end, and position is used in my render routine
lock (this)
{
position = asyncPosition;
}
No matter what I put anywhere in the program or how I try and organize the callback and the variables used I can't seem to find any way to get this to work properly, and I'd really appreciate a primer on how to properly handle a DatagramSocket

What is a best approach for multithreading on SerialPort

as I am new in multithreaded application I would like to have some advice from more experienced people before starting to write the code...
I need to queue data received on serial port in serial port event for further processing.
So I have the following event handler:
void jmPort_ReceivedEvent(object source, SerialEventArgs e)
{
SetStatusLabel("Iddle...", lbStatus);
SetPicVisibility(ledNotReceiving, true);
SetPicVisibility(ledReceiving, false);
String st = jmPort.ReadLine();
if (st != null)
{
lines.Enqueue(st); //"lines" is the ConcurrentQueue<string> object
StartDataProcessing(lines); //???
SetStatusLabel("Receiving data...", lbStatus);
SetPicVisibility(ledNotReceiving, false);
SetPicVisibility(ledReceiving, true);
}
else
{
jmPort.Close();
jmPort.Open();
}
}
Within the StartDataProcessing I need to dequeue strings and update MANY UI controlls (using the InvokeRequired...this I already know :-)).
What is the best approach and colision free (without deadlock) approach to achieve this?
How to call StartDataProcessing method in more threads and safely dequeue (TryDequeue) the lines queue, make all needed computations and update UI controlls?
I have to appoint that the communication is very fast and that I am not using the standard SerialPort class. If I simply write all received strings without further processing to console window it works just well.
I am working in .NET 4.5.
Thank you for any advice...
Updated question: Ok, so what will be the best way to run the task from the datareceived event using TPL? Is it necessary to create another class (object) that will process data and use callbacks to update UI or it is possible to load some form method from the event? I'll could be very happy if someone can give me the direction what exactly to do within the datareceived event. What to do as the first step because studying all possible ways is not the solution I have time for. I need to begin with some particular way... There is so many different possible multithreading approaches and after reading about them I am still more confused and I don't know what will be the best a fastest solution... Usual Thread(s), BackgroundWorker, TPL, async-await...? :-( Because my application uses .NET 4.5 I would like to use some state-of-the-art solution :-) Thank you for any advice...

So after a lot of trying it is working to my satisfaction now.
Finally I've used the standard .NET SerialPort class as the third-party Serial class causes somae problems with higher baudrates (115200). It uses WinAPI directly so the finall code was mixed - managed and unmanaged. Now, even the standard .NET 4.5 SerialPort class works well (I've let my application successfully running through a whole night).
So, for everyone that need to deal with C#, SerialPort and higher rates (only for clarification - the device sending messages to PC is the STM32F407 /using USART 2/. I've tried it also with Arduino Due and it works as well) my datareceived event is in the following form now:
private void serialPort1_DataReceived(object sender, System.IO.Ports.SerialDataReceivedEventArgs e)
{
//the SetXXXXX functions are using the .InvokeRequired approach
//because the UI components are updated from another thread than
//the thread they were created in
SetStatusLabel("Iddle...", lbStatus);
SetPicVisibility(Form1.frm.ledNotReceiving, true);
SetPicVisibility(Form1.frm.ledReceiving, false);
String st = serialPort1.ReadLine();
if (st != null)
{
lines.Enqueue(st);
Task.Factory.StartNew(() => StartDataProcessing(lines)); // lines is global ConcurrentQueue object so in fact there is no need to pass it as parameter
SetStatusLabel("Receiving data...", lbStatus);
SetPicVisibility(Form1.frm.ledNotReceiving, false);
SetPicVisibility(Form1.frm.ledReceiving, true);
}
}
Within the StartDataProcessing function:
1. TryDequeue(lines, out str)
2. Use the ThreadPool.QueueUserWorkItem(lCallBack1, tmp); where tmp is needed part of the str (without EOF, without the message number etc.)
lCallBack1 = new WaitCallback(DisplayData);
Within the DisplayData function all the UI controls are updated
This approach mixes the ThreadPool and TPL ways but it is not a problem because the ThreadPool is used by TPL in background operation anyway.
Another working method I've tried was the following:
ThreadPool.QueueUserWorkItem(lCallBack, lines);
instead of :
Task.Factory.StartNew(() => StartDataProcessing(lines));
This method was working well but I've not tested it in over night run.
By my subjective perception the Task.... method updated the controls more smoothly but it can be only my personal feeling :-)
So, I hope this answer will help someone as I know from forums that many people are dealing with with unreliable communication based on the micocontroller <--> PC
My (surprising :-) ) conclusion is that the standard .NET SerialPort is able to handle messages even at higher baudrates. If you still run into troubles with buffer overrun then try to play with the SerialPort buffer size and SerialPort threshold. For me the settings 1024/500 are satisfactory (max size of the message send by microcontroller is 255 bytes so 500 bytes means that 2 messages are in buffer before the event is fired.)
You can also remove all SetXXXX calls from the datareceived event as they are not really needed and they can slow down the communication a little...
I am very close to real-time data capturing now and it is exactly what I've needed.
Good luck to everyone :-)

Within the StartDataProcessing I need to dequeue strings and update MANY UI controlls
No, you do not. You need to dequeue strings and then enqueue them again into the multiple queues for the different segments of the UI.
If you want to be fast, you scatter all operations and definitely the UI into separate windows that run their own separate message pumps and thus can update independently in separate UI threads.
The general process would be:
1 thread handles the serial port and takes the data and queues it.
Another one dequeues it and distributes it to separate processing threads from which
the data goes to multiple output queues all responsible for one part of the UI (depending on whether the UI Will turn a bottleneck).
There is no need to be thread safe in dequeuing. How serial is the data? Can you skip data when another update for the same piece arrives?
Read up on TPL and tasks - there are base libraries for parallel processing which come with a ton of documentation.

Is it good practice to put try-catch in a loop until all statements in the try block is executed without any exceptions?

I was trying to develop a multicast receiver program and socket initialization was done as shown below:
public void initializeThread()
{
statuscheckthread = new Thread(SetSocketOptions);
statuscheckthread.IsBackground = true;
}
private void Form1_Load(object sender, EventArgs e)
{
rxsock = new Socket(AddressFamily.InterNetwork, SocketType.Dgram, ProtocolType.Udp);
iep = new IPEndPoint(IPAddress.Any, 9191);
rxsock.Bind(iep);
ep = (EndPoint)iep;
initializeThread();
statuscheckthread.Start();
}
public void SetSocketOptions()
{
initializeThread(); //re-initializes thread thus making it not alive
while (true)
{
if (NetworkInterface.GetIsNetworkAvailable())
{
bool sockOptnSet = false;
while (!sockOptnSet)
{
try
{
rxsock.SetSocketOption(SocketOptionLevel.IP, SocketOptionName.AddMembership, new MulticastOption(IPAddress.Parse("224.50.50.50")));
rxsock.SetSocketOption(SocketOptionLevel.IP, SocketOptionName.MulticastTimeToLive, 64);
sockOptnSet = true;
}
catch
{
//Catch exception here
}
}
}
break; // Break out from loop once socket options are set
}
}
When my PC is not connected to a network, SetSocketOption method was throwing exception and even after network is connected,
I was unable to receive data because socket options are not set.
To avoid this I used a thread which runs in the background checking
for network availability and once network is available, it sets the socket options.
It works properly in some PC's but in some others, NetworkInterface.GetIsNetworkAvailable()
returned true before network got connected
(while network was being identified) .
So, to make sure Socket options are set, I used a bool variable sockOptnSet
which is set as
true if all the statements in the try block is executed as shown inside the method public void SetSocketOptions()
This program works fine in all PC's I tried, but I am doubtful about how much I can rely on this to work.
My questions are:
1) Is this good practice?
2) If not, what are the possible errors or problems it may cause? And how can I implement it in a better way?

Is this a good practice?
No, not a good practice. The vast majority of exceptions, including your first one, fall in the category of vexing exceptions. Software is supposed to work, worked well when you tested it, but doesn't on the user's machine. Something went wrong but you do not know what and there isn't anything meaningful that you can do about it. Trying to keep your program going is not useful, it cannot do the job it is supposed to do. In your case, there's no hope that the socket is ever going to receive data when there is no network. And, as you found out, trying to work around the problem just begets more problems. That's normal.
If this is bad practice, how can I implement it in a better way?
You need help from a human. The user is going to have to setup the machine to provide a working network connection. This requires a user interface, you must have a way to tell a human what he needs to do to solve your problem. You can make that as intricate or as simple as you desire. Just an error message, a verbatim copy of the Exception.Message can be enough. Writing an event handler for the AppDomain.CurrentDomain.UnhandledException event is a very good (and required) strategy. Microsoft spent an enormous amount of effort to make exception messages as clear and helpful as possible, even localizing them for you in the user's native language, you want to take advantage of that. Even if the exception message is mystifying, a quick Google query on the message text returns hundreds of hits. With this event handler in place, you don't have to do anything special. Your program automatically terminates and your user knows what to do about it.
You can certainly make it more intricate, you discovered that SetSocketOption() is liable to fail right after the network becomes available but works when you wait long enough. So this is actually an error condition that you can work around, just by waiting long enough. Whether you should write the code to handle this is something that you have to decide for yourself. It is something you write when you have enough experience with the way your program behaves, you never write it up front. Usually as a result from feedback from the users of your program.

Some good advice in the comments, lets' expand on it.
Firstly, I would put all this socket code in to its' own class, outside of the form. This makes it its' own entity and semantically easier to understand. This class could have a property Initialised, which is initially set to false. The first thing you do in your form is call an Initialise method on this class which attempts to set socket options and catches the relevant exceptions if the network is not available. If it is available, we set our Initialised property to true.
If not available, we set a single timeout (see System.Threading.Timer) that calls this same function (potentially with a retry count) after 'x' seconds. Once again we'll find ourselves back in this Initialise function, perhaps with a retry count mentioned at the beginning. Once again, if it is available, we're good - if not, set the timer again. Eventually, after 'x' retries if we're not initialised we can throw an exception or set some other failure property to indicate that we can't proceed.
Your Form class can periodically check (or hook in to an event) to determine whether the socket is now ready for communication. In case of failure you can gracefully quit out, or because our class is nice and abstracted, attempt to start the whole process again.

Detecting unexpected socket disconnect

This is not a question about how to do this, but a question about whether it's wrong what I'm doing. I've read that it's not possible to detect if a socket is closed unexpectedly (like killing the server/client process, pulling the network cable) while waiting for data (BeginReceive), without use of timers or regular sent messages, etc. But for quite a while I've been using the following setup to do this, and so far it has always worked perfectly.
public void OnReceive(IAsyncResult result)
{
try
{
var bytesReceived = this.Socket.EndReceive(result);
if (bytesReceived <= 0)
{
// normal disconnect
return;
}
// ...
this.Socket.BeginReceive...;
}
catch // SocketException
{
// abnormal disconnect
}
}
Now, since I've read it's not easily possible, I'm wondering if there's something wrong with my method. Is there? Or is there a difference between killing processes and pulling cables and similar?

It's perfectly possible and OK to do this. The general idea is:
If EndReceive returns anything other than zero, you have incoming data to process.
If EndReceive returns zero, the remote host has closed its end of the connection. That means it can still receive data you send if it's programmed to do so, but cannot send any more of its own under any circumstances. Usually when this happens you will also close your end the connection thus completing an orderly shutdown, but that's not mandatory.
If EndReceive throws, there has been an abnormal termination of the connection (process killed, network cable cut, power lost, etc).
A couple of points you have to pay attention to:
EndReceive can never return less than zero (the test in your code is misleading).
If it throws it can throw other types of exception in addition to SocketException.
If it returns zero you must be careful to stop calling BeginReceive; otherwise you will begin an infinite and meaningless ping-pong game between BeginReceive and EndReceive (it will show in your CPU usage). Your code already does this, so no need to change anything.

Socket.SendAsync is not sending in-order on Mono/Linux

There is a a single-threaded server using .NET Socket with TCP protocol, and Socket.Pool(), Socket.Select(), Socket.Receive().
To send, I used:
public void SendPacket(int clientid, byte[] packet)
{
clients[clientid].socket.Send(packet);
}
But it was very slow when sending a lot of data to one client (halting the whole main thread), so I replaced it with this:
public void SendPacket(int clientid, byte[] packet)
{
using (SocketAsyncEventArgs e = new SocketAsyncEventArgs())
{
e.SetBuffer(packet, 0, packet.Length);
clients[clientid].socket.SendAsync(e);
}
}
It works fine on Windows with .NET (I don't know if it's perfect), but on Linux with Mono, packets are either dropped or reordered (I don't know). Reverting to slow version with Socket.Send() works on Linux. Source for whole server.
How to write non-blocking SendPacket() function that works on Linux?

I'm going to take a guess that it has to do with your using statement and your SendAsync call. Perhaps e falls out of scope and is being disposed while SendAsync is still processing the buffer. But then this might throw an exception. I am really just taking a guess. Try removing the using statement and see what happens.

I would say by not abusing the async method. YOu will find nowhere a documentation stating that this acutally is forced to maintain order. it queues iem for a scheuler which get distributed to threads, and by ignoring that the oder is not maintained per documentation you open yourself up to implementation details.
The best possibly is to:
Have a queue per socket.
When you write dasta into this queue, and there is no worker thread, start a work item (ThreadPool) to process the thread.
This way you have separate distinct queues that maintain order. Only one thread will ever process one queue / socket.

I got the same problem; Linux and windows react not in the same way with SendAsync. Sometimes linux truncate the data, but there is a workaround. First of all you need to use a queue. Each time you use SendAsync you have to check the callback.
If e.Offset + e.BytesTransferred < e.Buffer.Length, you just have to e.SetBuffer(e.Offset + e.BytesTransferred, e.Buffer.Length - e.BytesTransferred - e.Offset); and call SendAsync again.
I dont know why mono-linux believe it's completed before sending all the data and it's strange but i'm sure he does.

just like #mathieu, 10y later, I can confirm on Unity Mono+Linux complete callback is called without all bytes being sent in some cases. For me it was large packets only.