Develop Reference

How to stop my program from freezing up with TCP connection C#

I have created a TCP client and it connects fine but am a bit confused how to receive messages from the server without closing the connection ?
My current approach was to run a co-routine over the network stream read method but that freezes my program up so its obviously the wrong approach so am not sure how to fix it.
I want to keep the connection alive and read messages when ever they may arrive from the server.
This is what i have setup currently:
// the idea is to run a coroutine for recieving messages
private IEnumerator<float> _RunTCPSocket()
{
int timer = DateTime.Now.Second;
byte[] readBuffer = new byte[1024];
while (SocketManager.IsConnected)
{
// this is the keep alive packets to server to prevent timeout on server side
if (DateTime.Now.Second - timer > KeepAliveRate)
{
Debug.Log("Sending");
timer = DateTime.Now.Second;
SocketManager.Send(null);
}
int msgLength = SocketManager.Recieve(readBuffer);
if (msgLength > 0)
Debug.Log(Encoding.ASCII.GetString(readBuffer, 0, msgLength));
yield return Timing.WaitForOneFrame;
}
}
This is the code for the receive method:
public int Recieve(byte[] readBuffer)
{
if (!IsConnected)
return -1; //-1 signifies an error aka we are disconnected
try
{
// NetworkStream is from TcpClient.GetStream()
bytesRead = _networkStream.Read(readBuffer, 0, readBuffer.Length);
}
catch (Exception e)
{
IsConnected = false;
Debug.Log(e);
bytesRead = -1;
}
return bytesRead;
}
How do i prevent this from locking up my program ?

You can use Begin/End method to make your program responsible:
Document from microsoft
You can see that the using of BeginReceive method is so complex so personally, i don't think it's easy to use.
An alternative is to call the read/write method inside a Task.
The third option is use TcpClient which used on client side and TcpListener which used on server side. Those two class is just a wrapper for an underline TCP socket. Those wrapper can make your life much more easier with Stream and Async methods.
If you want to learn more about network programming with C#, i highly recomment this book: C# Network Programming by Richard Blum
Update
Code for working with Task:
public event EventHandler<ReceiveDataEventArgs> DataReceived = null;
public void StartReceive()
{
Task.Run(() =>
{
while (true)
{
var bytesRead = _networkStream.Read(readBuffer, 0, readBuffer.Length);
DataReceived?.Invoke(this, new ReceiveDataEventArgs
{
Data = bytesRead
});
}
});
}
public class ReceiveDataEventArgs : EventArgs
{
public byte[] Data { get; set; }
}

High CPU usage in System.net.Socket.beginReceive()

I have a server software that has a single listening socket that then spans off multiple sockets (10 -30) which I then stream data to.
If I startup my application it used about 2-3% cpu usage on my 8 vCPU VM. After some time, generally 1-2 weeks the application suddenly starts using 60-70% cpu usage, and the thread count seems to stay stable, it does not increase.
I have run my C# profiler on my code and it comes down to the following line of code System.net.Socket.beginReceive().
I am using .net async sockets. below is my ReceiveCallBack My suspicion is that I am not handling the case when bytesRead is NOT >0. How should I modify my function below to handle that case correctly?
public static void ReadCallback(IAsyncResult ar)
{
SocketState tmpRef = null;
try
{
if (ar != null)
{
tmpRef = (SocketState)ar.AsyncState;
if (tmpRef != null)
{
// Read data from the client socket.
int bytesRead = tmpRef.WorkSocket.Client.EndReceive(ar);
//Start Reading Again
tmpRef.BeginReading(tmpRef._receievCallbackAction);
if (bytesRead > 0)
{
// Check if we have a complete message yet
for (var i = 0; i < bytesRead; i++)
{
if (tmpRef._receiveBuffer[i] == 160)
{
var tmpBuffer = new byte[i];
Array.Copy(tmpRef._receiveBuffer, tmpBuffer, i);
//Execute callback
tmpRef._receievCallbackAction(tmpBuffer);
break;
}
}
}
}
}
}
catch (Exception e)
{
if (tmpRef != null)
{
//Call callback with null value to indicate a failier
tmpRef._receievCallbackAction(null);
}
}
}
Full code: (Sorry don't want to dirty the post)
https://www.dropbox.com/s/yqjtz0r3ppgd11f/SocketState.cs?dl=0

The problem is if you do not have enough bytes yet your code spins forever waiting for the next byte to show up.
What you need to do is make a messageBuffer that survive between calls and write to that till you have all the data you need. Also, by the way your code looks you look have the opportunity to overwrite tmpRef._receiveBuffer before you have copied all the data out, your BeginReading needs to start after the copy if you are sharing a buffer.
public class SocketState
{
private readonly List<byte> _messageBuffer = new List<byte>(BufferSize);
//...
/// <summary>
/// Async Receive Callback
/// </summary>
/// <param name="ar"></param>
public static void ReadCallback(IAsyncResult ar)
{
SocketState tmpRef = null;
try
{
if (ar != null)
{
tmpRef = (SocketState)ar.AsyncState;
if (tmpRef != null)
{
// Read data from the client socket.
int bytesRead = tmpRef.WorkSocket.Client.EndReceive(ar);
if (bytesRead > 0)
{
//Loop over the bytes we received this read
for (var i = 0; i < bytesRead; i++)
{
//Copy the bytes from the receive buffer to the message buffer.
tmpRef._messageBuffer.Add(tmpRef._receiveBuffer[i]);
// Check if we have a complete message yet
if (tmpRef._receiveBuffer[i] == 160)
{
//Copy the bytes to a tmpBuffer to be passed on to the callback.
var tmpBuffer = tmpRef._messageBuffer.ToArray();
//Execute callback
tmpRef._receievCallbackAction(tmpBuffer);
//reset the message buffer and keep reading the current bytes read
tmpRef._messageBuffer.Clear();
}
}
//Start Reading Again
tmpRef.BeginReading(tmpRef._receievCallbackAction);
}
}
}
}
catch (Exception e)
{
if (tmpRef != null)
{
//Call callback with null value to indicate a failier
tmpRef._receievCallbackAction(null);
}
}
}
//...

You are explaining that the problems occurs after 1-2 weeks, which is quite rare then.
I would suggest you to orientate your researchs by improving the exception handling in your readcallback.
Within this exception handling it turns out that you are invoking the callbackAction with null.
Maybe you should consider answering the following questions :
How does the callbackAction behaves when invoked with null tmpRef._receievCallbackAction(null);
What kind of exception is caught? If it is a SocketException, maybe look at the ErrorCode, which might give you an indication
Would it be possible to dump the stack trace to know exactly where it fails ?
Some other weak point : the begin receive uses this as state object.
WorkSocket.Client.BeginReceive(_receiveBuffer, 0, BufferSize, 0, ReadCallback, this);
So it means that the thread safeness of the readcallback is not entirely guaranteed, because the call to BeginReading will occurs while you didn't process the _receiveBufferyet.

.NET Client - Waiting for an MQTT response before proceeding to the next request

I have a MQTT calls inside a loop and in each iteration, it should return a response from the subscriber so that I could use the value being forwarded after I published. But the problem is I don't know how would I do it.
I hope you have an idea there or maybe if I'm just not implementing it right, may you guide me through this. Thanks.
Here's my code:
// MyClientMgr
class MyClientMgr{
public long CurrentOutput { get; set; }
public void GetCurrentOutput(MyObjectParameters parameters, MqttClient client)
{
MyMessageObject msg = new MyMessageObject
{
Action = MyEnum.GetOutput,
Data = JsonConvert.SerializeObject(parameters)
}
mq_GetCurrentOutput(msg, client);
}
private void mq_GetCurrentOutput(MyMessageObject msg, MqttClient client)
{
string msgStr = JsonConvert.SerializeObject(msg);
client.Publish("getOutput", Encoding.UTF8.GetBytes(msgStr),
MqttMsgBase.QOS_LEVEL_EXACTLY_ONCE, false);
client.MqttMsgPublishReceived += (sender, e) =>{
MyObjectOutput output = JsonConvert.DeserializeObject<MyObjectOutput>(Encoding.UTF8.GetString(e.Message));
CurrentOutput = output;
};
}
}
// MyServerMgr
class MyServerMgr
{
public void InitSubscriptions()
{
mq_GetOutput();
}
private void mq_GetOutput()
{
MqttClient clientSubscribe = new MqttClient(host);
string clientId = Guid.NewGuid().ToString();
clientSubscribe.Connect(clientId);
clientSubscribe.Subscribe(new string[] { "getOutput" }, new byte[] { MqttMsgBase.QOS_LEVEL_EXACTLY_ONCE });
MqttClient clientPublish = new MqttClient(host);
string clientIdPub = Guid.NewGuid().ToString();
clientPublish.Connect(clientIdPub);
clientSubscribe.MqttMsgPublishReceived += (sender, e) => {
MyMessageObj msg = JsonConvert.DeserializeObject<MyMessageObj>(Encoding.UTF8.GetString(e.Message));
var output = msg.Output;
clientPublish.Publish("getOutput", Encoding.UTF8.GetBytes(JsonConvert.SerializeObject(output)), MqttMsgBase.QOS_LEVEL_EXACTLY_ONCE, false);
}
}
}
// MyCallerClass
class MyCallerClass
{
var host = "test.mqtt.org";
var myServer = new MyServerMgr(host);
var myClient = new MyClientMgr();
myServer.InitSubscriptions();
MqttClient client = new MqttClient(host);
for(int i = 0; i < 10; i++)
{
long output = 0;
MyObjectParameters parameters = {};
myClient.GetCurrentOutput(parameters, client) // here I call the method from my client manager
// to publish the getting of the output and assigned
// below for use, but the problem is the value doesn't
// being passed to the output variable because it is not
// yet returned by the server.
// Is there a way I could wait the process to
// get the response before assigning the output?
output = myClient.CurrentOutput; // output here will always be null
// because the response is not yet forwarded by the server
}
}
I have a loop in my caller class to call the mqtt publish for getting the output, but I have no idea how to get the output before it was assigned, I want to wait for the response first before going to the next.
I've already tried doing a while loop inside like this:
while(output == 0)
{
output = myClient.CurrentOutput;
}
Yes, I can get the output here, but it will slow down the process that much. And sometimes it will fail.
Please help me. Thanks.

It looks like you are trying to do synchronous communication over an asynchronous protocol (MQTT).
By this I mean you want to send a message and then wait for a response, this is not how MQTT works as there is no concept of a reply to a message at the protocol level.
I'm not that familiar with C# so I'll just give an abstract description of possible solution.
My suggestion would be to use a publishing thread, wait/pulse (Look at the Monitor class) to have this block after each publish and have the message handler call pulse when it has received the response.
If the response doesn't contain a wait to identify the original request you will also need a state machine variable to record which request is in progress.
You may want to look at putting a time out on the wait in case the other end does not respond for some reasons.

You can use AutoResetEvent class that has WaitOne() and Set() methods. Using WaitOne() after publish will wait until the message is published and using Set() under client_MqttMsgPublishReceived event will release the wait when the subscriber received the message he subscribed for.

Asynchronous components and WinForms

I'm currently writing a component to communicate with an Ethernet based device and am having to use asynchronous sockets. At times when I receive specific 'commands' from the device, I need to raise an event for whatever program is using my component (most usually a WinForm.) I'm creating a sample form for the user but I am having difficulty allowing the client form to receive the events and modify the form; I'm getting the typical "Cross-thread operation not valid: Control 'listStrings' accessed from a thread other than the thread it was created on."
I've tried reading over Implementing the Event-based Asynchronous Pattern, and Walkthrough: Implementing a Component That Supports the Event-based Asynchronous Pattern, though it doesn't quite seem to be exactly what I need, especially when reading "Opportunities for Implementing the Event-based Asynchronous Pattern" in the first link.
.Net / C# is more of a hobby than profession, and in this project - this is the last piece I need to figure out before being able to complete it. Would it be better to use a "thread-safe" (I know, everyone throws that term around like it only means one thing) existing TCP/IP component rather than trying to implement it myself?
EDIT: Here's my network class code to show you how I'm implementing it now. I forget where I came across this snippet, but it's worked fine up until I've added the form.
internal class Network
{
private Device dev;
private TcpClient client;
private NetworkStream ns;
private byte[] buffer = new byte[2048];
private Queue<byte[]> _msgQ = new Queue<byte[]>();
public Network(Device d)
{
dev = d;
}
internal void Connect(string ipAddress, int port)
{
client = new TcpClient();
client.BeginConnect(ipAddress, port, new AsyncCallback(OnConnect), null);
}
internal byte[] getLocalIp()
{
return ((IPEndPoint)client.Client.LocalEndPoint).Address.GetAddressBytes();
}
private void OnConnect(IAsyncResult ar)
{
try
{
client.EndConnect(ar);
ns = new NetworkStream(client.Client);
ns.BeginRead(buffer, 0, 2048, new AsyncCallback(OnRead), null);
while (_msgQ.Count > 0)
{
byte[] message = _msgQ.Dequeue();
ns.Write(message, 0, message.Length);
}
dev.dvDevice._connected = true;
}
catch (Exception ex)
{
Console.WriteLine(ex.Message);
}
}
internal void Disconnect()
{
try
{
client.Close();
dev.dvDevice._connected = false;
}
catch (Exception ex)
{
Console.WriteLine(ex.Message);
}
}
internal void Write(byte[] message)
{
if ((!client.Connected) || ns == null)
{
_msgQ.Enqueue(message);
return;
}
ns.Write(message, 0, message.Length);
}
private void OnWrite(IAsyncResult ar)
{
try
{
ns.EndWrite(ar);
}
catch (Exception ex)
{
Console.WriteLine(ex.Message);
}
}
private void OnRead(IAsyncResult ar)
{
try
{
int recv = ns.EndRead(ar);
byte[] message = new byte[recv];
Buffer.BlockCopy(buffer, 0, message, 0, recv);
dev.dvDevice._mh.Parse(message);
ns.BeginRead(buffer, 0, 2048, new AsyncCallback(OnRead), null);
}
catch (Exception ex)
{
Console.WriteLine(ex.Message);
}
}
}
Device is the class which is exposed to the client. It contains a MessageHandler (_mh) class which does all the parsing. Device contains the public event which is called by the MessageHandler on specific responses. Hopefully this helps in what I have so far; I'd prefer not having to rewrite too much, but to make it right (and work properly), I will if I must.
EDIT (2):
My goal for this library is that the user should not at all have to manage any of the threads - so when an event is raised, say "ReceiveString", the user should just be able to act on it without any thought.
EDIT (3):
More code for completeness.
public delegate void OnStringEvent(byte[] str);
public class Device
{
internal struct _device
{
// other stuff too, but here's what's important
public bool _connected;
public bool _online;
public MessageHandler _mh;
public Network _net;
}
public event OnStringEvent OnString;
internal void ReceiveString(byte[] str)
{
OnString(str);
}
internal _device dvDevice;
public Device(int device_number, int system_number)
{
dvDevice = new _device(device_number, system_number);
dvDevice._mh = new MessageHandler(this);
dvDevice._net = new Network(this);
}
}
internal class MessageHandler
{
private Device dev;
public MessageHandler(Device d)
{
dev = d;
}
public void Parse(byte[] message)
{
// The code goes through the message and does what it needs to
// and determines what to do next - sometimes write back or something else
// Eventually if it receives a specific command, it will do this:
dev.ReceiveString(ParseMessage(ref _reader));
}
}

Do youself a favor and rely on TPL to do the synchronization lifting for you. Example:
NetworkStream stream = MySocket.NetworkStream;
// creat a Task<int> returning the number of bytes read based on the Async patterned Begin- and EndRead methods of the Stream
Task<int> task = Task<int>.Factory.FromAsync(
fs.BeginRead, fs.EndRead, data, 0, data.Length, null);
// Add the continuation, which returns a Task<string>.
return task.ContinueWith((task) =>
{
if (task.IsFaulted)
{
ExceptionTextBox.Text = task.Exception.Message;
}
else
{
ResultTextBox.Text = string.Format("Read {0} bytes into data", task.Result);
}
}, TaskScheduler.FromCurrentSynchronizationContext());

I like #Polity's answer, being an Rx fan I would say use Rx (Reactive Extensions)
//we convert a typical begin/end (IAsyncPattern) into an observable sequence
//it returns a Func -read() that takes a byte, two ints and returns one.
var read = Observable.FromAsyncPattern<byte[], int, int, int>
(networkStream.BeginRead, networkStream.EndRead)
.ObserveOn(Scheduler.Dispatcher);
// Now, you can get an IObservable instead of an IAsyncResult when calling it.
byte[] someBytes = new byte[10];
IObservable<int> observable = read(someBytes, 0, 10);
observable.Subscribe(x=>
//x will be the returned int. You can touch UI from here.
);
Based on your code I can see that another thread calls the OnString event, then I assume when you subcribe to it, you're just adding the string into the listbox.
device.OnString += new OnStringEvent(device_onstring);
void device_onstring(byte[] str)
{
listStrings.Items.Add(...);//this is wrong, will give cross thread op error.
//you do this:
this.Invoke(new MethodInvoker(delegate()
{
listStrings.Items.Add(..);
//or anything else that touches UI
});
// this should refer to a form or control.
}

You can handle this in 2 places depending on your design. If the event is raised from a different thread you can handle it in the event handler by checking the .invokeReqeuired property of the form( or other control) handling the event. If it returns true you should use the .beginInvoke method to marshal the call to the proper thread.
Depending on your design you can handle it from the other end by passing your component an instance of the form you want to marshal to. Before you raise the event, check .invokeRequired and marshal the call so that the event is raised in the proper thread. This way the code using your library doesn't have to worry about threads, but it required that your library have a reference to system.windows.forms.

This should be a pretty easy problem to solve: you just need to execute any code in your form that updates controls using Invoke.
The precise implementation will depend on how the async code is calling back into your form. If you add that code to your question, we can provide a more complete answer.

How to sync between a listening/sending tcp client thread and the main execution?

i have a simple windows service which runs and starts a thread which listen/receive heartbeat via tcp/ip. i'm having a hard time finding ways to sync between getting information from the tcp thread and using that value to update something in the main thread.
i try to use a thread.sleep method and keep on looping it for a few times while awaiting the answer back from the thread and then getting the value, but that method seems to be a bit volatile with the method sometimes working and sometimes not.
so what's a good way to sync between these two?
basically what i want to do is to start the listening tcp thread, get specific value and the update the main program.
attached are the receive function and the function which i used to start the thread.
p.s: i'm a totally noobie when it comes to tcp/ip and c# so any comments on any part of the code or the design is more than welcome :)
public virtual void Receive()
{
string eventMessage = string.Empty;
int bytesRcvd = 0;
int totalBytesRcvd = 0;
byte[] byteBuffer = new byte[maxBufferSize];
NetworkStream listenStream;
try
{
if (client.Connected)
{
listenStream = client.GetStream();
}
else
{
return;
}
while (true)
{
//message that is slot in from the object will get sent here.
if (!string.IsNullOrEmpty(MessageToSend))
{
Send(MessageToSend);
MessageToSend = string.Empty;
}
// must convert it back and look for the delimiter, cannot wait for the three heartbeat to pass
string leftoverMsg = string.Empty;
bytesRcvd = listenStream.Read(byteBuffer, totalBytesRcvd, maxBufferSize - totalBytesRcvd);
totalBytesRcvd += bytesRcvd;
//if more than heart beat size, can process to see if it's a heartbeat and proceed to send
if (totalBytesRcvd > msgHeartbeatSize)
{
eventMessage = Encoding.ASCII.GetString(byteBuffer, 0, totalBytesRcvd);
ProcessMessage(eventMessage, ref leftoverMsg, ref totalBytesRcvd, ref byteBuffer);
}
}
}
catch (ThreadAbortException thEx)
{
//do nothing as main thread has aborted and waiting to close
logger.Info(Thread.CurrentThread.Name + " is stopped. ");
}
catch (Exception exce)
{
bIsActive = false;
logger.Error(exce);
CleanUp();
}
finally
{
logger.Info(String.Format("Thread {0} Exiting. ", Thread.CurrentThread.Name));
}
}
public virtual void StartReceivingThread()
{
Thread thrReceive = new Thread(Receive);
try
{
if (!bIsActive && Connect())
{
//NOTE: exception thrown by a thread can only be captured by that thread itself
//start a listen thread
//wait until heartbeat message is accepted
thrReceive.Name = "thr" + serviceType.Name;
thrReceive.Start();
bIsActive = true;
//wait to get the heartbeat message
for (int i = 0; i < maxRetry; i++)
{
Thread.Sleep(maxTimeOutValue);
if (bIsReceivingHeartbeat)
break;
}
//if nothing happens close the connection and try again
if (!bIsReceivingHeartbeat)
{
bIsActive = false;
CleanUp();
logger.Info("Closing receiver thread - " + thrReceive.Name);
}
else
{
logger.Info("Starting receiver thread - " + thrReceive.Name);
}
}
}
catch(Exception ex)
{
logger.Error(ex);
}
//finally
//{
// logger.Info("Exiting receiver thread - " + thrReceive.Name);
//}
}

I assume bIsReceivingHeartbeat is a bool member variable of the class. If the value changed in one thread (receiver) is not visible in the other thread this is most likely due to memory barrier. I am saying this from my Java background but this is most likely true in .net as well.
Try declaring the variables volatile or use a property and make the getter and setter synchronized:
private bool bIsReceivingHeartbeat;
public bool IsReceivingHeartbeat
{
[MethodImpl(MethodImplOptions.Synchronized)]
get { return bIsReceivingHeartbeat; }
[MethodImpl(MethodImplOptions.Synchronized)]
set { bIsReceivingHeartbeat = value; }
}
And in the calling code:
if (!IsReceivingHeartbeat) ....
I am writing from Java background but the situation most likely similar

(Looks like you also posted this code in refactormycode.com.)
Anyway, instead of the loop with a sleep delay, I recommend using an Event object that pulsed by the code that sets IsReceivingHeartbeat. See the ManualResetEvent and AutoResetEvent classes in MSDN.