I have a .NET CORE 2.1 console application that is communicating with 2 devices which are both half-duplex via 2 comPort (rs232).
the device A on COM1, baud rate 9600, my app polling it every 200ms, and get response in 50ms.
the device B on COM2, baud rate 1200, my app receive its poll every 400ms, and responding it in 50ms.
the code for two comPort are totally separated, no share variable, no reference and etc.
For device A:
private ConcurrentQueue<object> outgoingMessageQueue = new ConcurrentQueue<object>();
this.comPort1.DataReceived += (a,b)=>{
while (this.comPort1.BytesToRead > 0){
var b = this.comPort.ReadByte();
buffer.Add((byte)b);
};
if(CheckIsFullMessage(buffer))
{//fire event for consume}
};
ThreadPool.QueueWorkerThread((_)=>{
while(true){
Thread.Sleep(200);
if (this.outgoingMessageQueue.TryDequeue(out object pendingForWrite))
{this.comPort1.Write(pendingForWrite); }
else
this.comPort1.Write(new PollMsg());
}};
//business logic, queue a request at any time.
this.outgoingMessageQueue.Add(new requestMessage());
For device B:
this.comPort2.DataReceived += (a,b)=>{
while (this.comPort2.BytesToRead > 0){
var b = this.comPort.ReadByte();
buffer.Add((byte)b);
};
if(CheckIsFullMessage(buffer))
{
//trigger business logic, consume the buffer and construct a response.
//this.comPort2.Write(response,0,response.length);
}
};
I noticed a thing, that if I turned on device B, the DataReceived for device A (comPort1) will be randomly delayed to be called(from ms to seconds), during the delay period, device A's 200ms polling is never stoped, so I'll suddenly get huge data from device A at one DataReceived.
Could anyone help, why these two comPorts affected each other?
-----more test----
I've did a test that connect 3 device A in 3 comPort into app, they works good, no DataReceived delayed.
After some testing and post from web, I confirmed this behavior on .NET CORE, that multiple SerialPort write&receive could delay the firing of the DataReceived, so rather than wait, I've added a code to actively pull
public void Start()
{
this.comPort.DataReceived += (_,__)=>{this.PullPortDataBuffer();};
//for very fast time accurate call(my app send&receive data every 200ms), use dedicated thread rather than timer.
this.pullingSerialPortBufferLoop = new Thread(() =>
{
while (true)
{
Thread.Sleep(200);
this.PullPortDataBuffer();
}
});
this.pullingSerialPortBufferLoop.Start();
};
var buffer = new List<byte>();
private void PullPortDataBuffer()
{
if (0 == Interlocked.CompareExchange(ref this.onPullingComPortBuffer, 1, 0))
try
{
while (this.comPort.BytesToRead > 0)
{
this.buffer.Add((byte)b);
}
this.ConsumeBufferIfFull(this.buffer);
}
catch (Exception ex)
{}
finally
{
this.onPullingComPortBuffer = 0;
}
else
{
if (logger.IsDebugEnabled)
logger.Debug(this.comPort.PortName + " concurrent enter Port_DataReceived");
}
}
from my testing, the issue is gone.
I have a client type application that is receiving packets from remote server.
Now and then it so happens that for some reason server disconnects me.
Sometimes there are some problems on my end ISP drops internet etc.
I have been trying to catch those exceptions and goog-ling for an answer but at the end every one
points to "make a timer and check periodically for received packets."
Now i have a function that is receiving all incoming traffic.
This function is executed every time packet is received.
Now my idea is to create a function that will create timer with let say 50 seconds time out.
This function will reset timer to 0 each time packet is received and restart it.
If timer reach 50 seconds it will throw an error "disconnected!" and some logic will follow
how to reconnect.
Now main problem i have is ... i can not "pause" my main packet receiving function.
I have tried to make it in another thread but program keep recreating new threads , killing threads by ID is a bad practice and i haven't gone down that road ... yet.
Is this a how i should handle my problem or someone has a better idea?
Below is my packet receive function.
public void OnReceive()
{
try
{
recv_pack = locSec.TrIncom();
if (recv_pack != null)
{
foreach (Packet packet in recv_pack)
{
byte[] packet_bytes = packet.GetBytes();
PacketHandler.HandlePacket(packet, locSec);
//here i would check for packet receive with timer
//CheckDisconnect();
}
}
}
catch()
{}
}
So far i have come up with this:
public bool CheckDisconnect()
{
bool KeepGoing = true;
for(int i = 0 ; i <= 50 && KeepGoing; i++ )
{
Thead.Sleep(1000);
if(i == 50)
{
KeepGoing = false;
Console.WriteLine("Disconnected!");
// ... starting reconnect procedure
}
}
}
Not sure if i understand completely, but if those two functions are in the same thread, can't you just make a global variable that controls the OnReceive() function and set it to false in your CheckDisconnect() function?
I have a small game server I'm making that will have dozens of connections sending player data constantly. While I've finally accomplished some basics and now have data sending/receiving, I now face a problem of flooding the server and the client with too much data. I've tried to throttle it back but even then I am hitting 90-100% cpu simply because of receiving and processing the data received running up the CPU.
The method below is a bare version of receiving data from the server. The server sends a List of data to be received by the player, then it goes through that list. I've thought perhaps instead just using a dictionary with a key based on type rather than for looping but I don't think that will significantly improve it, the problem is that it is processing data non-stop because player positions are constantly being updated, sent to the server, then send to other players.
The code below shows receive for the client, the server receive looks very similar. How might I begin to overcome this issue? Please be nice, I am still new to network programming.
private void Receive(System.Object client)
{
MemoryStream memStream = null;
TcpClient thisClient = (TcpClient)client;
List<System.Object> objects = new List<System.Object>();
while (thisClient.Connected && playerConnected == true)
{
try
{
do
{
//when receiving data, first comes length then comes the data
byte[] buffer = GetStreamByteBuffer(netStream, 4); //blocks while waiting for data
int msgLenth = BitConverter.ToInt32(buffer, 0);
if (msgLenth <= 0)
{
playerConnected = false;
thisClient.Close();
break;
}
if (msgLenth > 0)
{
buffer = GetStreamByteBuffer(netStream, msgLenth);
memStream = new MemoryStream(buffer);
}
} while (netStream.DataAvailable);
if (memStream != null)
{
BinaryFormatter formatter = new BinaryFormatter();
memStream.Position = 0;
objects = new List<System.Object>((List<System.Object>)formatter.Deserialize(memStream));
}
}
catch (Exception ex)
{
Console.WriteLine("Exception: " + ex.ToString());
if (thisClient.Connected == false)
{
playerConnected = false;
netStream.Close();
thisClient.Close();
break;
}
}
try
{
if (objects != null)
{
for (int i = 0; i < objects.Count; i++)
{
if(objects[i] != null)
{
if (objects[i].GetType() == typeof(GameObject))
{
GameObject p = (GameObject)objects[i];
GameObject item;
if (mapGameObjects.TryGetValue(p.objectID, out item))
{
mapGameObjects[p.objectID] = p;;
}
else
{
mapGameObjects.Add(p.objectID, p);
}
}
}
}
}
}
catch (Exception ex)
{
Console.WriteLine("Exception " + ex.ToString());
if (thisClient.Connected == false)
{
playerConnected = false;
netStream.Close();
break;
}
}
}
Console.WriteLine("Receive thread closed for client.");
}
public static byte[] GetStreamByteBuffer(NetworkStream stream, int n)
{
byte[] buffer = new byte[n];
int bytesRead = 0;
int chunk = 0;
while (bytesRead < n)
{
chunk = stream.Read(buffer, (int)bytesRead, buffer.Length - (int)bytesRead);
if (chunk == 0)
{
break;
}
bytesRead += chunk;
}
return buffer;
}
Based on the code shown, I can't say why the CPU utilization is high. The loop will wait for data, and the wait should not consume CPU. That said, it still polls the connection in checking the DataAvailable property, which is inefficient and can cause you to ignore received data (in the implementation shown...that's not an inherent problem with DataAvailable).
I'll go one further than the other answer and state that you should simply rewrite the code. Polling the socket is just no way to handle network I/O. This would be true in any scenario, but it is especially problematic if you are trying to write a game server, because you're going to use up a lot of your CPU bandwidth needlessly, taking it away from game logic.
The two biggest changes you should make here are:
Don't use the DataAvailable property. Ever. Instead, use one of the asynchronous APIs for dealing with network I/O. My favorite approach with the latest .NET is to wrap the Socket in a NetworkStream (or get the NetworkStream from a TcpClient as you do in your code) and then use the Stream.ReadAsync() along with async and await. But the older asynchronous APIs for Sockets work well also.
Separate your network I/O code from the game logic code. The Receive() method you show here has both the I/O and the actual processing of the data relative to the game state in the same method. This two pieces of functionality really belong in two separate classes. Keep both classes, and especially the interface between them, very simple and the code will be a lot easier to write and to maintain.
If you decide to ignore all of the above, you should at least be aware that your GetStreamByteBuffer() method has a bug in it: if you reach the end of the stream before reading as many bytes were requested, you still return a buffer as large as was requested, with no way for the caller to know the buffer is incomplete.
And finally, IMHO you should be more careful about how you shutdown and close the connection. Read about "graceful closure" for the TCP protocol. It's important that each end signal that they are done sending, and that each end receive the other end's signal, before either end actually closes the connection. This will allow the underlying networking protocol to release resources as efficiently and as quickly as possible. Note that TcpClient exposes the socket as the Client property, which you can use to call Shutdown().
Polling is rarely a good approach to communication, unless you're programming 16-bit microcontrollers (and even then, probably not the best solution).
What you need to do is to switch to a producer-consumer pattern, where your input port (a serial port, an input file, or a TCP socket) will act as a producer filling a FIFO buffer (a queue of bytes), and some other part of your program will be able to asynchronously consume the enqueued data.
In C#, there are several ways to do it: you can simply write a couple of methods using a ConcurrentQueue<byte>, or a BlockingCollection, or you can try a library like the TPL Dataflow Library which IMO doesn't add too much value over existing structures in .NET 4. Prior to .NET 4, you would simply use a Queue<byte>, a lock, and a AutoResetEvent to do the same job.
So the general idea is:
When your input port fires a "data received" event, enqueue all received data into the FIFO buffer and set a synchronization event to notify the consumer,
In your consumer thread, wait for the synchonization event. When the signal is received, check if there is enough data in the queue. If yes, process it, if not, continue waiting for the next signal.
For robustness, use an additional watchdog timer (or simply "time since last received data") to be able to fail on timeout.
You want to use the Task-based Asynchronous Pattern. Probably making liberal use of the async function modifier and the await keyword.
You'd be best replacing GetStreamByteBuffer with a direct call to ReadAsync.
For instance you could asynchronously read from a stream like this.
private static async Task<T> ReadAsync<T>(
Stream source,
CancellationToken token)
{
int requestLength;
{
var initialBuffer = new byte[sizeof(int)];
var readCount = await source.ReadAsync(
initialBuffer,
0,
sizeof(int),
token);
if (readCount != sizeof(int))
{
throw new InvalidOperationException(
"Not enough bytes in stream to read request length.");
}
requestLength = BitConvertor.ToInt32(initialBuffer, 0);
}
var requestBuffer = new byte[requestLength];
var bytesRead = await source.ReadAsync(
requestBuffer,
0,
requestLength,
token);
if (bytesRead != requestLength)
{
throw new InvalidDataException(
string.Format(
"Not enough bytes in stream to match request length." +
" Expected:{0}, Actual:{1}",
requestLength,
bytesRead));
}
var serializer = new BinaryFormatter();
using (var requestData = new MemoryStream(requestBuffer))
{
return (T)serializer.Deserialize(requestData);
}
}
Like your code this reads an int from the stream to get the length, then reads that number of bytes and uses the BinaryFormatter to deserialize the data to the specified generic type.
Using this generic function you can simplify your logic,
private Task Receive(
TcpClient thisClient,
CancellationToken token)
{
IList<object> objects;
while (thisClient.Connected && playerConnected == true)
{
try
{
objects = ReadAsync<List<object>>(netStream, token);
}
catch (Exception ex)
{
Console.WriteLine("Exception: " + ex.ToString());
if (thisClient.Connected == false)
{
playerConnected = false;
netStream.Close();
thisClient.Close();
break;
}
}
try
{
foreach (var p in objects.OfType<GameObject>())
{
if (p != null)
{
mapGameObjects[p.objectID] = p;
}
}
}
catch (Exception ex)
{
Console.WriteLine("Exception " + ex.ToString());
if (thisClient.Connected == false)
{
playerConnected = false;
netStream.Close();
break;
}
}
}
Console.WriteLine("Receive thread closed for client.");
}
You need to put a Thread.Sleep(10) in your while loop. This is also a very fragile way to receive tcp data because it assumes the other side has sent all data before you call this receive. If the other side has only sent half of the data this method fails. This can be countered by either sending fixed sized packages or sending the length of a package first.
Your player position update is similar to the framebuffer update in the VNC protocol where the client request a screen frame & server responds to it with the updated screen data. But there is one exception, VNC server doesn't blindly send the new screen it only sends the changes if there is one. So you need to change the logic from sending all the requested list of objects to only to the objects which are changed after the last sent. Also in addition to it, you should send entire object only once after that send only the changed properties, this will greatly reduce the size of data sent & processed both at clients & server.
I have 2 threads running simultaneously and each is writing to the socket.Send() stream,
While (soc.Connected)
{
byte[] byData = new byte[2];
byData = System.Text.Encoding.ASCII.GetBytes("A");
soc.Send(BitConverter.GetBytes(byData.Length));
soc.Send(byData);
}
The other thread uses the exact same code, except it's sending "1" instead of "A".
How will the data at the other end look like? Will it be either a stream of AAAAAAAA or 111111111s or randomly mixed like A1A1111AAAA1 ?
Should I avoid this way of sending entirely and block the sending until the other thread finishes?
Should I avoid this way of sending entirely and block the sending until the other thread finishes?
Yes and no, you should avoid this entirely but it is not necessary to block sending till the other thread finishes.
What you could do is have a 3rd thread who's responsibility is to send data and your two threads who need to send data put their data on to a thread safe queue. Then the sending thread would dequeue the work to be done and send it out on the wire.
const int MAX_QUEUE_LENGTH = 10;
private BlockingCollection<MyMessageContainer> messageQueue = new BlockingCollection<MyMessageContainer>(new ConcurrentQueue<MyMessageContainer>(), MAX_QUEUE_LENGTH);
void ProcessMessages()
{
foreach (var message in messageQueue.GetConsumingEnumerable())
{
if(soc.Connected == false)
break;
soc.Send(message.ToPaylod());
}
}
void GenerateMessageOne()
{
while(true)
{
messageQueue.Add(new MyMessageContainer("A"));
}
}
void GenerateMessageTwo()
{
while(true)
{
messageQueue.Add(new MyMessageContainer("1"));
}
}
class MyMessageContainer
{
public MyMessageContainer(string message)
{
_message = message;
}
private string _message;
public byte[] ToPayload()
{
var lengthBytes = BitConverter.GetBytes(byData.Length);
return lengthBytes.Concat(() => System.Text.Encoding.ASCII.GetBytes(_message)).ToArray();
}
}
The above code will let both threads queue work at the same time without blocking till the queue reaches a length of MAX_QUEUE_LENGTH, once there calling messageQueue.Add( will now start blocking till the sending thread has had a chance to clear up some room, once room has been made it will unblock one of the functions and let it continue.
If you want randomly-sequenced output, the easiest solution is to simply put a lock around the line that actually writes to the socket. I would also recommend adding a call to Thread.Sleep for fairness, though that is somewhat optional.
While (soc.Connected)
{
byte[] byData = new byte[2];
byData = System.Text.Encoding.ASCII.GetBytes("A");
lock(soc)
{
soc.Send(BitConverter.GetBytes(byData.Length));
soc.Send(byData);
}
Thread.Sleep(0);
}
Ok I need some help. When a packet is sent out that requires reliability it gets passed to the ReliabilityLayer. From there the ReliabilityLayer adds that packet to a list then writes it once to the SocketLayer. The ReliabilityLayer then spawns a thread that has 2 timers. While the packet is still in the list the first timer continuously sends the packet to the SocketLayer every 250ms. The second timer is the timeout time. It throws an exception after 2s. The ReliabilityLayer hooks into the packet receive event and when an ACK packet comes back containing the checksum of a packet in the ReliabilityLayer packet list, it should remove it allowing the thread to exit. Problem is multithreading...Accessing the list across threads is giving me random null pointers and other problems. So I have to either make it thread safe somehow or rethink this whole thing. I was wondering if anyone could help me out? Thanks
public void Write(NetworkPacket packet, ClientInfo client, Action<byte[], EndPoint> action)
{
if (CSL)
throw new Exception("ReliabilityLayer loaded for client use.");
if (!packet.Command.RequiresReliability())
throw new ArgumentException("Packet does not require reliability.");
//Add the packet to the reliability list
packetList.Add(packet);
//Send the packet to the socket layer.
action.Invoke(packet.RawData, client.EndPoint);
new Thread(() =>
{
Stopwatch timeout = new Stopwatch();
Stopwatch timer = new Stopwatch();
timer.Start();
timeout.Start();
while (packetList.Contains(packet))
{
//Still no response from the remote connection -> send another packet
if (timer.ElapsedMilliseconds > 256)
{
action.Invoke(packet.RawData, client.EndPoint);
timer.Restart();
}
//No response after 2 seconds -> throw exception
if (timeout.ElapsedMilliseconds > 2048)
{
throw new Exception("Client has not responded to the request.");
}
}
}).Start();
}
private void ssl_OnPacketReceived(object sender, ServerPacketEventArgs e)
{
if (e.Packet.Command != Command.Ack)
return;
//Find matching packet in the packetList
NetworkPacket packet = packetList.Find(pkt => pkt.Checksum == e.Packet.Data[0]); //e.Packet.Data[0] is the checksum of the packet that was send out.
if (packet != null)
{
//Remove it to allow thread to exit
packetList.Remove(packet);
}
}
The easiest way to resolve the problem is to "guard" any calls to List with lock().
You can check here how to do it.
In short explanation is the following:
You should "guard" not thread safe operations following way
private object private_obj_to_be_used = new object();
lock(private_obj_to_be_used)
{
/// not thread safe operation goes here<br/>
}
Please note that you have to "guard" not only inserts or removes but the reads also.
Or you can check if there any "Concurrent" class is suitable for you.