Reading all data from stream of TcpClient - c#

I am working on a new machine at my works which is controlled via a PC. Current stuck on talking to a camera system that is connected via Ethernet.
Here is my code of opening a connection.
TcpClient client;
NetworkStream stream;
public bool OpenConnection()
{
client = new TcpClient();
try
{
//Camera.Open();
client.Connect("192.168.0.10", 8500);
stream = client.GetStream();
}
catch (Exception ex)
{
return false;
}
return true;
}
So far so good. Once the connection is open I then request some information from the camera
The message format I am using has STX(0x02) to indicate the start of the message and ETX(0x03) as the end of the message.
char StartOfPacket = (char)0x02;
char EndOfPacket= (char)0x03;
public bool RetrieveDigits(out string Digits)
{
// Send the trigger cammand to the vision system
Digits = "";
bool EverythingOK = true;
string DataToSend = StartOfPacket + "T1" + EndOfPacket;
byte[] buff = Encoding.ASCII.GetBytes(DataToSend);
if (LogCameraEvent != null)
LogCameraEvent(">> " + DataToSend);
try
{
stream.Write(buff, 0, buff.Length);
}
catch (Exception ex)
{
Logging.Instance.LogExceptionToFile(MethodBase.GetCurrentMethod(), ex);
EverythingOK = false;
}
Thread.Sleep(100);
byte[] buffer;
if (EverythingOK)
{
// Check the response
buffer = ReadCamera(10);
// Process the packets
string[] packets = ProcessPackets(buffer);
if (packets != null)
{
if (packets.Length > 0)
{
bool TriggerFound = false;
for (int i = 0; i < packets.Length; i++)
{
if (packets[i] == "T1")
{
TriggerFound = true;
continue;
}
else if (TriggerFound)
{
// If we are here then we should now be the data that was requested
if (string.IsNullOrEmpty(packets[i]))
{
Digits = packets[i-1]; // previous packet may have data from previous trigger. Need to look into why this happens.
}
else
Digits = packets[i];
EverythingOK = true;
break;
}
else
EverythingOK = false;
}
}
else
{
Console.WriteLine("No Packets Recieved");
EverythingOK = false;
}
}
else
{
Console.WriteLine("No Packets Recieved");
EverythingOK = false;
}
}
return EverythingOK;
}
Here is the part where I think the issue may lie, getting the response.
private byte[] ReadCamera(int ExpectedLength)
{
if(ExpectedLength < 1)
{
ExpectedLength = 100;
}
byte[] Buffer = new byte[ExpectedLength];
int read = 0;
int chunk;
while(stream.DataAvailable)
{
chunk = stream.Read(Buffer, read, Buffer.Length-read);
read += chunk;
// If we have reached the end of our buffer, check to see if theres any more information
if(read == Buffer.Length)
{
if(!stream.DataAvailable)
return Buffer;
// Nope. Resize the buffer, put the byte we've just read and continue
byte[] newBuffer = new byte[Buffer.Length * 2];
Array.Copy(Buffer, newBuffer, Buffer.Length);
Buffer = newBuffer;
//read++;
}
else if(!stream.DataAvailable)
return Buffer;
Thread.Sleep(50);
}
// Buffer is now too big, shrink it
byte[] ret = new byte[read];
Array.Copy(Buffer, ret, read);
return ret;
}
private string[] ProcessPackets(byte[] data)
{
// look for the stat char
List<string> Packets = new List<string>();
byte Start = 0x02;
bool StartFound = false;
byte End = 0x03;
StringBuilder sb = new StringBuilder();
for(int i =0; i<data.Length; i++)
{
if(StartFound)
{
// Check to see if its the end
if (data[i] == End)
{
Packets.Add(sb.ToString());
sb.Clear();
}
else
sb.Append(Encoding.ASCII.GetChars(data, i, 1));
}
// Find the start
if (!StartFound)
{
if (data[i] == Start)
{
StartFound = true;
}
}
}
return Packets.ToArray();
}
Let me explain what should happen and then what is happening. Through RetrieveDigits method I am sending "T1" to the camera. What the camera will do is respond with "T1" followed by 2 ascii characters, for now we will say AA. Using a diagnostic app on the camera system I can monitor the Ethernet and I see the following.
(>> means Received by Camera)
(<< means Sent from Camera)
>>[STX]T1[ETX]
<<[STX]T1[ETX][STX]AA[ETX]
So I see what the camera has sent. I confirmed the PC has has received the data using wire shark.
Now further down in the RetrieveDigits method you can see I process the packets received, loop through the packets until I find "T1" and then assume the next packet will be the data I am after and I set Digits to this value.
What I am finding is that sometimes when I run the app I see that Digits is set to "".
I am also finding that sometimes my data received will be "AA" then "T1" rather than "T1" then "AA". What I suspect is happening is that when its back to front, the "AA" is actually from the previous data sent from the camera and for some reason it was missed when reading from the stream.
Any idea why this could be happening as I am reading the data until Stream.Available is false.
Edit:
Modified the ReadCamera code to process the packets, reading 1 byte at a time.
private string[] ReadCamera(int ExpectedLength, int ExpectedPackets)
{
List<string> Packets = new List<string>();
bool StartFound = false;
StringBuilder sb = new StringBuilder();
if(ExpectedLength < 1)
{
ExpectedLength = 100;
}
byte[] Buffer = new byte[ExpectedLength];
int read = 0;
while (true)
{
read += stream.Read(Buffer, read, 1);
// Check to see if the byte read is the start of a packet
if (Buffer[read - 1] == StartOfPacket)
{
StartFound = true;
}
else if (StartFound)
{
// Check to see if the byte read is the end of a packet
if (Buffer[read - 1] == EndOfPacket)
{
Packets.Add(sb.ToString());
sb.Clear();
StartFound = false;
if (Packets.Count == ExpectedPackets)
break;
}
else
{
sb.Append(Encoding.ASCII.GetChars(Buffer, read - 1, 1));
}
}
}
// For Debuggin purposes
foreach(string s in Packets)
if (LogCameraEvent != null)
LogCameraEvent("<< " + s);
return Packets.ToArray();
}
and modified calling the method like so
// Check the response
string[] packets = ReadCamera(10,2);
// Process the packets
//string[] packets = ProcessPackets(buffer);

Related

Send following byte received via SerialPort (USB VCP) without delay

I am trying to test a flash memory chip on an electronic device having USB comms. Using SerialPort (USB VCP).
I am sitting in a for loop scanning through the memory addresses in sequence with a small delay following each read. However I want to get rid of the delay and have this run as follows:
C# App sends byte command to i.e. read address 0 and return 32 bytes (address 0 to 31)
When C# app receives the 32 bytes then immediately send request for address 31 etc
repeat...
So how do I do it in this fast manner and not having to have a delay giving the serial port time to receive the bytes but rather have the next request send immediately following receipt of previous request results ?
Update #2
async private void buttonMemoryTest_Click(object sender, EventArgs e)
{
byte[] bytes = new byte[6];
memoryAddress = 0x00000000;
bytes[0] = 0x90;
bytes[1] = 0x01;
try
{
sendStopWatch.Stop();
sendStopWatch.Reset();
sendStopWatch.Start();
//2097152
for (UInt32 counter = 0; counter < 100; counter++)
{
bytes[2] = (byte)(memoryAddress >> 24);
bytes[3] = (byte)(memoryAddress >> 16);
bytes[4] = (byte)(memoryAddress >> 8);
bytes[5] = (byte)(memoryAddress);
serialPortComms.Write(bytes, 0, 6);
await Task.Delay(1);
memoryAddress += 32;
time = sendStopWatch.Elapsed;
textBoxMemoryTestTime.Text = Math.Round(time.TotalSeconds, 2) + "s";
textBoxPageCounter.Text = Convert.ToString(counter);
}
}
catch (InvalidOperationException)
{
}
catch (TimeoutException)
{
}
}
void serialPortComms_DataReceived(object sender, SerialDataReceivedEventArgs e)
{
int dataLength = serialPortComms.BytesToRead;
byte[] data = new byte[dataLength];
int nbrDataRead = serialPortComms.Read(data, 0, dataLength);
if (nbrDataRead == 0)
return;
string newStr = "";
//See if there was data send back
try
{
if (selfTestClicked == 1)
{
if (selfTestResponseASCII == 1)
{
newStr = Encoding.ASCII.GetString(data, 0, data.Length);
}
else
{
newStr = BitConverter.ToString(data).Replace("-", " ");
}
textBoxReceiveByte_AppendText(newStr);
}
else
{
//
if (checkBoxASCII.Checked)
{
//newStr = Encoding.UTF8.GetString(data, 0, data.Length);
newStr = Encoding.ASCII.GetString(data, 0, data.Length);
}
else
{
newStr = BitConverter.ToString(data).Replace("-", " ");
}
if (checkBoxCR_LF.Checked)
{
newStr += "\r\n";
}
textBoxReceiveByte_AppendText(newStr);
//textBoxReceiveByte_ChangeText(newStr);
//processRxData(data);
}
}
catch (IndexOutOfRangeException)
{
}
}

Read from C# Stream without busy waiting

I need to read content from Stream in C#. I do not know which kind of Stream it would be. Let's assume it is Network Stream and network is very slow. So I can't read all bytes immediately. I know that 4 bytes in stream is integer and this int defines content length that we need to get from stream. How to do it without busy waiting (looping)?
private (int headerValue, bool isSuccess) ReadHeader()
{
var bytesRead = 0;
var headerBuffer = new byte[BufferHeaderLength];
var headerIsReady = false;
while (!headerIsReady)
{
try
{
var availableBytesToRead = Math.Min(BufferHeaderLength - bytesRead, underlyingStream.Length); // should be a number from 0 to 4
bytesRead += underlyingStream.Read(headerBuffer, bytesRead, (int)availableBytesToRead);
if (bytesRead == 0)
{
break;
}
}
catch (Exception)
{
break;
}
headerIsReady = bytesRead == BufferHeaderLength;
}
var headerValue = headerIsReady ? BitConverter.ToInt32(headerBuffer, 0) : 0;
return (headerValue, headerIsReady);
}

How to chunk and stream a file

I need to simulate a stream using a file...
This is what I was able to produce so far... the code works, but the streaming does not occur.
I have no experience with streams, so expect to find lot of bugs.
Stream chunkedStream = new MemoryStream();
var task = session.AnalyzeAsync(chunkedStream);
FillStream(chunkedStream);
while (!task.Wait(500))
{
Console.Write("\r" + ((float)chunkedStream.Position / 16000) + " Seconds");
if (!chunkedStream.CanWrite)
{
break;
}
}
private static async Task FillStream(Stream chunkedStream)
{
var stream = Properties.Resources.ResourceManager.GetStream("Sample");
byte[] chunk = new byte[16000];
while (true)
{
Thread.Sleep(1000);
int index = 0;
// There are various different ways of structuring this bit of code.
// Fundamentally we're trying to keep reading in to our chunk until
// either we reach the end of the stream, or we've read everything we need.
while (index < chunk.Length)
{
int bytesRead = stream.Read(chunk, index, chunk.Length - index);
if (bytesRead == 0)
{
break;
}
index += bytesRead;
}
if (index != 0) // Our previous chunk may have been the last one
{
// SendChunk(chunk, index); // index is the number of bytes in the chunk
chunkedStream.Write(chunk, 0, chunk.Length);
}
if (index != chunk.Length) // We didn't read a full chunk: we're done
{
chunkedStream.Close();
return;
}
}
}
public async Task AnalyzeAsync(Stream voiceStream)
{
IntervalHandler();
_StreamTask = UpStreamVoiceData(voiceStream);
await _StreamTask;
}
private async Task<Result<AnalysisResult>> UpStreamVoiceData(Stream voiceStream)
{
HttpWebRequest webRequest = WebRequestExtensions.CreateJsonPostRequest(_actions.upStream);
webRequest.ReadWriteTimeout = 1000000;
webRequest.Timeout = 10000000;
webRequest.SendChunked = true;
webRequest.AllowWriteStreamBuffering = false;
webRequest.AllowReadStreamBuffering = false;
using (var requeststream = webRequest.GetRequestStream())
{
await voiceStream.CopyStreamWithAutoFlush(requeststream);
requeststream.Close();
}
return webRequest.ReadJsonResponseAs<Result<AnalysisResult>>();
}

Why does my TCP application misses some packets when sent rapidly

I'm writing a multiple clients per server application in C# using async sockets. Each client on connection sends 10 items. The problem is, when starting lots of clients quickly, it appears that each client sent less than the 10 items and sometimes it sends nothing, the server just logs them connecting only.
The packet structure is the first 4 bytes are an int with size of data in them. Here's a part of server code. Each connected client has its own receive buffer which BeginReceive should write to.
private void Recieve(IAsyncResult iar) //Called when socket receives something.
{
Socket server_conn = (Socket)iar.AsyncState;
if (!SocketConnected(server_conn))
{
server_conn.Close();
logthis("Client Disconnected");
return;
}
int n = server_conn.EndReceive(iar); //Stop Receiving and parse data, n is number of bytes received
ClientData asdf = null;
foreach (ClientData cls in clientlist)
{
if (server_conn.RemoteEndPoint == cls.clientsock.RemoteEndPoint) //Who sent this data
{
asdf = cls; //cls is who sent this data
//Start a new thread and pass received bytes to it in order to be parsed
var t = new Thread(() => parse(cls, n,cls.recvbuffer));
t.Start();
Thread.Sleep(100);
break;
}
}
asdf.recvbuffer = new byte[1024]; //Clear buffer of client
server_conn.BeginReceive(asdf.recvbuffer, 0, asdf.recvbuffer.Length, SocketFlags.None, new AsyncCallback(Recieve), server_conn); //Start receiving again
}
private void parse(ClientData theclient, int nobytesreceived, byte[] bytesreceived)
{
ClientData cls = theclient;
int n = nobytesreceived;
byte[] receivedbytes = bytesreceived;
lock(s)
{
if (!cls.dataphase) //If there's no fragmented packets still waiting to be read
{
cls.dataphase = true;
byte[] sizeinbytes = new byte[4];
for (int i = 0; i < 4; i++)
{
sizeinbytes[i] = receivedbytes[i];
}
int size = BitConverter.ToInt32(sizeinbytes, 0); //Read first four bytes of packet to get size of data
if ((n - 4) == size) //If received number of bytes - 4 is equals to datasize
{
byte[] payload = new byte[size];
Array.Copy(receivedbytes, 4, payload, 0, (n - 4)); //Copy data to separately to payload array to be displayed to user
logthis(cls.clientsock.RemoteEndPoint.ToString());
logthis(Encoding.ASCII.GetString(payload));
cls.dataphase = false; //packet read successfully
}
else if ((n - 4) < size) //If received amount of bytes is less than data size (fragmented data)
{
cls.data = new byte[size];
for (int i = 4; i <= n - 4; i++)
{
cls.data[i - 4] += receivedbytes[i];
}
cls.datasize = size; //And cls.dataphase will remain true so it can be handled correctly the next time we receive something from same client
}
else if((n-4) > size) //If received amount of bytes is bigger than data (lumped packets)
{
byte[] payload = new byte[size];
byte[] otherpacket = new byte[(n - 4) - size];
for(int i = 0; i < size; i++)
{
payload[i] += receivedbytes[i + 4];
}
logthis(cls.clientsock.RemoteEndPoint.ToString());
logthis(Encoding.ASCII.GetString(payload));
Array.Copy(receivedbytes, (size + 4), otherpacket, 0, ((n - 4) - size));
receivedbytes = new byte[(n - 4) - size];
receivedbytes = otherpacket;
cls.dataphase = false;
parse(cls, ((n - 4) - size), receivedbytes); //Send rest of packet to read again
}
}
else
{
//not implemented, supposed to handle fragmented packets
if (n >= cls.datasize)
{
}
else if (n < cls.datasize)
{
}
}
}
}

Your problem comes from
else
{
//not implemented, supposed to handle fragmented packets
I would put money on the fact that you are hitting that else statement and loosing data. As soon as you don't have a complete read of a packet or two packets lumped together returned from your read function (which is a lot more common than you think) your client is now stuck in cls.dataphase = true; and will never get out of it.

Socket OnData event

I'm currently in a little bit of a pickle. I have some code that reads data from a socket whenever data is available but currently it is in a while loop on a separate thread and chews through 50% of the CPU when the function returns because of no data available. What I would really like is a blocking function for Sockets that blocks until data is available, or at least a OnData event that could be listened on. I originally converted this code from AS3 (Flash) but their sockets class has the OnData event I need... just in the wrong language.
I currently have this code in the code that handles a client connecting:
ServerThread = new Thread(() =>
{
while (server.Connected && ServerContinue)
{
ReceiveFromServer(server, client);
}
Disconnect(server, client, false);
});
ServerThread.Start();
And this is the code in ReceiveFromServer:
bool isReady = false;
int messageLength = 0;
int dataAvailable = 0;
UInt16 packetSize = 0;
byte[] temp = new byte[2];
do
{
dataAvailable = server.Available;
if (isReady)
{
if (dataAvailable >= messageLength)
{
byte[] temp1 = new byte[2000];
int bytesRead = server.Receive(temp1, 0, messageLength, SocketFlags.None);
byte[] data = new byte[bytesRead + 2];
Buffer.BlockCopy(temp1, 0, data, 2, messageLength);
Helpers.ByteArray tempo = data;
tempo.writeByte(temp[1]);
tempo.writeByte(temp[0]);
if (!VersionCheckPass)
{
Send(tempo, client);
return;
}
ServerPacketHandler(tempo, client);
messageLength = 0;
isReady = false;
temp = new byte[2];
}
}
else if(dataAvailable > 2)
{
server.Receive(temp, 0, 2, SocketFlags.None);
temp = temp.Reverse().ToArray();
packetSize = BitConverter.ToUInt16(temp, 0);
if (packetSize > 0)
{
messageLength = packetSize;
isReady = true;
}
}
}
while (dataAvailable > 2 && dataAvailable >= messageLength && ServerContinue);
But the issue here is that when dataAvailable is 0 the function simply returns, and then RecevieFromServer is called again in the thread. This means that alot of the CPU is used by simply calling ReceiveFromServer and then returning again.
I currently have Thread.Sleep(10) after ReceiveFromServer in the ServerThread but this is inefficient. So my question is, Is there a way to block until data is available or is there an event that I can handle? Or does anyone else have any suggestions on how to do the same thing I am currently doing but it doesn't loop endlessly whilst there is no data available.

Found a really easy (and obvious) solution to block until data is available. Call Socket.Receive with a receive size of 0. The socket blocks until there is data to receive, then reads 0 bytes from the socket, and unblocks. Its really quite marvelous :) Heres how I implemented it:
ServerThread = new Thread(() =>
{
byte[] zero = new byte[0];
while (Server.Connected && ServerContinue)
{
server.Receive(zero, 0, SocketFlags.None);
ReceiveFromServer(server, client);
}
Disconnect(server, client, false);
});
Thanks for all the help.
Josh

There is not a lot of re-write needed. Your code looks like it's just receiving the message and then passing off to another routine to process it.
My reply to this thread pretty much covers what you're wanting to do:
C# Sockets and Multithreading
My socketReadCallBack function is:
private void OnDataReceived(IAsyncResult asyn)
{
ReceiveState rState = (ReceiveState)asyn.AsyncState;
Socket client = rState.Client;
SocketError socketError = SocketError.TypeNotFound;
if (!client.Connected)
{
// Not Connected anymore ?
return;
}
_LastComms = DateTime.Now;
_LastIn = _LastComms;
int dataOffset = 0;
int restOfData = 0;
int dataRead = 0;
Boolean StreamClosed = false;
long rStateDataLength = 0;
long LastrStateDataLength = 0;
try
{
dataRead = client.EndReceive(asyn, out socketError);
}
catch (Exception excpt)
{
// Handle error - use your own code..
}
if (socketError != SocketError.Success)
{
// Has Connection been lost ?
OnConnectionDropped(client);
return;
}
if (dataRead <= 0)
{
// Has connection been lost ?
OnConnectionDropped(client);
return;
}
while (dataRead > 0)
{
//check to determine what income data contain: size prefix or message
if (!rState.DataSizeReceived)
{
//there is already some data in the buffer
if (rState.Data.Length > 0)
{
restOfData = PrefixSize - (int)rState.Data.Length;
rState.Data.Write(rState.Buffer, dataOffset, restOfData);
dataRead -= restOfData;
dataOffset += restOfData;
}
else if (dataRead >= PrefixSize)
{ //store whole data size prefix
rState.Data.Write(rState.Buffer, dataOffset, PrefixSize);
dataRead -= PrefixSize;
dataOffset += PrefixSize;
}
else
{ // store only part of the size prefix
rState.Data.Write(rState.Buffer, dataOffset, dataRead);
dataOffset += dataRead;
dataRead = 0;
}
if (rState.Data.Length == PrefixSize)
{ //we received data size prefix
rState.DataSize = BitConverter.ToInt32(rState.Data.GetBuffer(), 0);
rState.DataSizeReceived = true;
//reset internal data stream
rState.Data.Position = 0;
rState.Data.SetLength(0);
}
else
{ //we received just part of the prefix information
//issue another read
client.BeginReceive(rState.Buffer, 0, rState.Buffer.Length,
SocketFlags.None, new AsyncCallback(socketReadCallBack),
rState);
return;
}
}
//at this point we know the size of the pending data
// Object disposed exception may raise here
try
{
rStateDataLength = rState.Data.Length;
LastrStateDataLength = rStateDataLength;
}
catch (ObjectDisposedException Ode)
{
StreamClosed = true;
}
if (!StreamClosed)
{
if ((rStateDataLength + dataRead) >= rState.DataSize)
{ //we have all the data for this message
restOfData = rState.DataSize - (int)rState.Data.Length;
rState.Data.Write(rState.Buffer, dataOffset, restOfData);
//Console.WriteLine("Data message received. Size: {0}",
// rState.DataSize);
// Is this a heartbeat message ?
if (rState.Data.Length == 2)
{
// Yes
HeartBeatReceived();
}
else
{
//charArray = new char[uniEncoding.GetCharCount(
//byteArray, 0, count)];
//uniEncoding.GetDecoder().GetChars(
// byteArray, 0, count, charArray, 0);
//Console.WriteLine(charArray);
//rState.Data.Position = 0;
DecodeMessageReceived(GetStringFromStream(rState.Data));
}
dataOffset += restOfData;
dataRead -= restOfData;
//message received - cleanup internal memory stream
rState.Data = new MemoryStream();
rState.Data.Position = 0;
rState.DataSizeReceived = false;
rState.DataSize = 0;
if (dataRead == 0)
{
//no more data remaining to process - issue another receive
if (_IsConnected)
{
client.BeginReceive(rState.Buffer, 0, rState.Buffer.Length,
SocketFlags.None, new AsyncCallback(socketReadCallBack),
rState);
return;
}
}
else
continue; //there's still some data to process in the buffers
}
else
{ //there is still data pending, store what we've
//received and issue another BeginReceive
if (_IsConnected)
{
rState.Data.Write(rState.Buffer, dataOffset, dataRead);
client.BeginReceive(rState.Buffer, 0, rState.Buffer.Length,
SocketFlags.None, new AsyncCallback(socketReadCallBack), rState);
dataRead = 0;
}
}
}
else
{
// Stream closed, but have we read everything ?
if (LastrStateDataLength + dataRead == rState.DataSize)
{
// We're equal, get ready for more
//no more data remaining to process - issue another receive
if (_IsConnected)
{
client.BeginReceive(rState.Buffer, 0, rState.Buffer.Length,
SocketFlags.None, new AsyncCallback(socketReadCallBack),
rState);
}
return;
}
else
{
// We should have more..
// Report Error
}
}
// If we've been disconnected, provide a graceful exit
if (!_IsConnected)
dataRead = -1;
}
}
I've got a few more things in here than you need like provision for a heartbeat message and raising events on connection dropped etc.

Categories

Resources