What I'm doing is taking a user entered string, creating a packet with the data, then sending the string out to a serial port. I am then reading the data I send via a loopback connector. My send is working flawlessly, however my receive is randomly throwing an arithmetic overflow exception.
I say randomly because it is not happening consistently. For example, I send the message "hello" twice. The first time works fine, the second time outputs nothing and throws an exception. I restart my program, run the code again, and send hello only to receive "hell" and then an exception. On rare occasion, I'll receive the packet 3 or 4 times in a row without error before the exception.
Here is my relevant code:
public void receivePacket(object sender, SerialDataReceivedEventArgs e)
{
byte[] tempByte = new byte[2];
int byteCount = 0;
while (serialPort1.BytesToRead > 0)
{
if (byteCount <= 1)
{
tempByte[byteCount] = (byte)serialPort1.ReadByte();
}
if (byteCount == 1)
{
receivedString = new byte[tempByte[byteCount]];
receivedString[0] = tempByte[0];
receivedString[1] = tempByte[1];
}
else if (byteCount > 1)
{
byte b = (byte)serialPort1.ReadByte();
receivedString[byteCount] = b;
}
byteCount++;
}
int strLen = (byteCount - 3);
tempByte = new byte[strLen];
int newBit = 0;
for (int i = 2; i <= strLen+1; i++)
{
tempByte[newBit] = receivedString[i];
newBit++;
}
string receivedText = encoder.GetString(tempByte);
SetText(receivedText.ToString());
}
I'm well aware that my implementation using byteCount (which I use to traverse the byte array) is rather sloppy. When I step through the code, I find that when I get the error byteCount == 1, which is making strLen a negative number (since strLen is byteCount - 3, which is done because the packet contains a header, length, and CRC i.e. byteCount - 3 == # of actual data bytes received). This leads to by tempByte having a size of -2, which throws my exceptions. I, however, am having a very hard time figuring out why byteCount is being set to 1.
The code after this basically just traverses the data section of the array, copies it into the tempByte, then is sent off to a function to append the text in another thread.
I am guessing that byteCount is 1 because you only received one byte - or rather, you processed the first byte before the second one arrived in the buffer.
The ReadByte function will wait for a certain amount of time for a byte to arrive if there isn't one waiting.
Maybe if instead of checking BytesToRead, you did something more like this:
byte headerByte = serialPort1.ReadByte();
byte length = serialPort1.ReadByte();
receivedString = new byte[length];
receivedString[0] = headerByte;
receivedString[1] = length;
for (int i = 2; i < length; i++) {
receivedString[i] = serialPort1.ReadByte();
}
Related
I am working with filestream read: https://msdn.microsoft.com/en-us/library/system.io.filestream.read%28v=vs.110%29.aspx
What I'm trying to do is read a large file in a loop a certain number of bytes at a time; not the whole file at once. The code example shows this for reading:
int n = fsSource.Read(bytes, numBytesRead, numBytesToRead);
The definition of "bytes" is: "When this method returns, contains the specified byte array with the values between offset and (offset + count - 1) replaced by the bytes read from the current source."
I want to only read in 1 mb at a time so I do this:
using (FileStream fsInputFile = new FileStream(strInputFileName, FileMode.Open, FileAccess.Read)) {
int intBytesToRead = 1024;
int intTotalBytesRead = 0;
int intInputFileByteLength = 0;
byte[] btInputBlock = new byte[intBytesToRead];
byte[] btOutputBlock = new byte[intBytesToRead];
intInputFileByteLength = (int)fsInputFile.Length;
while (intInputFileByteLength - 1 >= intTotalBytesRead)
{
if (intInputFileByteLength - intTotalBytesRead < intBytesToRead)
{
intBytesToRead = intInputFileByteLength - intTotalBytesRead;
}
// *** Problem is here ***
int n = fsInputFile.Read(btInputBlock, intTotalBytesRead, intBytesToRead);
intTotalBytesRead += n;
fsOutputFile.Write(btInputBlock, intTotalBytesRead - n, n);
}
fsOutputFile.Close(); }
Where the problem area is stated, btInputBlock works on the first cycle because it reads in 1024 bytes. But then on the second loop, it doesn't recycle this byte array. It instead tries to append the new 1024 bytes into btInputBlock. As far as I can tell, you can only specify the offset and length of the file you want to read and not the offset and length of btInputBlock. Is there a way to "re-use" the array that is being dumped into by Filestream.Read or should I find another solution?
Thanks.
P.S. The exception on the read is: "Offset and length were out of bounds for the array or count is greater than the number of elements from index to the end of the source collection."
Your code can be simplified somewhat
int num;
byte[] buffer = new byte[1024];
while ((num = fsInputFile.Read(buffer, 0, buffer.Length)) != 0)
{
//Do your work here
fsOutputFile.Write(buffer, 0, num);
}
Note that Read takes in the Array to fill, the offset (which is the offset of the array where the bytes should be placed, and the (max) number of bytes to read.
That's because you're incrementing intTotalBytesRead, which is an offset for the array, not for the filestream. In your case it should always be zero, which will overwrite previous byte data in the array, rather than append it at the end, using intTotalBytesRead.
int n = fsInputFile.Read(btInputBlock, intTotalBytesRead, intBytesToRead); //currently
int n = fsInputFile.Read(btInputBlock, 0, intBytesToRead); //should be
Filestream doesn't need an offset, every Read picks up where the last one left off.
See https://msdn.microsoft.com/en-us/library/system.io.filestream.read(v=vs.110).aspx
for details
Your Read call should be Read(btInputBlock, 0, intBytesToRead). The 2nd parameter is the offset into the array you want to start writing the bytes to. Similarly for Write you want Write(btInputBlock, 0, n) as the 2nd parameter is the offset in the array to start writing bytes from. Also you don't need to call Close as the using will clean up the FileStream for you.
using (FileStream fsInputFile = new FileStream(strInputFileName, FileMode.Open, FileAccess.Read))
{
int intBytesToRead = 1024;
byte[] btInputBlock = new byte[intBytesToRead];
while (fsInputFile.Postion < fsInputFile.Length)
{
int n = fsInputFile.Read(btInputBlock, 0, intBytesToRead);
intTotalBytesRead += n;
fsOutputFile.Write(btInputBlock, 0, n);
}
}
I want to communicate with a DSP using RS232, so I use System.IO.SerialPort to achieve this. Everything goes well except the reading performance.
Every 200ms, the port can received a package of 144 bytes. But in the tests, the applications almost skip every other package. I try to print the system time in the console. It amaze me that the code below (when length = 140) take me over 200ms. It let the application can not handle the data in time.
Does anything wrong I do?
Port Property:
BaudRate = 9600
Parity = None
StopBits = One
private byte[] ReadBytesInSpicifiedLength(int length)
{
byte[] des = new byte[length];
for (int i = 0; i < length; i++)
{
des[i] = (byte)serialPort.ReadByte();
}
return des;
}
You're doing a lot of individual I/O calls, which means a lot of kernel transitions. Those are expensive. Not being able to reach 720 bytes per second is surprising, but you can make the data handling an order of magnitude faster by doing block reads:
private byte[] ReadBytesWithSpecifiedLength(int length)
{
byte[] des = new byte[length];
serialPort.BaseStream.Read(des, 0, des.Length);
return des;
}
If you have timeouts enabled, you could get partial reads. Then you need to do something like:
private byte[] ReadBytesWithSpecifiedLength(int length)
{
byte[] des = new byte[length];
int recd = 0;
do {
int partial = serialPort.BaseStream.Read(des, recd, length - recd);
if (partial == 0) throw new IOException("Transfer Interrupted");
recd += partial;
} while (recd < length);
return des;
}
The nice thing about BaseStream is that it also has async support (via ReadAsync). That's what new C# code should be using.
I'm trying to send a long string from a python server to a C# client. The string is 230400 bytes long. I'm both sending and receiving in chunks of 64 bytes. Server code:
import socket
def initialize():
global s
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
s.bind(('', 1719))
s.listen()
initialize()
while(1):
sock, addr = s.accept()
msgstr = generate_msg_string() # irrelevant
msglen = len(msgstr)
totalsent = 0
while totalsent < msglen:
sent = sock.send(msgstr[totalsent:totalsent+64])
totalsent = totasent + sent
sock.close()
Client code:
TcpClient tcpClient = new TcpClient();
tcpClient.Connect(ip, 1719);
byte[] ba = new byte[230400];
byte[] buffer = new byte[64];
tcpClient.ReceiveBufferSize = 64
int i=0;
while(i != 230400)
{
stream.Read(buffer, 0, 64);
buffer.CopyTo(ba, i);
i += 64;
}
tcpClient.Close();
I've checked a few connections in a row - the first 1523 bytes are correct and all the rest are gibberish - at least seemingly random.
Any idea what might be the cause?
while(i != 230400)
{
stream.Read(buffer, 0, 64);
buffer.CopyTo(ba, i);
i += 64;
}
The fundamental error here is assuming that Read read 64 bytes. It can read any of:
0 if the socket gets closed for any reason
64 bytes if that happens to be available and it chooses to
1-63 bytes, just for fun
You are not guaranteed anything other than "non-positive if the stream is closed, else at least 1 byte and no more than 64 bytes"
You must must must read the return value of Read and only process that much of the buffer. This remains the case if you switch to Socket.Receive, by the way.
Also - why don't you just fill ba in the first place, increment in the offset and decrementing the count each time?
int count = 230400, offset = 0, read;
byte[] ba = new byte[count];
while(count > 0 && (read=stream.Read(ba, offset, count)) > 0)
{
offset += read;
count -= read;
}
if(read!=0) throw new EndOfStreamException();
It seems that I hurried with the question.
Changing TcpClient to Socket fixed the problem. The approach remained the same.
I've written an application which listens to a port and receives some packets,according to my customized protocol, the packets are either 49 byte to 1500 byte, which i can tell from data length in the packet. the way i should interpret and deal with data in 49 byte packets and bigger packets are different.
The problem is that when i receive packets less than 1374 byte everything is ok, but when the packet length gets more, i receive the following exception and i also lose 4 last bytes of my data(i've tested with a 1384byte packet and i lost the last 4 bytes)
Exception which is raised:
Index was out of range. Must be non-negative and less than the size of the collection.
Parameter name: startIndex
each 49 byte packet has 35 byte of data, and the data length of bigger packets are non-deterministic(because of compression).
i found out sometimes the last 4 bytes are in a seperate "bytes" and "result" variables,meaning they are being treated like new packets and are not being attached to the packet they belong to.
here's the code for receiving data:
TcpClient Client = obj as TcpClient;
EndPoint ep = Client.Client.RemoteEndPoint;
List<Byte> result = new List<byte>();
result.Capacity = 2000;
try
{
NetworkStream stream = Client.GetStream();
int i = 49;
while ((i = stream.Read(bytes, 0,49)) != 0)
{
for (int id = 0; id < i; id++)
{
result.Add(bytes[id]);
}
//reading data length to determine packet length
byte[] tmp = new byte[2];
tmp = BitConverter.GetBytes(BitConverter.ToUInt16(result.ToArray(), 9));
if (BitConverter.IsLittleEndian)
{
Array.Reverse(tmp);
}
Int16 l = BitConverter.ToInt16(tmp, 0);
if (l>35)
{
stream.Read(bytes, result.Count, l - 35);
for (int id = 49; id <((l-35)+49); id++)
{
result.Add(bytes[id]);
}
if (this.TCPDataReceivedHandler != null)
{
this.TCPDataReceivedHandler(ep, result.Count, result.ToArray());
result.Clear();
Array.Clear(bytes, 0, 2000);
result.Capacity = 2000;
}
}
else
{
if (this.TCPDataReceivedHandler != null)
{
this.TCPDataReceivedHandler(ep, result.Count, result.ToArray());
result.Clear();
Array.Clear(bytes, 0, 2000);
result.Capacity = 2000;
}
}
}
System.Diagnostics.Debug.WriteLine("client Close");
Client.Close();
}
catch (System.Exception ex)
{
throw ex;
}
finally
{
Client.Close();
this.clients.Remove(Client);
}
According to Greg Suggestion and my researches,i also tried using following method:
NetworkStream stream = Client.GetStream();
int bytesread = 0, OffsetTemp = 0;
while (stream.CanRead)
{
OffsetTemp = 0;
bytesread += stream.Read(bytess, OffsetTemp, 11);
OffsetTemp = OffsetTemp + 11;
byte[] tmp = new byte[2];
tmp = BitConverter.GetBytes(BitConverter.ToUInt16(bytess.ToArray(), 9));
if (BitConverter.IsLittleEndian)
{
Array.Reverse(tmp);
}
Int16 l = BitConverter.ToInt16(tmp, 0);
bytesread += stream.Read(bytess, OffsetTemp++, 11 + l + 3);
for (int id = 0; id < l + 14; id++)
{
result.Add(bytess[id]);
}
if (this.TCPDataReceivedHandler != null)
{
this.TCPDataReceivedHandler(ep, result.Count, result.ToArray());
result.Clear();
Array.Clear(bytess, 0, 2000);
}
}
When using TCP, you must keep in mind that the size of the data blocks you put in is not guaranteed to be the same size of data blocks you get out on the receiving side. TCP is a stream protocol, so it guarantees that the same bytes get to the other side in the same order as they were sent (if not, the socket connection will be reset). TCP does not maintain any kind of block boundaries between calls to send(), and blocks may be arbitrarily split or coalesced depending on network conditions.
Your receiver must be prepared to handle receiving any amount of data from the calls to stream.Read(), but your code does not appear to do this. For example, correctly written receiver code should continue to work correctly even if stream.Read() receives only one byte at a time.
I have a C# client/server network program I've written using
TCPListener and TCPClient classes. The server is reading everything
from the client (small amounts of xml) just fine until I try to send a
large file (100k) back to the client.
I'm using stream functions for
both client and server with non-blocking socket functions. When I do a
socket.SendFile("filename") back to the client, the file is getting
cut off - I've set the receive buffer size on the client to well past
100k but it still gets cut off around 25k and the communication
between client and server is unreliable afterwords.
My basic question
is what happens if data is somehow left in the pipe ? i.e.. will it be
read by the next socket.Read... Does every Send call require exactly
one and only one Read ? Maybe I'm not giving the client enough time to
read the file but their both on the same machine and I've tried
sleeping for a few seconds in various places w/o success.
It is very possible that you cannot read the entire message through one Read call (perhaps all data has not arrived yet). In network programming, you would often place the call to Read in a while loop and simply Read() until you have received the entire expected message.
1 Send call might take more than one Read call to receive, and 1 Read call might read the data send by several Send call.
TCP just provides a stream, so it's up to you to define how the data or messages you send are partitioned.
In this case, you probably just need to loop ,doing Read until the stream is closed.
You probably want something like this:
socket.Blocking = false;
const int ChunkSize = 1492;
const int ReceiveTimeout = 10000;
const int SendTimeout = 10000;
public void Send(Byte[] data)
{
var sizeBytes = BitConverter.GetBytes(data.Length);
SendInternal(sizeBytes);
SendInternal(data);
}
public Byte[] Receive()
{
var sizeBytes = ReceiveInternal(4);
var size = BitConverter.ToInt32(sizeBytes, 0);
var data = ReceiveInternal(size);
return data;
}
private void SendInternal(Byte[] data)
{
var error = SocketError.Success;
var lastUpdate = Environment.TickCount;
var size = data.Length;
var count = 0;
var sent = 0;
while (sent < size)
{
count = Math.Min(ChunkSize, size - sent);
count = socket.Send(data, sent, count, SocketFlags.None, out error);
if (count > 0)
{
sent += count;
lastUpdate = Environment.TickCount;
}
if (error != SocketError.InProgress && error != SocketError.Success && error != SocketError.WouldBlock)
throw new SocketException((Int32)error);
if (Environment.TickCount - lastUpdate > SendTimeout)
throw new TimeoutException("Send operation timed out.");
if (count == 0 && !socket.Poll(100, SelectMode.SelectWrite))
throw new SocketException((Int32)SocketError.Shutdown);
}
}
private Byte[] ReceiveInternal(Int32 size)
{
var error = SocketError.Success;
var lastUpdate = Environment.TickCount;
var buffer = new Byte[ChunkSize];
var count = 0;
var received = 0;
using (var ms = new MemoryStream(size))
{
while (received < size)
{
count = Math.Min(ChunkSize, size - received);
count = socket.Receive(buffer, 0, count, SocketFlags.None, out error);
if (count > 0)
{
ms.Write(buffer, 0, count);
received += count;
lastUpdate = Environment.TickCount;
}
if (error != SocketError.InProgress && error != SocketError.Success && error != SocketError.WouldBlock)
throw new SocketException((Int32)error);
if (Environment.TickCount - lastUpdate > ReceiveTimeout)
throw new TimeoutException("Receive operation timed out.");
if (count == 0 && socket.Poll(100, SelectMode.SelectRead) && socket.Available == 0)
throw new SocketException((Int32)SocketError.Shutdown);
}
return ms.ToArray();
}
}
What I would usually do is create a header structure that is sent
Header Size (int, 4 bytes)
File Name Offset (int, 4 bytes)
File Name Size (int , 4 bytes)
File Data Offset (int, 4 bytes)
File Data Size (int , 4 bytes)
[ message data here]
and then that header is read using either a BinaryReader or copying the bytes to a struct using Marshal. This way you always know what data is arriving and how many times you need to call Read().
The header size field is also helps with versioning the protocol (keep the structure the same but add to it for later clients so you can keep backwards compatibility). If you define the structure in C# be sure to do it like so:
[StructLayout LayoutKind.Sequential]
struct MessageHeader
{
public int HeaderSize;
public int FileNameOffset;
public int FileNameSize;
public int FileDataOffset;
public int FileDataSize;
}
Then Marshal.PtrToStructure will allow you do create an instance of this struct right from the byte[] you read from the socket
Try sending the chunk from the server side in chunks. Just as the other said, posting the code would be of great help to us.