Currently, I'm receiving data whenever it is 17 bytes. However, I have two types of data, 17 bytes and 10 bytes. How can I make it process when I have two types of data?
byte[] message = new byte[17];
int bytesRead;
while (true)
{
bytesRead = 0;
try
{
//blocks until a client sends a message
bytesRead = clientStream.Read(message, 0, 17);
}
catch
{
//a socket error has occured
break;
}
if (bytesRead == 0)
{
//the client has disconnected from the server
break;
}
I have seen similar questions asked but it's in C and I couldn't understand. Kindly assist me.
You are trying to implement a message exchange on top of a stream-based protocol (like TCP). When the messages have different lengths and/or types, there are two common approaches
framed messages: Each message will consist of a header of known length that contains the message's length and type and possibly other metadata (e.g. a timestamp). After reading the header, the appropriate amount of bytes (i.e. the payload) is read from the stream.
self-delimiting messages: The end of a message can be detected by the content of the stream read so far. One example for self-delimiting is the HTTP Header, which is delimited by a double newline (2x CRLF).
IMHO framed messaging is easier to implement since you always know how many bytes to read. For self-delimiting messages you have to employ buffering and parsing to detect the end of the message. Furthermore you have to make sure that the end-of-message-marker does not appear in the message's payload.
To implement the receiving side of a framed messaging protocol you can use the System.IO.BinaryReader class.
read the length of the message using ReadByte() or one of the ReadUInt*() methods if the messages will become longer than 255 bytes
read the payload using Read(Byte[], Int32, Int32). Please note that Read may return even if less bytes than specified have been read. You have to use a loop to fill byte[] message.
Related
I am writing a Small HttpServer, sometime I encounter a problem with missing POST Data.
By using Wireshark I discovered, that the Header is split into two segments.
I only get the first segment (636 Bytes), the second one (POST Data in this case) gets totally lost.
Here is a the relevant C# Code
string requestHeaderString = "";
StreamSocket socketStream = args.Socket;
IInputStream inputStream = socketStream.InputStream;
byte[] data = new byte[BufferSize];
IBuffer buffer = data.AsBuffer();
try
{
await inputStream.ReadAsync(buffer, BufferSize, InputStreamOptions.Partial);
// This is where things go missing, buffer.ToArray() should be 678 Bytes long,
// so Segment 1 (636 Bytes) and Segment 2 (42 Bytes) combined.
// But is only 636 Bytes long, so just the first Segment?!
requestHeaderString += Encoding.UTF8.GetString(buffer.ToArray());
}
catch (Exception e)
{
Debug.WriteLine("inputStream is not readable" + e.StackTrace);
return;
}
This code is in part of the StreamSocketListener ConnectionReceived Event.
Do I manually have to reassemble the TCP Segments, isn't this what the Systems TCP Stack should do?
Thanks,
David
The problem is the systems TCP stack treats the TCP stream just like any other stream. You don't get "messages" with streams, you just get a stream of bytes.
The receiving side has no way to tell when one "message" ends and where the next begins without you telling it some how. You must implement message framing on top of TCP, then on your receiving side you must repeatedly call Receive till you have received enough bytes to form a full message (this will involve using the int returned from the receive call to see how many bytes where processed).
Important note: If you don't know how many bytes you are expecting to get in total, for example you are doing message framing by using '\0' to seperate messages you may get the end of one message and the start of the next in a single Receive call. You will need to handle that situation.
EDIT: Sorry, I skipped over the fact you where reading HTTP. You must follow the protocol of HTTP. You must read in data till you see the pattern \r\n\r\n, once you get that you must parse the header and decode how much data is in the content portion of the HTTP message then repeatatly call read till you have read the number of bytes needed.
I have a 3Rd party server where I send it data and then wait for a response.
It will first respond with ack then with the full response
I am doing something like this. Is there a better way.
//Connect
eftSocket.Connect(remoteEp);
// Encode the data string into a byte array.
byte[] msg = Encoding.ASCII.GetBytes(strRequest);
// Send the data through the socket.
int bytesSent = eftSocket.Send(msg);
//receive first time
int bytesRec = eftSocket.Receive(bytes);
//Do we get an ack,
if (bytes[0] != 6)
{
throw new Exception("Could not connect to EFT terminal");
}
//receive 2nd time
bytesRec = eftSocket.Receive(bytes);
That code looks broken. At first you have to embrace that TCP is a stream and that there are no packets. That means even if the client sends something of a given length you can never expect that you receive it in one turn. Which means if you do eftSocket.Receive(bytes) you might receive any amount of bytes which were sent from the remote, not necessarily the ack byte in the first call and the remaining stuff in a second call. The only thing that you know is that you won't receive more than bytes.Length with it. If you hand it over a significantly large buffer you might receive everything (ACK + remaining data) in one turn. Therefore you should always check the return value first. And if you expect more bytes repeat the Receive calls with the required offsets.
Then you should at first check whether your receive suceeded at all. You might receive 0, which means the remote closed the socket. If that's the case bytes[0] will yield anything which was stored in it before the Receive started.
You should also properly close your socket. It's not obvious from the code example if it happens, but if you return from the code with an Exception it might be missing.
This question already has answers here:
Receiving data in TCP
(10 answers)
Closed 2 years ago.
I have written a simple TCP client and server. The problem lies with the client.
I'm having some trouble reading the entire response from the server. I must let the thread sleep to allow all the data be sent.
I've tried a few times to convert this code into a loop that runs until the server is finished sending data.
// Init & connect to client
TcpClient client = new TcpClient();
Console.WriteLine("Connecting.....");
client.Connect("192.168.1.160", 9988);
// Stream string to server
input += "\n";
Stream stm = client.GetStream();
ASCIIEncoding asen = new ASCIIEncoding();
byte[] ba = asen.GetBytes(input);
stm.Write(ba, 0, ba.Length);
// Read response from server.
byte[] buffer = new byte[1024];
System.Threading.Thread.Sleep(1000); // Huh, why do I need to wait?
int bytesRead = stm.Read(buffer, 0, buffer.Length);
response = Encoding.ASCII.GetString(buffer, 0, bytesRead);
Console.WriteLine("Response String: "+response);
client.Close();
The nature of streams that are built on top of sockets is that you have an open pipeline that transmits and receives data until the socket is closed.
However, because of the nature of client/server interactions, this pipeline isn't always guaranteed to have content on it to be read. The client and server have to agree to send content over the pipeline.
When you take the Stream abstraction in .NET and overlay it on the concept of sockets, the requirement for an agreement between the client and server still applies; you can call Stream.Read all you want, but if the socket that your Stream is connected to on the other side isn't sending content, the call will just wait until there is content.
This is why protocols exist. At their most basic level, they help define what a complete message that is sent between two parties is. Usually, the mechanism is something along the lines of:
A length-prefixed message where the number of bytes to be read is sent before the message
A pattern of characters used to mark the end of a message (this is less common depending on the content that is being sent, the more arbitrary any part of the message can be, the less likely this will be used)
That said you aren't adhering to the above; your call to Stream.Read is just saying "read 1024 bytes" when in reality, there might not be 1024 bytes to be read. If that's the case, the call to Stream.Read will block until that's been populated.
The reason the call to Thread.Sleep probably works is because by the time a second goes by, the Stream has 1024 bytes on it to read and it doesn't block.
Additionally, if you truly want to read 1024 bytes, you can't assume that the call to Stream.Read will populate 1024 bytes of data. The return value for the Stream.Read method tells you how many bytes were actually read. If you need more for your message, then you need to make additional calls to Stream.Read.
Jon Skeet wrote up the exact way to do this if you want a sample.
Try to repeat the
int bytesRead = stm.Read(buffer, 0, buffer.Length);
while bytesRead > 0. It is a common pattern for that as i remember.
Of course don't forget to pass appropriate params for buffer.
You dont know the size of data you will be reading so you have to set a mechanism to decide. One is timeout and another is using delimiters.
On your example you read whatever data from just one iteration(read) because you dont set the timeout for reading and using default value thats "0" milisecond. So you have to sleep just 1000 ms. You get same effect with using recieve time out to 1000 ms.
I think using lenght of data as prefix is not the real solution because when socket is closed by both sides, socket time-wait situation can not handled properly. Same data can be send to server and cause server to get exception . We used prefix-ending character sequence. After every read we check the data for start and end character sequence, if we cant get end characters, we call another read. But of course this works only if you have the control of server side and client side code.
In the TCP Client / Server I just wrote I generate the packet I want to send to a memory stream, then take the length of that stream and use it as a prefix when sending the data. That way the client knows how many bytes of data it's going to need to read for a full packet.
Is there a way to determine the receive buffer size of a TCPIP socket in c#. I am sending a message to a server and expecting a response where I am not sure of the receive buffer size.
IPEndPoint ipep = new IPEndPoint(IPAddress.Parse("192.125.125.226"),20060);
Socket server = new Socket(AddressFamily.InterNetwork,
SocketType.Stream, ProtocolType.Tcp);
server.Connect(ipep);
String OutStr= "49|50|48|48|224|48|129|1|0|0|128|0|0|0|0|0|4|0|0|32|49|50";
byte[] temp = OutStr.Split('|').Select(s => byte.Parse(s)).ToArray();
int byteCount = server.Send(temp);
byte[] bytes = new byte[255];
int res=0;
res = server.Receive(bytes);
return Encoding.UTF8.GetString(bytes);
The size of the buffer used to receive data is application or protocol dependent. There's no way within the language to tell what the size of your receive buffer ought to be. Nor is there any socket function that can be used that says 'you need a 23867 bytes to receive this message'. In general your application has to work out from the protocol what size the receive buffer should be and how to handle this. Typically a protocol will either:
specify the number of bytes in the message.
specify a terminating character (for example hdlc using 0x7e to indicate the end of the message)
A consequence of this is that your application might need to deal with split messages. For example the server might send a message that is 2000 bytes but your receive buffer is only 1000 bytes, you'll have to write some code to maintain the state telling you if you've completed a message or are partially complete.
TCP is a stream of bytes. It knows nothing of your concept of messages.
As such it's up to you to provide the necessary message framing information within that stream of bytes. Common ways to do this include prefixing the message with a header which contains the total length of the message or terminating the message with a character that cannot otherwise appear in a valid message.
I speak about TCP message framing here: http://www.serverframework.com/asynchronousevents/2010/10/message-framing-a-length-prefixed-packet-echo-server.html though it's in reference to C++ code so it might not be any use to you.
It's usually slightly more performant for a message consumer to deal with length prefixed messages and it's often slighly more performant for a message producer to produce character delimited messages. Personally I prefer length prefixed messages wherever possible.
With a length prefixed message you would first send x bytes of data which are the length of the message, the peer would then know that it always has to read at least x bytes to work out the length and from that point it knows the size of the resulting message and can read until it has that many bytes.
With character delimited messages you simply keep reading and scanning all of the data that you have read until you find the message delimiter. You have then got a whole message, and possibly more data (part of the next message?) in the buffer to process after that.
I'm trying to write a simple SNPP (Simple Network Paging Protocol) client using sockets. Everything seems to be working well, except for a small inconsistency between servers.
When I send a command, I need to read the reply, which is usually a single chunk of data. However, Sprint's SNPP server sends replies in two parts. The first chunk of data is the first digit of the status code. The second chunk is the remainder. For example, when I attempt to receive the "220 Gateway ready" reply, it arrives like this:
2
I have to send another empty command to retrieve the rest:
20 Gateway ready
For the moment, I'm using:
byte[] buffer = new byte[256];
socket.Receive(buffer);
How can I make sure that I receive all of the available data after issuing a command without allocating a separate buffer for each chunk of data?
For chunked responses I would recommend you reading data like this:
using (var resultStream = new MemoryStream())
{
const int CHUNK_SIZE = 2 * 1024; // 2KB, could be anything that fits your needs
byte[] buffer = new byte[CHUNK_SIZE];
int bytesReceived;
while ((bytesReceived = socket.Receive(buffer, buffer.Length, SocketFlags.None)) > 0)
{
byte[] actual = new byte[bytesReceived];
Buffer.BlockCopy(buffer, 0, actual, 0, bytesReceived);
resultStream.Write(actual, 0, actual.Length);
}
// Do something with the resultStream, like resultStream.ToArray() ...
}
Try to check Socket.Available property to determine if do you need to call Receive again.
I think I understand your question. This overload of Socket.Receive allows you to pass an integer to specify an offset position to start placing data. If you got 1 byte, on the first call as in your example, you can call this overload with an offset of 1 and use the same buffer.
Socket.Receive returns an integer that is the number of bytes that were received. You could check if this is 1 and call Receive again.
byte[] buffer = new byte[256];
int len = socket.Receive(buffer);
if (len == 1)
socket.Receive(buffer, 1, buffer.Length - 1, SocketFlags.None);
All,
When a web-server is sending data in "chunks" it precedes each chunk with it's length (as a string indicating the hex value).
chunk-size[;chunk-extensions]
chunk-data
eg: to chunk 15 bytes:
F;
123456789ABCDEF
The biggest issue with receiving HTTP data from a Socket is determining how much data to read: if you have received all the available data and call the Recieve method again, then the Recieve method will block until the remote socket sends more data, which for a HTTP/1.0 connection will never happen.
You need to implement a reader that wraps around the socket; it should receive data into a buffer, and provide a "ReadLine" method that reads the contents of the buffer until it reads 13 followed by 10 (CRLF)