I am working on socket C#. I've implemented a client server application using socket, but the problem is that the client doesn't receive all data sent by the server.
Here is the client application code. What should I do so that it would receive all data sent by the server?
strRecieved = "";
Socket soc = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);
IPEndPoint endPoint = new IPEndPoint(IPAddress.Parse("127.0.0.1"), 9001);
soc.Connect(endPoint);
byte[] msgBuffer = Encoding.Default.GetBytes(path);
soc.Send(msgBuffer, 0, msgBuffer.Length, 0);
byte[] buffer = new byte[2000];
int rec = soc.Receive(buffer);
strRecieved = String.Format(Encoding.Default.GetString(buffer));
First of all. If you're implementing some kind of streaming feature ( tcp/udp/file ) you should consider using some kind of protocol.
What is a protocol? It's just a scheme to use when streaming data. Example:
[4Bytes - length][lengthBytes - message][1Byte - termination indicator]
Knowing the protocol you can read all of the incoming bytes simply as such :
byte[] buffer = new byte[4];
stream.ReadBytes(buffer, 0, 4); // cast that to int and read the rest
int packetLen = BitConverter.ToInt32(buffer, 0);
buffer = new byte[packetLen];
stream.ReadBytes(buffer, 0, buffer.Length); // all bytes that was sent
Remember that you have to subtract thease 4 bytes in the length before sending the message.
EDIT:
Simple example on how to send and receive data using shared protocol.
// sender.cs
string _stringToSend = "some fancy string";
byte[] encodedString = Encoding.UTF8.GetBytes(_stringToSend);
List<byte> buffer = new List<byte>();
buffer.AddRange(BitConverter.GetBytes(encodedString.Length));
buffer.AddRange(encodedString);
netStream.WriteBytes(buffer.ToArray(), 0, buffer.Count);
// netStream sent message in protocol [#LEN - 4Bytes][#MSG - #LENBytes]
// simply speaking something like: 5ABCDE
// receiver.cs
byte[] buffer = new byte[sizeof(int)];
netStream.ReadBytes(buffer, 0, buffer.Length);
// receiver got the length of the message eg. 5
int dataLen = BitConverter.ToInt32(buffer, 0);
buffer = new byte[dataLen];
// now we can read an actual message because we know it's length
netStream.ReadBytes(buffer, 0, buffer.Length);
string receivedString = Encoding.UTF8.GetString(buffer);
// received string is equal to "some fancy string"
Making it simpler
This technique forces you to use desired protocol which in this example will be :
First 4 bytes sizeof(int) are indicating the length of the incoming packet
Every byte further is your packet until the end.
So right now you should make ProtocolHelper object:
public static class ProtocolHelper
{
public byte[] PackIntoProtocol(string message)
{
List<byte> result = new List<byte>();
byte[] messageBuffer = Encoding.UTF8.GetBytes(message);
result.AddRange(BitConverter.GetBytes(messageBuffer.Length), 0); // this is the first part of the protocol ( length of the message )
result.AddRange(messageBuffer); // this is actual message
return result.ToArray();
}
public string UnpackProtocol(byte[] buffer)
{
return Encoding.UTF8.GetString(buffer, 0, buffer.Length);
}
}
Now ( depending on method you've chosen to read from network ) you have to send and receive your message.
// sender.cs
string meMessage = "network message 1";
byte[] buffer = ProtocolHelper.PackIntoProtocol(meMessage);
socket.Send(buffer, 0, buffer.Length, 0);
// receiver.cs
string message = string.Empty;
byte[] buffer = new byte[sizeof(int)]; // or simply new byte[4];
int received = socket.Receive(buffer);
if(received == sizeof(int))
{
int packetLen = BitConverter.ToInt32(buffer);// size of our message
buffer = new byte[packetLen];
received = socket.Receive(buffer);
if( packetLen == received ) // we have full buffer
{
message = PacketHelper.UnpackProtocol(buffer);
}
}
Console.WriteLine(message); // output: "network message 1"
You're limiting the size of received messages by 2KB as you're using new byte[2000].
I think you could either:
Size up you buffer to meet you message's size needs; and/or
Split you message into more than one socket messages.
Given that 4-8K is a good size for buffering socket messages and assuming RAM size is not a issue I would start with that, say, new byte[8000].
Also, you can send socket messages splitted in chunks. Maybe this is a good idea for the case. For example, if you have msg as the message (or object) you want to send:
private static async Task SendAnswer(Message msg, WebSocket socket)
{
var answer = System.Text.Encoding.UTF8.GetBytes(JsonConvert.SerializeObject(msg).ToCharArray());
var bufferSize = 8000;
var endOfMessage = false;
for (var offset = 0; offset < answer.Length; offset += bufferSize)
{
if (offset + bufferSize >= answer.Length)
{
bufferSize = answer.Length - offset;
endOfMessage = true;
}
await socket.SendAsync(new ArraySegment<byte>(answer, offset, bufferSize),
WebSocketMessageType.Text, endOfMessage, CancellationToken.None);
}
}
And when receiving, you can also split the reception in chunks, so you can control you buffer (and therefore you memory consumption). After hanlding the whole message, you should wait for another message from the client to do more stuff. Source
private async Task ReceiveMessage(WebSocket webSocket)
{
var buffer = new byte[8000];
var result = await webSocket.ReceiveAsync(buffer, CancellationToken.None);
while (!result.CloseStatus.HasValue)
{
string msg = Encoding.UTF8.GetString(new ArraySegment<byte>(buffer, 0, result.Count).ToArray());
while (!result.EndOfMessage)
{
result = await webSocket.ReceiveAsync(buffer, CancellationToken.None);
msg += Encoding.UTF8.GetString(new ArraySegment<byte>(buffer, 0, result.Count).ToArray());
}
//At this point, `msg` has the whole message you sent, you can do whatever you want with it.
// [...]
//After you handle the message, wait for another message from the client
result = await webSocket.ReceiveAsync(buffer, CancellationToken.None);
}
await webSocket.CloseAsync(result.CloseStatus.Value, result.CloseStatusDescription, CancellationToken.None);
}
Related
I recently started to work with TCP in C#. I'm now at the point where I want the client to receive the data sent by the server.
I know that there is no guarantee for the client to receive all data at once. If the size of the data sent is bigger than the buffer's size at the client's side, then the data will be sent in parts. So my question is: how can I store all my received data in a byte array, and then convert it to the actual message when all is received?
I've set the buffer size to 1, so I can see what happens when all sent data doesn't fit in the buffer. Here are my methods in which I call stream.BeginRead() in Client.cs:
// Deliberately setting the buffer size to 1, to simulate what happens when the message doesn't fit in the buffer.
int bufferSize = 1;
byte[] receiveBuffer;
private void ConnectCallback(IAsyncResult result)
{
client.EndConnect(result);
Console.WriteLine("Connected to server.");
stream = client.GetStream();
// At this point, the client is connected, and we're expecting a message: "Welcome!"
receiveBuffer = new byte[bufferSize];
stream.BeginRead(receiveBuffer, 0, receiveBuffer.Length, new AsyncCallback(ReadCallback), stream);
}
private void ReadCallback(IAsyncResult result)
{
int bytesLength = stream.EndRead(result);
// Should be "Welcome!". But of course it's "W", because the bufferSize is 1.
string message = Encoding.UTF8.GetString(receiveBuffer, 0, bytesLength);
Console.WriteLine("Received message: {0}", receivedMessage);
// Reset the buffer and begin reading a new message, which will be "e".
// However, I want the whole message ("Welcome!") in one byte array.
receiveBuffer = new byte[bufferSize];
stream.BeginRead(receiveBuffer, 0, receiveBuffer.Length, ReadCallback, null);
}
This is the output when sending the message "Welcome!":
Connected to server.
Received message: W
Received message: e
Received message: l
Received message: c
Received message: o
Received message: m
Received message: e
Received message: !
Should I temporary store the data until the whole message has arrived, and then convert that to a string?
Follow up question: What if 2 messages are sent closely after each other, for example Welcome! and then What's your name? How do I distinguish the two messages then?
Should I temporary store the data until the whole message has arrived, and then convert that to a string?
Yes, exactly.
Follow up question: What if 2 messages are sent closely after each other, for example Welcome! and then What's your name? How do I distinguish the two messages then?
The general approach is to send the length of the message before the message itself. That way the receiving end will know when it has received a complete package.
As 500 - Internal Server Error already pointed out, you use a buffer for it. Here is some Code example:
Receiving:
while (true) //you should use a bool variable here to stop this on disconnect.
{
byte[] bytes;
bytes = ReadNBytes(ns, 4);
//read out the length field we know is there, because the server always sends it.
int msgLenth = BitConverter.ToInt32(bytes, 0);
bytes = ReadNBytes(ns, msgLenth);
//working with the buffer...
if (bytes.Length > 0)
{
try
{
//do stuff here. bytes contains your complete message.
}
catch (Exception e) { Log(e.Message); }
}
}
public static byte[] ReadNBytes(NetworkStream stream, int n)
{
byte[] buffer = new byte[n];
try
{
int bytesRead = 0;
int chunk;
while (bytesRead < n)
{
chunk = stream.Read(buffer, (int)bytesRead, buffer.Length - (int)bytesRead);
if (chunk == 0)
{
// error out
Log("Unexpected disconnect");
stream.Close();
}
bytesRead += chunk;
}
}
catch (Exception e) { Log(e.Message); }
return buffer;
}
To send stuff use something linke this:
public static void SendObject(NetworkStream ns, byte[] data)
{
byte[] lengthBuffer = BitConverter.GetBytes(data.Length);
ns.Write(lengthBuffer, 0, lengthBuffer.Length);
ns.Write(data, 0, data.Length);
}
I hope that helped!
So I am sending a simple string from my TCP Client to my server and then when I receive it it decodes the bytes and prints out what it got.. However I am sending
Client connecting..
and I receive
Client connecting..\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0
Why is that?
The buffer seems to be the same, 18 bytes with the empty ones being 0
is this due to the Encoding method? I've tried different ones likes Default and UTF8 but it seems to do the same still.
_listener.Start();
Console.WriteLine("Waiting for connection..");
//Assign our client the value of the first accepted request.
_client = _listener.AcceptTcpClient();
Console.WriteLine("Client connected.");
//Set the stream to listen for incoming requests.
_stream = _client.GetStream();
//Build the package
byte[] buffer = new byte[128];
var bufferLength = _stream.Read(buffer, 0, buffer.Length);
return buffer;
And it passes the byte array to this
public void SendPacket(byte[] buffer)
{
TcpClient client = new TcpClient(hostName, portNum);
NetworkStream ns = client.GetStream();
Console.ForegroundColor = ConsoleColor.Cyan;
Console.WriteLine($"Received from remote client: {Encoding.UTF8.GetString(buffer, 0, buffer.Length)}");
Console.ForegroundColor = ConsoleColor.Green;
Console.WriteLine($"Relaying to the remote server: {Encoding.UTF8.GetString(buffer, 0, buffer.Length)}");
ns.Write(buffer, 0, buffer.Length);
}
And then
Console.WriteLine("Data Received..");
var data = Encoding.UTF8.GetString(buffer, 0, length);
Console.ForegroundColor = ConsoleColor.Yellow;
Console.WriteLine("Received: " + data);
According to your code
var bufferLength = _stream.Read(buffer, 0, buffer.Length);
return buffer;
bufferLength is dropped on the floor.
var data = Encoding.UTF8.GetString(buffer, 0, length);
I don't know where length came from, but I suppose it's the buffer's length. Since the receive didn't fill up the whole buffer but only part of it, you must use bufferLength to know how many bytes to work with.
Same problem here:
var eh = ns.Read(newBuffer, 0, newBuffer.Length);
var received = Encoding.ASCII.GetString(newBuffer, 0, newBuffer.Length);
Use eh!
var eh = ns.Read(newBuffer, 0, newBuffer.Length);
var received = Encoding.ASCII.GetString(newBuffer, 0, eh);
\0 is the "empty" value for a char object, so most likely it is reading the chars.
I'm currently working on a TCPClient and Server. Lately I added an encryption for the messages, and had no trouble. Once I started noticing that I'm getting a weird error like this:
But It's totally random, and no idea why. It happens at larger messages, but as I said, not always.
Checking the byte[] length at the server side says 1920 (Sometimes it says 1920 on the client too, and thats the point when i dont have error)
On client it says a lot lesser.
I actually think that sometimes the client doesn't receive the full byte that It should, this is how I do It:
Client:
byte[] bb = new byte[12288];
int k = stm.Read(bb, 0, 12288);
string message = Encoding.UTF8.GetString(bb, 0, k);
MessageBox.Show(message.Length.ToString()); // This sometimes says 1460, and 1920
message = Encrypter.DecryptData(message); // Error here If the length is not 1920
Server:
bmsg = Encrypter.EncryptData(((int)Codes.XYZEnum) + "=" + data);
Logger.Log(bmsg.Length.ToString()); // Original msg, always says 1920
s.Send(asen.GetBytes(bmsg));
s.Close();
What could be the problem? Should I try async sending?
SOLUTION:
Server code, took me a little while to make it cool:
System.Net.Sockets.Socket s = myList.AcceptSocket(); // Accept the connection
Stream stream = new NetworkStream(s); // Create the stream object
byte[] leng = new byte[4]; // We will put the length of the upcoming message in a 4 length array
int k2 = s.Receive(leng); // Receive the upcoming message length
if (BitConverter.IsLittleEndian)
{
Array.Reverse(leng);
}
int upcominglength = (BitConverter.ToInt32(leng, 0)); // Convert it to int
byte[] b = ByteReader(upcominglength, stream); // Create the space for the bigger message, read all bytes until the received length!
string message = Encoding.UTF8.GetString(b, 0, b.Length); // Convert it to string!
internal byte[] ByteReader(int length, Stream stream)
{
byte[] data = new byte[length];
using (MemoryStream ms = new MemoryStream())
{
int numBytesRead;
int numBytesReadsofar = 0;
while (true)
{
numBytesRead = stream.Read(data, 0, data.Length);
numBytesReadsofar += numBytesRead;
ms.Write(data, 0, numBytesRead);
if (numBytesReadsofar == length)
{
break;
}
}
return ms.ToArray();
}
}
Client code, and it is working nicely!:
var result = tcpclnt.BeginConnect(User.IP, User.Port, null, null);
var success = result.AsyncWaitHandle.WaitOne(TimeSpan.FromSeconds(3)); // Connect with timeout
if (!success)
{
return "Failed to connect!";
}
Stream stm = tcpclnt.GetStream(); // get the stream
UTF8Encoding asen = new UTF8Encoding();
byte[] ba = asen.GetBytes(msg); // get the bytes of the message we are sending
byte[] intBytes = BitConverter.GetBytes(ba.Length); // Get the length of that in bytes
if (BitConverter.IsLittleEndian)
{
Array.Reverse(intBytes);
}
stm.Write(intBytes, 0, intBytes.Length); // Write the length in the stream!
stm.Flush(); // Clear the buffer!
stm.Write(ba, 0, ba.Length); // Write the message we are sending!
// If we have answers....
byte[] bb = new byte[10000];
int k = stm.Read(bb, 0, 10000);
string mmessage = Encoding.UTF8.GetString(bb, 0, k);
// If we have answers....
tcpclnt.Close(); // Close the socket
Because only 8Kb can to send by once packet. if you have large data you need use cycle.
Using a blocking, streaming .NET socket I'm connecting to a server. Whenever I'm reading small bits of data, all goes well and the data is received into my buffer:
using (var socket = new Socket(SocketType.Stream, ProtocolType.IP))
{
socket.Connect(IPAddress.Parse("127.0.0.1"), 5000);
byte[] buffer = new byte[BufferSize];
socket.Receive(buffer);
// Here buffer doesn't always contain all data the server sent me?
Console.WriteLine(Encoding.Default.GetString(buffer));
}
In some cases though, I'm not receiving everything the server sends me. Data seems to be chopped off. What can be the cause of this?
This is documented in the Receive() method, emphasis mine:
The Receive method reads data into the buffer parameter and returns the number of bytes successfully read. You can call Receive from both connection-oriented and connectionless sockets.
When ignoring the return value, you will not know what portion of your buffer actually contains relevant data. Depending on the protocol being used, you may or may not know the content length in advance. Some protocols provide this length, others close the connection when done, yet another can use a message boundary.
You'll have to hold the received data in another buffer, and return or output the entire message buffer when no more data is available or expected. This can be done like this:
int BufferSize = 1024;
using (var socket = new Socket(SocketType.Stream, ProtocolType.IP))
{
socket.Connect(IPAddress.Parse("127.0.0.1"), 5000);
byte[] buffer = new byte[BufferSize];
string message = "";
int bytesReceived;
do
{
bytesReceived = socket.Receive(buffer);
message += Encoding.ASCII.GetString(buffer, 0, bytesReceived);
} while (bytesReceived > 0);
Console.WriteLine(message);
}
The received bytes are ASCII characters (as defined in the made-up protocol), so each byte received indicates one character (you can't convert partially received multibyte unicode characters). The bytes are converted to a string and appended to the message variable. The code loops until the server closes the connection.
When the message size is known on beforehand, again, depending on the protocol being used, you can create a message buffer and copy the data there on each Receive():
// Received using another Receive() call
int messageSize = 1234;
int totalBytesReceived = 0;
byte[] messageBuffer = new byte[messageSize];
byte[] buffer = new byte[BufferSize];
int bytesReceived;
do
{
bytesReceived = socket.Receive(buffer);
// Copy the receive buffer into the message buffer, appending after
// previously received data (totalBytesReceived).
Buffer.BlockCopy(buffer, 0, messageBuffer, totalBytesReceived, bytesReceived);
totalBytesReceived += bytesReceived;
} while (bytesReceived > 0);
// This assumes the connection wasn't closed prematurely.
Console.WriteLine(Encoding.ASCII.GetString(messageBuffer));
This can in turn be put in a resuable method:
public byte[] ReceiveMessage(Socket socket, int messageSize)
{
byte[] messageBuffer = new byte[messageSize];
int bytesReceived = 0;
int totalBytesReceived = 0;
do
{
byte[] buffer = new byte[BufferSize];
// Receive at most the requested number of bytes, or the amount the
// buffer can hold, whichever is smaller.
int toReceive = Math.Min(messageSize - totalBytesReceived, BufferSize);
bytesReceived = socket.Receive(buffer, toReceive, SocketFlags.None);
// Copy the receive buffer into the message buffer, appending after
// previously received data (totalBytesReceived).
Buffer.BlockCopy(buffer, 0, messageBuffer, totalBytesReceived, bytesReceived);
totalBytesReceived += bytesReceived;
} while (bytesReceived > 0);
if (totalBytesReceived < messageSize)
{
throw new Exception("Server closed connection prematurely");
}
return messageBuffer;
}
There's the NetworkStream which wraps a socket, but it has the same reading issues as the socket itself. You will have to monitor the return value and keep calling Read() until you've received all bytes. Same goes for the TcpClient that has a GetStream() method, on whose return value you'll also have to continue reading until you've read all data.
I have the following code:
private string Connect()
{
string responseData;
try
{
TcpClient client = new TcpClient(ServerIp, Port);
client.ReceiveBufferSize = Int32.MaxValue;
Byte[] data = Encoding.GetEncoding(1251).GetBytes(ReadyQuery);
NetworkStream stream = client.GetStream();
// send data
stream.Write(data, 0, data.Length);
// buffer
data = new Byte[65536];
Int32 bytes = stream.Read(data, 0, data.Length);
responseData = Encoding.GetEncoding(1251).GetString(data, 0, bytes);
// close all
stream.Close();
client.Close();
return responseData;
}
I have problem with a big message. The receive message size is 22K chars. I get only part of message.
How can I receive big messages?
PS. In the debugger bytes equal 4096.
You call stream.Read in a loop until you read the entire message. If you know the message size in advance it's relatively easy:
int messageSize = 22000;
int readSoFar = 0;
byte [] msg = new byte[messageSize];
while(readSoFar < messageSize)
{
var read = stream.Read(msg, readSoFar, msg.Length - readSoFar);
readSoFar += read;
if(read==0)
break; // connection was broken
}
If the message size is part of the message (say, encoded in the first 4 bytes), you should read those first and then do as I suggested.