First I'll apologize if any of my code doesn't make sense as i am fairly new to coding and currently in University.
Any way my problem is as described.
I have this code in my python server to send the data which is gathered from my local oracle server:
query = "SELECT * FROM DATABASE"
cursor.execute(query)
for row in cursor:
for cell in row:
cellF = str.format("{0}$",cell)
self.send(cellF.encode())
And this code to receive the data on my C# client:
public void Read()
{
var buffer = new byte[1024];
var stream = client.socket.GetStream();
stream.BeginRead(buffer, 0, buffer.Length, EndRead, buffer);
}
public void EndRead(IAsyncResult result)
{
var stream = client.socket.GetStream();
int endBytes = stream.EndRead(result);
if (endBytes == 0)
{
return;
}
var buffer = (byte[])result.AsyncState;
string data = Encoding.ASCII.GetString(buffer, 0, endBytes);
SetRecieved(data);
stream.BeginRead(buffer, 0, buffer.Length, EndRead, buffer);
}
Revision:
I realized i had some misc code so i removed it and now it consistently works but only when the method is ran for the second time.
public void Read()
{
while (client.stream.DataAvailable)
{
client.stream.BeginRead(client.buffer, 0, client.buffer.Length, EndRead, client.buffer);
}
}
public void EndRead(IAsyncResult result)
{
var stream = client.socket.GetStream();
int endBytes = stream.EndRead(result);
var buffer = (byte[])result.AsyncState;
string data = Encoding.ASCII.GetString(buffer, 0, endBytes);
SetRecieved(data);
}
The first time i send the request to the server to get this data it works as expected but the second time i request the data i loose a random string of data along the way, the first time always works perfectly however and i receive all strings so I'm not sure what is happening.
Lets do some analysis...
You send some data from the server to the client that is less then size of the buffer. You read it and that's all. You call return in the asynccallback function. If you don't use while loop then you will not be able to receive new data from server.
You send two messages from the server to the client. This two messages are in the same buffer, because they summed size is less then buffer size. You read them as one with your approach and your data is corrupted, because you don't know the data size.
If you want to treat your data as separate messages you need to know the message size and handle each case differently. Data is less then buffer size - you need to continue waiting for incoming data. Data is greater then buffer size - you need to read whole message. And so on.
Related
I'm writing a screen mirroring app on WPF. My original code sends a bitmap over TCP from a server to a client. The original code works fine, but closes and recreates the tcp connection every time it sends a frame. This results in 30 socket open and close per second, which I assume isn't the ideal way to do it.
So I tried to rewrite it to reuse the stream each time it sends the data, but the stream starts to spit out wrong data after a while.
public void SendStream(byte[] byteArray)
{
/*
_client = IsServer ? _server.AcceptTcpClient() : new TcpClient(IP.ToString(), Port);
using (var clientStream = _client.GetStream())
{
var comp = Compress(byteArray);
clientStream.Write(comp, 0, comp.Length);
}
*/
var comp = Compress(byteArray);
_stream.Write(BitConverter.GetBytes(comp.Length), 0, 4);
_stream.Write(comp, 0, comp.Length);
}
public byte[] ReceiveStream()
{
/*
_client = IsServer ? _server.AcceptTcpClient() : new TcpClient(IP.ToString(), Port);
var stream = _client.GetStream();
return Decompress(stream);
*/
var lengthByte = new byte[4];
_stream.Read(lengthByte, 0, 4);
var length = BitConverter.ToInt32(lengthByte, 0);
var data = new byte[length];
_stream.Read(data, 0, length);
return Decompress(new MemoryStream(data));
}
Compress and Decompress function are just wrappers around the built in DeflateStream.
I have checked that the sent comp.Length and received length are the same when the error happens.
Any ideas on whats going on? Thanks. It always throws an exception after at least a few frames, never the first one (at least that I've tried so far)
(It seems to happen faster when the bitmaps are larger in size i.e. when the compression algorithm doesn't do as much cause the screen is more complicated. Not 100% sure though)
Try doing the following:
int receivedBytesCount = _stream.Read(data, 0, length);
The length variable you pass to the Read method is the maximum. The Read method may actually read less bytes than length. It will return the number of bytes it actually read. This will happen when your data is fragmented into TCP packets.
You need to keep calling Read until you receive enough bytes and combine everything to get the full frame. You will need to adjust the offset in order to avoid overwriting the buffer. In the code you posted it is hardcoded to 0.
I am relativity new to C#. In my TCP client have the following function which sends data to the server and returns the response:
private static TcpClient tcpint = new TcpClient(); //Already initiated and set up
private static NetworkStream stm; //Already initiated and set up
private static String send(String data)
{
//Send data to the server
ASCIIEncoding asen = new ASCIIEncoding();
byte[] ba = asen.GetBytes(data);
stm.Write(ba, 0, ba.Length);
//Read data from the server
byte[] bb = new byte[100];
int k = stm.Read(bb, 0, 100);
//Construct the response from byte array to string
StringBuilder sb = new StringBuilder();
for (int i = 0; i < k; i++)
{
sb.Append(bb[i].ToString());
}
//Return server response
return sb.ToString();
}
As you can see here, when I am reading the response from the server, I am reading it into a fix byte[] array of length 100 bytes.
byte[] bb = new byte[100];
int k = stm.Read(bb, 0, 100);
What do i do if the response from the server is more than 100 bytes? How can I read the data without me knowing what the max length of data form the server will be?
Typically, where there is not some specific intrinsic size of something, tcp protocols explicitly send the length of objects they are sending. One possible method for illustration:
size_t data_len = strlen(some_data_blob);
char lenstr[32];
sprintf(lenstr, "%zd\n", data_len);
send(socket, lenstr, strlen(lenstr));
send(socket, some_data_blob, data_len);
then when the receiver reads the length string, it knows exactly how mush data should follow (good programming practice is to trust but verify though -- if there is more or less data really sent -- say by an 'evil actor' -- you need to be prepared to handle that).
Not with respect to C# but a general answer on writing TCP application:
TCP is steam based protocol. It does not maintain message boundaries. So, the applications using TCP should take care of choosing the right method of data exchange between server and client. Its becomes more paramount if multiple messages gets sent and received on one connection.
One widely used method is to prepend the data message with the length bytes.
Ex:
[2 byte -length field][Actual Data].
The receiver of such data (be it server or client needs to decode length field, wait for until such event where as many bytes are received or raise an alarm on timeout and give up.
Another protocol that can be used is to have applications maintain message boundaries.
Ex:
`[START-of-MSG][Actual Data][END-of-MSG]
The reciever has to parse the data for Start-byte and End-byte (predefined by application protocol) and treat anything in between as data of interest.
hello i solved it with a list, i don't know the size of the complete package but i can read it in parts
List<byte> bigbuffer = new List<byte>();
byte[] tempbuffer = new byte[254];
//can be in another size like 1024 etc..
//depend of the data as you sending from de client
//i recommend small size for the correct read of the package
NetworkStream stream = client.GetStream();
while (stream.Read(tempbuffer, 0, tempbuffer.Length) > 0) {
bigbuffer.AddRange(tempbuffer);
}
// now you can convert to a native byte array
byte[] completedbuffer = new byte[bigbuffer.Count];
bigbuffer.CopyTo(completedbuffer);
//Do something with the data
string decodedmsg = Encoding.ASCII.GetString(completedbuffer);
I do this whith images and looks good, i thik than you dont know the size of the data if the porpouse is read a complete source with a unknow size
I was looking around for an answer to this, and noticed the Available property was added to TcpClient. It returns the amount of bytes available to read.
I'm assuming it was added after most of the replies, so I wanted to share it for others that may stumble onto this question.
https://learn.microsoft.com/en-us/dotnet/api/system.net.sockets.tcpclient.available?view=netframework-4.8
I'm developing one TCP server application with TCP listener class. here my server application getting data on every second from client.
Client sending data in a predefined format. received data contains 15 messages separated by "\0". e.g "12\012345\012.12\0" and so on. after getting data i split data and convert it in to string array. so i have string array of 15 elements. after that each element get converted to specific data type and whole record goes in to database.
here data send/receive happens on every second. the problem i'm facing is my application not sending data on every second to client application.
when i remove data type conversion code all is working well as expected. but with conversion code its took more milliseconds and my application not able to send data back to client in time.
below is my code. if i remove data type conversion code from "MapVariables" function its working well.
Please please can any one help me on this?
private async void ProcessClient(TcpClient tcpClient, CancellationToken ct)
{
try
{
while (!ct.IsCancellationRequested)
{
var stream = tcpClient.GetStream();
var buffer = new byte[bufferSize];
var amountRead = await stream.ReadAsync(buffer, 0, bufferSize);
var message = Encoding.ASCII.GetString(buffer, 0, amountRead);
string[] dataFromClient = Code.Common.SplitByLength(message, messageSize).ToArray();
dataFromClient = dataFromClient.Select(x => ParseMessage(x)).ToArray();
common.MapVariables(dataFromClient);
string serverResponse = string.Join(", ", dataFromClient);
//Byte[] sendBytes = Encoding.ASCII.GetBytes(serverResponse);
Byte[] sendBytes = Encoding.ASCII.GetBytes(message);
await stream.WriteAsync(sendBytes, 0, sendBytes.Length, ct);
stream.Flush();
}
}
catch (System.IO.IOException ex)
{
//loge exception
}
catch (Exception ex)
{
//loge exception
}
}
public void MapVariables(string[] variables)
{
Variables.Variable1 = int.Parse(variables[0]);
Variables.Variable2 = int.Parse(variables[1]);
Variables.Variable3 = int.Parse(variables[2]);
Variables.Variable4 = int.Parse(variables[3]);
Variables.Variable5 = int.Parse(variables[4]);
Variables.Variable6 = int.Parse(variables[5]);
Variables.Variable7 = int.Parse(variables[6]);
Variables.Variable8 = decimal.Parse(variables[7]);
Variables.Variable9 = decimal.Parse(variables[8]);
Variables.Variable10 = decimal.Parse(variables[9]);
Variables.Variable11 = decimal.Parse(variables[10]);
Variables.Variable12 = int.Parse(variables[11]);
Variables.Variable13 = int.Parse(variables[12]);
Variables.Variable14 = decimal.Parse(variables[13]);
Variables.Variable15 = decimal.Parse(variables[14]);
InsertintoDatabase();
}
Looking at the code, you just send back what they sent you except comma delimited.
You could move your MapVariables(...) call into a separate thread and use that. My guess is that your InsertintoDatabase call is the true bottleneck.
You could also attempt to just move MapVariables(...) to be done after you reply since it doesn't look like it will affect anything by doing it afterwards.
Im currently a bit stuck with my c# project.
I have 2 applications, they both have a common class definition I call a NetMessage
a NetMessage contains a MessageType string property, as well as 2 List lists.
The idea is that I can pack this class with classes, and data to send across the network as a byte[].
Because Network Streams do not advertise the amount of data they are receiving, I modified my Send method to send the size of the NetMessage byte[] ahead of the actual byte[].
private static byte[] ReceivedBytes(NetworkStream MainStream)
{
try
{
//byte[] myReadBuffer = new byte[1024];
int receivedDataLength = 0;
byte[] data = { };
long len = 0;
int i = 0;
MainStream.ReadTimeout = 60000;
//MainStream.CanTimeout = false;
if (MainStream.CanRead)
{
//Read the length of the incoming message
byte[] byteLen = new byte[8];
MainStream.Read(byteLen, 0, 8);
len = BitConverter.ToInt64(byteLen, 0);
data = new byte[len];
//data is now set to the appropriate size for the expected message
//While we have not got the full message
//Read each individual byte and append to data.
//This method, seems to work, but is ridiculously slow,
while (receivedDataLength < data.Length)
{
receivedDataLength += MainStream.Read(data, receivedDataLength, 1);
}
//receivedDataLength += MainStream.Read(data, receivedDataLength, data.Length);
return data;
}
}
catch (Exception E)
{
//System.Windows.Forms.MessageBox.Show("Exception:" + E.ToString());
}
return null;
}
I have tried to change the size argument below to something like 1024 or to be the data.Length, but I get funky results.
receivedDataLength += MainStream.Read(data, receivedDataLength, 1);
setting it to data.Length seems to cause problems when the Class being sent is a few mb in size.
Setting the buffer size to be 1024 like I have seen in other examples, causes failures when the size of the incoming message is small, like 843 bytes, it errors out saying that I tried to read out of bounds or something.
Below is the type of method being used to send the data in the first place.
public static void SendBytesToStream(NetworkStream TheStream, byte[] TheMessage)
{
//IAsyncResult r = TheStream.BeginWrite(TheMessage, 0, TheMessage.Length, null, null);
// r.AsyncWaitHandle.WaitOne(10000);
//TheStream.EndWrite(r);
try
{
long len = TheMessage.Length;
byte[] Bytelen = BitConverter.GetBytes(len);
TheStream.Write(Bytelen, 0, Bytelen.Length);
TheStream.Flush();
// <-- I've tried putting thread sleeps in this spot to see if it helps
//I've also tried writing each byte of the message individually
//takes longer, but seems more accurate as far as network transmission goes?
TheStream.Write(TheMessage, 0, TheMessage.Length);
TheStream.Flush();
}
catch (Exception e)
{
//System.Windows.Forms.MessageBox.Show(e.ToString());
}
}
I'd like to get these two methods setup to the point where they are reliably sending and receiving data.
The application I am using this for, monitors a screenshots folder in a game directory,
when it detects a screenshot in TGA format, it converts it to PNG, then takes its byte[] and sends it to the receiver.
The receiver then posts it to Facebook (I don't want my FB tokens distributed in my client application), hence the server / proxy idea.
Its strange, but when I step through the code, the transfer is invariably successful.
But if I run it full speed, no breakpoint, it typically tells me that the connection was closed by the remote host etc.
The client typically finishes sending the data almost instantly, even though its a 4mb file.
The receiver spends about 2 minutes reading from the Network Stream, which doesnt make sense, if the client finished sending the data, does that mean the data is just floating in cyber space, and being pulled down?
Surely it should be synchronous?
I suspect I know where my code was going wrong.
It turns out that the scope I was creating my TCPClient that was doing the sending, was declared and instantiated within a method.
This being the case, the GAC was disposing of it, even though the Receiving Server had not finished downloading the stream.
I managed to resolve it by creating a method that can detect when the Client has disconnected on the server end, and until it has actually disconnected, it will keep looping/waiting until disconnected.
This way, we are waiting until the server lets go of us.
This question already has answers here:
Receiving data in TCP
(10 answers)
Closed 2 years ago.
Here's my code:
private void OnReceive(IAsyncResult result)
{
NetStateObject state = (NetStateObject)result.AsyncState;
Socket client = state.Socket;
int size = client.EndReceive(result);
byte[] data = state.Buffer;
object data = null;
using (MemoryStream stream = new MemoryStream(data))
{
BinaryFormatter formatter = new BinaryFormatter();
data = formatter.Deserialize(stream);
}
//todo: something with data
client.BeginReceive(
state.Buffer,
0,
NetStateObject.BUFFER_SIZE,
SocketFlags.None,
OnReceive,
state
);
}
state.Buffer has a maximum size of NetStateObject.BUFFER_SIZE (1024). Firstly, is this too big or too small? Second, if I send something larger than that, my deserialize messes up because the object it is trying to deserialize doesnt have all the information (because not all the data was sent). How do I make sure that all my data has been received before I try to construct it and do something with it?
Completed Working Code
private void OnReceive(IAsyncResult result)
{
NetStateObject state = (NetStateObject)result.AsyncState;
Socket client = state.Socket;
try
{
//get the read data and see how many bytes we received
int bytesRead = client.EndReceive(result);
//store the data from the buffer
byte[] dataReceived = state.Buffer;
//this will hold the byte data for the number of bytes being received
byte[] totalBytesData = new byte[4];
//load the number byte data from the data received
for (int i = 0; i < 4; i++)
{
totalBytesData[i] = dataReceived[i];
}
//convert the number byte data to a numan readable integer
int totalBytes = BitConverter.ToInt32(totalBytesData, 0);
//create a new array with the length of the total bytes being received
byte[] data = new byte[totalBytes];
//load what is in the buffer into the data[]
for (int i = 0; i < bytesRead - 4; i++)
{
data[i] = state.Buffer[i + 4];
}
//receive packets from the connection until the number of bytes read is no longer less than we need
while (bytesRead < totalBytes + 4)
{
bytesRead += state.Socket.Receive(data, bytesRead - 4, totalBytes + 4 - bytesRead, SocketFlags.None);
}
CommandData commandData;
using (MemoryStream stream = new MemoryStream(data))
{
BinaryFormatter formatter = new BinaryFormatter();
commandData = (CommandData)formatter.Deserialize(stream);
}
ReceivedCommands.Enqueue(commandData);
client.BeginReceive(
state.Buffer,
0,
NetStateObject.BUFFER_SIZE,
SocketFlags.None,
OnReceive,
state
);
dataReceived = null;
totalBytesData = null;
data = null;
}
catch(Exception e)
{
Console.WriteLine("***********************");
Console.WriteLine(e.Source);
Console.WriteLine("***********************");
Console.WriteLine(e.Message);
Console.WriteLine("***********************");
Console.WriteLine(e.InnerException);
Console.WriteLine("***********************");
Console.WriteLine(e.StackTrace);
}
}
TCP is a stream protocol. It has no concept of packets. A single write call can be sent in multiple packets, and multiple write calls can be put into the same packet. So you need to implement your own packetizing logic on top of TCP.
There are two common ways to packetize:
Delimiter characters, this is usually used in text protocols, with the new-line being a common choice
Prefix the length to each packet, usually a good choice with binary protocols.
You store the size of a logical packet at the beginning of that packet. Then you read until you received enough bytes to fill the packet and start deserializing.
How do I make sure that all my data has been received before I try to construct it and do something with it?
You have to implement some protocol so you know.
While TCP is reliable, it does not guarantee that the data from single write at one end of the socket will appear as a single read at the other end: retries, packet fragmentation and MTU can all lead to data being received in different sized units by the receiver. You will get the data in the right order.
So you need to include some information when sending that allows the receiver to know when it has the complete message. I would also recommend including what kind of message and what version of the data (this will form the basis of being able to support different client and server versions together).
So the sender sends:
- Message type
- Message version
- Message size (in bytes)
And the receiver will loop, performing a read with a buffer and appending this to a master buffer (MemoryStream is good for this). Once the complete header is received it knows when the complete data has been received.
(Another route is to include some pattern as an "end of message" marker, but then you need to handle the same sequence of bytes occurring in the content—hard to do if the data is binary rather than text.)