I am have a pretty weird issue right now with how i'm sending my data over sockets. My read buffer is getting all the data i'm sending through the socket, but for some reason the File.Write or File.WriteAsync methods i'm calling are not writing all the bytes, or just stopping(I think that's the issue) OR it could be something wrong with the sockets and how they're being read and written too.
So here is a snippet of my code, keep in mind, both of these are on the same file but are different threads.
byte[] Buffer = new Byte[1024];
int bytesRead;
while((bytesRead = ReceiverSocket.Receive(Buffer)) > 0)
{
totalBytesRead += bytesRead;
//myDownload.Write(Buffer, 0, bytesRead);
myDownload.WriteAsync(Buffer, 0, bytesRead);
}
The code above is my receive function for my download client. Keep in mind i'm getting the same results regardless of what write method I use.
And I am simply using this:
byte[] fileData = File.ReadAllBytes(fileToSend);
serverSocket.SendFile(fileToSend);
I send a file with filesize: 10560 and actually retrieve all the bytes over the receive loop, yet when I check my file after i'm done it only receives a portion like so:
TotalBytesRead
1024
2048
3072
...
...8192
9216
10240
10560
Which means it reads all the data, though in my program, it's still stuck inside the loop and when i close it to check the filesize, it's size is 8192, so it seems to me it didn't read the next 3 socket streams.
I'm at a loss for this, is there anything i'm doing wrong? Could this be a threading issue i'm not aware of?
I figured out the answer, I was locking on my
while((bytesRead = ReceiverSocket.Receive(Buffer)) > 0)
due to the socket always waiting on a receive, so the SendFile wouldn't necessarily send 0, so it was always stuck in the while loop, but one I closed the while, it finished writing to file.
Related
I have an exe which simulates a video stream. I connect to it and very occasionally read the expected data, but usually I get only the first 28 bytes and then 65508 bytes of zeros. Assume the video stream is working correctly.
TcpClient tcpClient = new TcpClient ();
int port = 13000;
myIP = IPAddress.Loopback.ToString();
tcpClient.Connect (myIP, port);
NetworkStream netStream = tcpClient.GetStream ();
byte[] bytes = new byte[tcpClient.ReceiveBufferSize];
netStream.Read (bytes, 0, (int)tcpClient.ReceiveBufferSize);
string dataString = Encoding.ASCII.GetString (bytes);
Console.WriteLine("\ndataString: "+dataString.Substring(0,1000));
Console.WriteLine("\nnumber of bytes read: "+bytes.Length);
tcpClient.Close ();
// Closing the tcpClient instance does not close the network stream.
netStream.Close();
How can I make it so that I get the expected output every time?
TCP represents a (bi-directional) stream of data. You're supposed to keep reading from it in a loop, and parsing the data as you need them. It doesn't have a concept of messages - ten writes on one side can result in a single read on the other side just as easily as one write on one side can result in ten reads on the other side.
The contract you have with TCP is as follows:
If there is data in the receive buffer, Read returns immediately, filling the buffer you provided with as much data as is available, up to the length of the buffer. The number of bytes read is the return value of Read.
If there is no data in the receive buffer, Read will block until there's at least a single byte of data. Then it follows as in the first case.
If the other sides shuts down the socket, Read will return zero.
So to get TCP working, you need a loop. How exactly you form the loop depends on what you're trying to do. If you're really working with data that is logically a stream (e.g. audio data), just keep reading as fast as you can and process whatever data you get as it comes in. If you need to send messages, you need to implement a message protocol. If you need a one-off message, you can just keep reading until Read returns zero.
Your case can be handled with the first approach - keep reading until the stream closes, and push the received data forward. Assuming the data is actually a UTF8 stream of text, the basic receiver would look something like this:
using (var client = new TcpClient())
{
tcpClient.Connect(myIP, port);
var stream = client.GetStream();
var buffer = new byte[4096]; // Adapt the size based on what you want to do with the data
var charBuffer = new char[4096];
var decoder = Encoding.UTF8.GetDecoder();
int bytesRead;
while ((bytesRead = stream.Read(buffer, 0, buffer.Length)) != 0)
{
var expectedChars = decoder.GetCharCount(buffer, 0, bytesRead);
if (charBuffer.Length < expectedChars) charBuffer = new char[expectedChars];
var charCount = decoder.GetChars(buffer, 0, bytesRead, charBuffer, 0);
Console.Write(new string(charBuffer, 0, charCount));
}
client.Close();
}
Note that this does no error handling, so don't use it as is in any production code. You would probably also want to use Async methods if you're expecting more than a few simultaneous connections. It's just to illustrate the basic way one would handle a stream of data being received over TCP.
If you want some more insight into dealing with TCP, I have a few very simple examples at https://github.com/Luaancz/Networking. I haven't found any good tutorials or code samples for C#, so if this isn't enough, you'll probably have to dig deeper into the documentation around sockets, TCP and all that.
Or just use an existing networking library, rather than trying to write your own :) TCP is still very low level.
I'm using the following code to send a file over tcp.
If i send many times the same file consecutively to test if it is robust, i receive the first file correctly and the other messed up.
All messed up files have the same incorrect bytes and if i Sleep(a while) all files are transfered correctly. I noticed I must instantiate a new buffer while reading my file to get everything done right. But i don't get why.
I fear my solution to reinstantiate a buffer could be just hiding another major problem. Any suggestion?
using(var fileStream = new FileStream(file, FileMode.Open, FileAccess.Read))
{
using(var binaryReader = new BinaryReader(fileStream))
{
var _sendingBuffer = new byte[BUFFER_SIZE];
int length = (int)fileStream.Length;
int bytesRead = 0;
//Ensure we reached the end of the stream regardless of encoding
while (binaryReader.BaseStream.Position != binaryReader.BaseStream.Length)
{
bytesRead = binaryReader.Read( _sendingBuffer, 0, _sendingBuffer.Length);
_socket.BeginSend(_sendingBuffer, 0, bytesRead, SocketFlags.None, SendFileCallback, null);
//without this i received some messed up data
_sendingBuffer = new byte[BUFFER_SIZE];
}
}
}
BeginSend is an asynchronous operation. It will only be guaranteed to be started after you call it, it won't be finished immediatly. As long as the socket is sending the passed data, that data must not be mutated.
The end of the operation will be signaled through the AsyncCallback callback parameter.
Your problem is exactly that you mutated the transmit buffer while the transmit was still in progress. Creating a new array for each transmit call fixes this.
Other ways to fix the problem:
Use the blocking Socket.Send function which will block until the whole data was sent and the buffer can be reused. This will also make your error handling much easier, because the error will not show up through the AsyncCallback.
Make your complete program acting asynchronously, e.g. using C#5's async Task and async/await functionalities
Therefore:
First read one part of the file asynchronously.
When the async read finishes send it asynchronously through the socket
When this completes and there is more data to read go back to 1.
I'm wanting to do a protocol analysis that uses SSL/TLS fortunately I can install my own certificate and the DNS portion won't be an issue. My problem is what do I use to do this. I've considered using paros but it will be more trouble than it's worth. So I thought I could write two C# applications. The first is the pseudo server and the other is the pseudo client. The both have a tcp connection between the two of them that I can then use wireshark on. The problem is, is that I have very little experience with streams. So if anyone could point me to helpful articles or if the code is pretty short a sample would be great. Thank you in advanced.
It's not terribly hard to Read/Write to/from streams, you can't just connect the streams, you'll need to have your own code to do this. Preferably on it's own thread (or worker process or task or whatever threading concept you need).
public void ConnectStreams(Stream inStream, Stream outStream)
{
byte[] buffer = new byte[1024];
int bytesRead = 0;
while((bytesRead = inStream.Read(buffer, 0, 1024)) != 0)
{
outStream.Write(buffer, 0, bytesRead);
outStream.Flush();
}
}
Basically Streams operate on byte arrays. When we run this line:
while((bytesRead = inStream.Read(buffer, 0, 1024)) != 0)
We are basically saying, perform Read on inStream, put the read bytes into buffer, at index 0 (in buffer) and read a max of 1024 bytes.
Then we assign the return value into bytesRead which is the number of ACTUAL bytes read (between 0 and 1024 in this case) and if that is not equal to 0, continue looping.
Then we simply write it back into the outStream with the buffer containing the data, and the number of bytes actually read. We perform a flush to actually force the output out vs. stacking up in an internal buffer.
When the stream reaches the EOF, .Read will return 0, the loop will exit and you can continue on. This is how you "Connect" two streams at the most simple level.
This question already has answers here:
Receiving data in TCP
(10 answers)
Closed 2 years ago.
I have written a simple TCP client and server. The problem lies with the client.
I'm having some trouble reading the entire response from the server. I must let the thread sleep to allow all the data be sent.
I've tried a few times to convert this code into a loop that runs until the server is finished sending data.
// Init & connect to client
TcpClient client = new TcpClient();
Console.WriteLine("Connecting.....");
client.Connect("192.168.1.160", 9988);
// Stream string to server
input += "\n";
Stream stm = client.GetStream();
ASCIIEncoding asen = new ASCIIEncoding();
byte[] ba = asen.GetBytes(input);
stm.Write(ba, 0, ba.Length);
// Read response from server.
byte[] buffer = new byte[1024];
System.Threading.Thread.Sleep(1000); // Huh, why do I need to wait?
int bytesRead = stm.Read(buffer, 0, buffer.Length);
response = Encoding.ASCII.GetString(buffer, 0, bytesRead);
Console.WriteLine("Response String: "+response);
client.Close();
The nature of streams that are built on top of sockets is that you have an open pipeline that transmits and receives data until the socket is closed.
However, because of the nature of client/server interactions, this pipeline isn't always guaranteed to have content on it to be read. The client and server have to agree to send content over the pipeline.
When you take the Stream abstraction in .NET and overlay it on the concept of sockets, the requirement for an agreement between the client and server still applies; you can call Stream.Read all you want, but if the socket that your Stream is connected to on the other side isn't sending content, the call will just wait until there is content.
This is why protocols exist. At their most basic level, they help define what a complete message that is sent between two parties is. Usually, the mechanism is something along the lines of:
A length-prefixed message where the number of bytes to be read is sent before the message
A pattern of characters used to mark the end of a message (this is less common depending on the content that is being sent, the more arbitrary any part of the message can be, the less likely this will be used)
That said you aren't adhering to the above; your call to Stream.Read is just saying "read 1024 bytes" when in reality, there might not be 1024 bytes to be read. If that's the case, the call to Stream.Read will block until that's been populated.
The reason the call to Thread.Sleep probably works is because by the time a second goes by, the Stream has 1024 bytes on it to read and it doesn't block.
Additionally, if you truly want to read 1024 bytes, you can't assume that the call to Stream.Read will populate 1024 bytes of data. The return value for the Stream.Read method tells you how many bytes were actually read. If you need more for your message, then you need to make additional calls to Stream.Read.
Jon Skeet wrote up the exact way to do this if you want a sample.
Try to repeat the
int bytesRead = stm.Read(buffer, 0, buffer.Length);
while bytesRead > 0. It is a common pattern for that as i remember.
Of course don't forget to pass appropriate params for buffer.
You dont know the size of data you will be reading so you have to set a mechanism to decide. One is timeout and another is using delimiters.
On your example you read whatever data from just one iteration(read) because you dont set the timeout for reading and using default value thats "0" milisecond. So you have to sleep just 1000 ms. You get same effect with using recieve time out to 1000 ms.
I think using lenght of data as prefix is not the real solution because when socket is closed by both sides, socket time-wait situation can not handled properly. Same data can be send to server and cause server to get exception . We used prefix-ending character sequence. After every read we check the data for start and end character sequence, if we cant get end characters, we call another read. But of course this works only if you have the control of server side and client side code.
In the TCP Client / Server I just wrote I generate the packet I want to send to a memory stream, then take the length of that stream and use it as a prefix when sending the data. That way the client knows how many bytes of data it's going to need to read for a full packet.
I have an HTTP connection with the server side using System.IO.Stream.Read to read the HTTP request body message.
The problem is that once in a couple of minutes the server gets stuck on the Read statement and doesn't continue until the socket timeout has been reached or the client has closed the connection.
int bytesRead = 0;
while (bytesRead < contentLength)
{
int got = stream.Stream.Read(buffer.Buffer, bytesRead, contentLength - bytesRead);
bytesRead += got;
}
It could happen if the stream did not have the amount of data specified by contentLength variable.
This is not the case because when following the tcp stream with WireShark I see that the whole message body (as specified by contentLength) has reached the server machine.
It happens only in the first time that the while loop has been "used", i.e. only in the first time that the stream didn't have "contentLength" number of bytes to read in one try and the while loop had to be re-entered.
Why does it get stuck and does not continue reading data?
I wonder if the stream is reporting early termination; you should also look at whether Read returned a non-positive number, i.e.
while (bytesRead < contentLength)
{
int got = stream.Stream.Read(
buffer.Buffer, bytesRead, contentLength - bytesRead);
if(got <= 0) throw new EndOfStreamException(string.Format(
"Expected {0} bytes; {1} bytes received", contentLength, bytesRead));
bytesRead += got;
}
Basically, if the stream has closed, every call to Read will return non-positive (probably 0) - so your while loop will become a tight cycle of "read 0, add 0, read 0, add 0, read 0, add 0, read 0, add 0".
As a final point, your approach suggests you are allocating a byte[] based on the incoming content-length header; just a warning: make sure you sanity-check this and limit it to sane values, otherwise a DOS attack is trivial. Also, if possible I would suggest trying to use the streaming API where possible, to avoid having to load it all into memory at once (unless you have limited the incoming size such that this isn't a concern).
Try this implementation, your count (contentLength - bytesRead) is wrong. It should be the buffer size.
byte[] buffer = new byte[bufferSize];
int count;
while ((count = stream.Stream.Read(buffer, 0, buffer.Length)) != 0)
{
// do something with the buffer using count as the end marker
destination.Write(buffer, 0, count);
}
If you just want a byte array from the stream, which is more like what you are attempting:
byte[] buffer = stream.Stream.ToArray()
Or to copy to another buffer:
byte[] data = stream.Stream.ToArray();
Array.CopyTo(data , buffer.Buffer, data.Length)
I've had similar errors only with clients that didn't flush their streams. MSDN doc on System.IO.Stream.Read says: "The implementation will block until at least one byte of data can be read, in the event that no data is available.". So for some reason there is no data available. I think you could set a certain ReadTimeout and stop waiting for more data after a reasonably short time.
A related question has also been posted here: C# NetworkStream.Read oddity. Maybe his solution can also help you.
It seems like it had something to do with the fact that I was using the Read method of both the NetworkStream (that's the call seen in the code above) and the StreamReader that was created by passing the NetworkStream object to it's constructor.
The NetworkStream.Read was used to read the http request message body while the StreamReader.Read was used to read the rest of the request (startline, headers...).
Although the calls happened synchroniously by the same thread, it may have been the cause to the behavior I experienced.
When changing the code to work only with a Socket and perform the reading directly from the socket, it has fixed the problem.