Develop Reference

C# StreamReader causing +200ms overhead to read data from source

There is an issue where we are seeing some periodic +200ms overhead on reading the Input Stream from a Stream Reader when there is load on the system. I am wondering has anyone else seen this and if they have done anything to fix it?
The following is the code:
string requestBody;
var streamReaderTime = Stopwatch.StartNew();
using (var streamReader = new StreamReader(context.Request.InputStream, context.Request.ContentEncoding))
{
var allLines = streamReader.ReadLines();
var request = new StringBuilder();
allLines.ForEach(line => request.Append(line));
requestBody = request.ToString();
}
streamReaderTime.Stop();
ReadLine is just as follows:
public static IEnumerable<string> ReadLines(this StreamReader reader)
{
while (!reader.EndOfStream)
{
yield return reader.ReadLine();
}
}
Note: Using ReadLines() or ReadToEnd() makes very little difference if any.
We run performance tests overnight and we are seeing the following behavior just from graphing streamReaderTime.
A single request takes between 45ms and 70ms to execute but it can be seen from the screenshot that it is adding on a fixed value and sometimes an even bigger spike. I saw it before being at around 1.5 seconds.
If anyone has any solutions/suggestions it would be greatly appreciated.
Edit : I did have ReadToEnd() instead of ReadLines() and that got rid of the StringBuilder but it was still the same overhead. Is there an alternative to StreamReader, just to test out even? It does seem like GC cost since having a request ever ten seconds does not effect it, but the exact same request per second will cause this overhead to happen. Also I am not able to reproduce it locally either, it is only in the virtual environment that this is happening.

This issue is not with the above code at all. The issue is from the caller. The service calling is using a library that is cutting the connections to early and that overhead is the connection being re-established again.

StreamReader ReadLine returns null when not EOF

Having a strange problem that I've never encountered nor heard of happening. It seems that occasionally, the ReadLine() function of the StreamReader class will return NULL, as if it's at the end of the file, BUT it's not.
My log file indicates that everything is happening just as if it had actually reached the end of the file, but yet it's only processing part of it. There doesn't appear to be any consistency, because if I restart the process from scratch, the whole file is processed without incident.
Clearly, there is nothing funky in the file itself, or it would do this on the same line each time, plus it has happened with a few different files, and each time they are re-run, it works fine.
Anyone run across anything similar, or have any suggestions on what might be causing such a thing?
Thanks,
Andrew
Sample:
line = _readerStream.ReadLine();
if (null != line)
{
eventRetVal = loadFileLineEvent(line);
}
else
{
// do some housecleaning and log file completed
}
_readerStream is the stream which has been opened elsewhere.
loadFileLineEvent is a delegate that gets passed in which processes the line. This has its own error handling (with logging), so there's no issue in there.
The routine above (not shown in its entirety) has error handling around it also (with logging), which is not being triggered either.
It's getting to the "else" and logging that it reached the end of the file, but it's obvious from the number of records I got that it didn't.

Have you tried a more traditional approach to reading the stream? This way your checking for the end of the stream before reading the next potentially empty/null line. Seems like your code should work, but with a possible null exception thrown for trying to read a line that doesn't exists(not sure if SR throws for that though).
using (StreamReader SR = new StreamReader(OFD.FileName))
{
while (!SR.EndOfStream)
{
string CurrentLine = SR.ReadLine();
var eventRetVal = loadFileLineEvent(CurrentLine);
}
}

No idea why tcpClient isn't working for me

I've tried checking the server:port with telnet and I'm getting the expected results. So either writer.Write() or reader.ReadLine() isn't working cause I get nothing from the server.
TcpClient socket = new TcpClient(hostname, port);
if (!socket.Connected) {
Console.WriteLine("Failed to connect!");
return;
}
TextReader reader = new StreamReader(socket.GetStream());
TextWriter writer = new StreamWriter(socket.GetStream());
writer.Write("PING");
writer.Flush();
String line = null;
while ((line = reader.ReadLine()) != null) {
Console.WriteLine(line);
}
Console.WriteLine("done");
EDIT: I might have found the issue. This code was based off examples I found on the web. I tried another irc server: open.ircnet.net:6669 and I got a response:
:openirc.snt.utwente.nl 020 * :Please wait while we process your connection.
It seems as if I probably need to run the reader in a Thread so it can just constantly wait for a response. However it does seem weird that the program got caught on the while loop without ever printing done to the console.

I think you need to provide further details. I'm just going to assume that because you can easily telnet to the server using the same port your problem lies in the evaluation of the Connected property...
if (!socket.Connected) {
Console.WriteLine("Failed to connect!");
return;
}
this is wrong because Microsoft clearly specifies in the documentation that the Connected property is not reliable
Because the Connected property only reflects the state of the connection as of the most recent operation, you should attempt to send or receive a message to determine the current state. After the message send fails, this property no longer returns true. Note that this behavior is by design. You cannot reliably test the state of the connection because, in the time between the test and a send/receive, the connection could have been lost. Your code should assume the socket is connected, and gracefully handle failed transmissions.
That said, you should not use this property to determine the state of the connection. Needless to say that using this property to control the flow of your console app will result in unexpected results.
Suggestion
Remove the evaluation of the Connected property
Wrap your GetStream and Write method calls in a try/catch block to handle network communication errors

reader.ReadLine() will just wait for any data to arrive. If no data arrive, it seems to hang. That's a feature of tcp (I don't like it either). You need to find out how the end of the message is defined and stop based on that end criterion. Be careful, the end of message identifier may be split into two or more lines...

RFC for ping says that the server may not respond to it & such connections has to be closed after a time. Please check the RFC: https://www.rfc-editor.org/rfc/rfc1459#section-4.6.2

Sending multiple data in TCPSocket

I'm trying to create a chat with file transfer application using TCPSocket and here is my code..
SENDER:
public void sendData(string message)
{
StreamWriter streamWriter = new StreamWriter(netStream); // netStream is
// connected
streamWriter.WriteLine(message);
streamWriter.WriteLine(message);
logs.Add(string.Format("Message Sent! :{0}", message));
//netStream.Flush();
streamWriter.Flush();
}
RECEIVER:
private void ReceiveData()
{
StreamReader streamReader = new StreamReader(ChatNetStream);
StringBuilder dataAppends = new StringBuilder();
bool doneTransfer = false;
string data;
while (!doneTransfer)
{
while ((data = streamReader.ReadLine()) != null)
{
dataAppends.Append(data);
}
doneTransfer = true;
//ChatNetStream.Close();
//streamReader
}
//do whatever i want with dataAppends.ToString() here..
ReceiveData()
}
the problem is i always turn into infinite loop inside this statement
while ((data = streamReader.ReadLine()) != null)
{
dataAppends.Append(data);
}
even if i put streamWriter.Flush() on my sender..
do i need to close/dispose the netStream/NetworkStream?
anyway, can i use only 1 socket or connection to send a File and send a chat at the same time..? or do i need to use a new socket connection everytime i send a file..

You get an infinite loop because StreamReader.ReadLine will only return null when the end of the stream is reached. For a network stream, "end of stream" means "the other side has closed its half of the connection". Since the other side is your client, and it keeps the connection open while waiting for the user to type in more data, you will end up with an infinite loop.
What you want to do instead is fire off an operation that only completes if there is more data to read. There are two ways to go about this: either use a blocking read operation (on a dedicated thread, so that you don't block your application's other processing while waiting for messages), or use an async (event- or callback-based) approach.
For the synchronous (blocking) approach, see the documentation on NetworkStream.Read which includes example code that shows how to check if there is incoming data and how you can read it. The one point you absolutely need to know here is that when Read returns zero, it means that all data has been read and the connection has been closed from the other side (so you should close your end as well and not loop; the client has disconnected).
For low-level async network reads, the relevant operation is NetworkStream.BeginRead, which comes with its own example.
Both approaches are lower-level than what you currently have and will require you to manually assemble data inside a buffer and decide when "enough data" (i.e. a full line) has accumulated for you to process. You will then have to carefully pull that data out of the buffer and continue.
For a higher-level approach that still allows you some degree of orchestrating things, look into using client sockets (and in particular the two sync and async options there). This functionality is introduced by the TcpClient (and server-side the corresponding TcpListener) classes.
Finally, as jValdron's comment says, you will either need a separate connection for transferring file data or engineer some custom protocol that allows you to interleave multiple kinds of data over the same network stream. The second solution is has generally more technical merit, but it will also be harder for you to implement correctly.

Checkout the BasicSend example in networkComms.net which demonstrates a simple chat application using an open source library.

Read contents from SslStream

I have an application using which I execute IMAP commands using:
TcpClient to connect to the IMAP server
SslStream to write and read commands
Problem:
Cannot read the complete ouput content from the stream
while loop on the SslStream.Read seems not to work
StreamReader.ReadLine, ReadToEnd, Read methods do not work
Sample code:
while ((l = reader.ReadLine()) != null)
{
output.AppendLine(l);
}
This code snippet would read 1 to 2 lines and hang in reader.Readline().
Workaround I tried with setting the ReadTimeout property:
try
{
_output=new byte[_tcpclient.ReceiveBufferSize];
_sslstream.Read(_output, 0, _output.Length);
textBox1.Text = Encoding.ASCII.GetString(_output);
}
catch (IOException ex)
{
textBox1.Text ="ERROR !! " + ex.Message;
}
Help:
How can I read the complete output of a command from the stream?
Note: I do not want to use any third party libraries.

The TCP stream cannot know whether the current response has finished. All it knows is whether it has just received data on the wire; it cannot know whether the next packet is going to come right now (a multi-packet response) or whether it will come much later (if the response is finished).
Instead, you need to predict when you'll get more data; you should keep reading until you receive a tagged completion response, as documented in the IMAP protocol.
However, IMAP seems to be intended to be read continuously on a background thread, since the server can send you information at any time. Therefore, you probably ought to have a separate thread which is always in ReadLine().