Develop Reference

Gracefully closing a named pipe and disposing of streams

I have a two-way named pipe. I'm not sure how to shut it down gracefully, though, once I'm done with it - if I close the connection from the client side, the server side throws an exception when it tries to dispose of the StreamReader and StreamWriter I'm using. I'm currently catching it, but that seems like a kludge job to me.
Server side code:
Thread pipeServer = new Thread(ServerThread);
pipeServer.Start();
private void ServerThread(object data)
{
int threadId = Thread.CurrentThread.ManagedThreadId;
log.Debug("Spawned thread " + threadId);
PipeSecurity ps = new PipeSecurity();
SecurityIdentifier sid = new SecurityIdentifier(WellKnownSidType.WorldSid, null);
ps.AddAccessRule(new PipeAccessRule(sid, PipeAccessRights.ReadWrite, System.Security.AccessControl.AccessControlType.Allow));
ps.AddAccessRule(new PipeAccessRule(WindowsIdentity.GetCurrent().Owner, PipeAccessRights.FullControl, System.Security.AccessControl.AccessControlType.Allow));
log.Debug("Pipe security settings set [Thread " + threadId + "]");
NamedPipeServerStream pipeServer =
new NamedPipeServerStream("RDPCommunicationPipe", PipeDirection.InOut, numThreads, PipeTransmissionMode.Message, PipeOptions.None, 0x1000, 0x1000, ps);
log.Debug("Pipe Servers created");
// Wait for a client to connect
log.Info("Pipe created on thread " + threadId + ". Listening for client connection.");
pipeServer.WaitForConnection();
log.Debug("Pipe server connection established [Thread " + threadId + "]");
Thread nextServer = new Thread(ServerThread);
nextServer.Start();
try
{
// Read the request from the client. Once the client has
// written to the pipe its security token will be available.
using (StreamReader sr = new StreamReader(pipeServer))
{
using (StreamWriter sw = new StreamWriter(pipeServer) { AutoFlush = true })
{
// Verify our identity to the connected client using a
// string that the client anticipates.
sw.WriteLine("I am the one true server!");
log.Debug("[Thread " + threadId + "]" + sr.ReadLine());
log.Info(string.Format("Client connected on thread {0}. Client ID: {1}", threadId, pipeServer.GetImpersonationUserName()));
while (!sr.EndOfStream)
{
log.Debug("[Thread " + threadId + "]" + sr.ReadLine());
}
}
}
}
// Catch the IOException that is raised if the pipe is broken
// or disconnected.
catch (IOException e)
{
log.Error("ERROR: " + e);
}
}
Client side code:
class Program
{
static void Main(string[] args)
{
Console.WriteLine("Starting...");
var client = new NamedPipeClientStream(".", "RDPCommunicationPipe", PipeDirection.InOut);
client.Connect();
Console.WriteLine("Pipe connected successfully");
using (StreamReader sr = new StreamReader(client))
{
using (StreamWriter sw = new StreamWriter(client) { AutoFlush = true })
{
string temp;
do
{
temp = sr.ReadLine();
Console.WriteLine(temp);
} while (temp.Trim() != "I am the one true server!");
sw.WriteLine("Message received and understood");
while (!string.IsNullOrEmpty(temp = Console.ReadLine()))
{
sw.WriteLine(temp);
}
}
}
client.Close();
}
}
It works perfectly until I hit enter on an empty line in the client app, which terminates it, closing the client. The server app then throws a System.IO.IOException: Pipe is broken. when it hits the end of the StreamWriter using block. How do I properly dispose of my stream handlers?
(Code based on ideas found here and here.)

I'm currently catching it, but that seems like a kludge job to me.
IMHO, it's about as good as you're going to get, if you want to be a good neighbor and dispose your owned StreamWriter object and still invest a minimum of effort.
That said, it seems to me that in this particular situation, it'd also be fine to comment out the call to Dispose() — or in your case, not use the using statement — and include another comment explaining that at that point in the sequence of execution of your code, you know that all that call is going to do is throw an exception, and so there's no point in making it.
Of course, if you just don't bother disposing the StreamWriter, then you'll want to explicitly dispose your pipe stream. You might also want to use the StreamWriter constructor that has the leaveOpen parameter, and pass true for that parameter, as a way of documenting your intent to not have the StreamWriter own the pipe stream object.
Either way, you're going to wind up leaving object in the finalizer queue, because the exception bypasses the call to GC.SuppressFinalize(), as does (of course) not bothering to call Dispose() at all. As long as you aren't dealing with a high-volume scenario (i.e. lots of these objects), that's probably fine. But it's certainly not ideal.
Unfortunately, named pipes themselves don't have semantics that provide for the kind of "graceful closure" that sockets do. That is, the only way for an endpoint to indicate they are done writing is to disconnect (for a server pipe) or close (for server or client pipes). Neither option leaves the pipe available for reading, so implementing a graceful closure on a pipe requires handshaking within the application protocol itself, rather than relying on the I/O object .
In addition to this inconvenience (which I admit, isn't really directly related to your question), the implementation of PipeStream.Flush() checks to see whether the pipe is writeable. Even though it has no intention of writing anything! It's that last part I find really annoying, and of course directly causes the issue you're asking about. It seems unreasonable to me for code in the .NET Framework to go out of its way to throw exceptions in scenarios where those exceptions cause more trouble than good.
All that said, you do have some other options:
Subclass the NamedPipeServerStream and NamedPipeClientStream types, and override the Flush() method so that it really does do nothing. Or rather, it would be nice if you could do this. But those types are sealed, so you can't.
Alternative to subclassing those types, you can wrap them in your own Stream implementation. This is a lot more of a hassle, especially since you are likely going to want to override all of the async members, at least if you intend to use these objects in any situation where I/O performance is of any interest.
Use separate one-directional pipes for reading and writing. In this implementation, you can close the StreamWriter itself as a way of closing the connection, which results in the correct order of things (i.e. the flush happens before the close on the pipe). This also addresses the graceful closure issue, because with two pipes for each connection, you can have the same basic "half-closed" semantics that sockets have. Of course, this option is significantly complicated by the challenge in identifying which pairs of pipe connections go with each other.
Both of these (that is, the second and third ones, i.e. the ones that are actually possible) have some obvious drawbacks. Having to have your own Stream class is a pain, due to all the duplication of code required. And doubling the pipe object count seems like a drastic way to address the exception (but it could be an acceptable and desirable implementation to support the graceful closure semantics, with the happy side-effect of eliminating the thrown-exception issue with StreamWriter.Dispose()).
Note that in a high-volume scenario (but then, why are you using pipes? 😊 ), throwing and catching exceptions with high frequency could be a problem (they are expensive). So one or the other of these two alternative options would probably be preferable in that case, over either catching the exception and just not bothering to close/dispose your StreamWriter (both of which add inefficiencies that would interfere with a high-volume scenario).

ReaderWriterLockSlim throws LockRecursionException with Socket.BeginReceive

Given the simple socket client class below, connect it to a TCP server (I use SocketTest3, freely available online). Then disconnect the server and wait for a bit. You should get a LockRecursionException.
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Net;
using System.Net.Sockets;
using System.Threading;
namespace SocketRwlTest
{
public class SocketRwlTest
{
private Socket client = new Socket(AddressFamily.InterNetwork,
SocketType.Stream,
ProtocolType.Tcp);
private readonly ReaderWriterLockSlim rwl = new ReaderWriterLockSlim();
private const int maxLength = 200;
public SocketRwlTest(IPAddress address, ushort port)
{
client.Connect(new IPEndPoint(address, port));
ReceiveOne();
}
private void ReceiveOne()
{
rwl.EnterReadLock();
try
{
var inArray = new byte[maxLength];
client.BeginReceive(inArray, 0, maxLength, 0,
new AsyncCallback(ReceivedCallback),
inArray);
}
finally
{
rwl.ExitReadLock();
}
}
private void ReceivedCallback(IAsyncResult ar)
{
client.EndReceive(ar);
ReceiveOne();
}
}
}
I don't understand why it happens in the simplified example given. I know I should stop calling ReceiveOne as soon as I receive a zero-length message, but this is more of an exercise. I wondered if a similar bug could maintain a constant stream of callbacks running in the background and stealing resources without obviously bad things happening. I must admit I wasn't expecting this particular exception.
Question 1: Why does this happen? Are BeginXYZ methods perhaps allowed to execute callbacks instantly, on the same thread? If that's the case, who's to say this couldn't happen during normal runtime?
Question 2: Are there ways to avoid getting this exception while still maintaining the "desired" behaviour in this case? I mean fire a non-stopping stream of callbacks.
I'm using Visual Studio 2010 with .NET 4.

Question 1: Why does this happen? Are BeginXYZ methods perhaps allowed to execute callbacks instantly, on the same thread? If that's the case, who's to say this couldn't happen during normal runtime?
As described by mike z in the comments, the BeginReceive() method is not required to execute asynchronously. If data is available it will execute synchronously, calling the callback delegate in the same thread. This is by definition a recursive call, and so would not be compatible with the use of a non-recursive lock object (such as the ReaderWriterLockSlim you're using here).
This certainly can happen "during normal runtime". I'm not sure I understand the second part of your question. Who's to say it can't happen? No one. It can happen.
Question 2: Are there ways to avoid getting this exception while still maintaining the "desired" behaviour in this case? I mean fire a non-stopping stream of callbacks.
I'm afraid I also don't know what you mean by "fire a non-stopping stream of callbacks".
One obvious workaround is to enable recursion on the ReaderWriterLockSlim object by passing LockRecursionPolicy.SupportsRecursion to its constructor. Alternatively, you could check the IsReadLockHeld property before trying to take the lock.
It is not clear from your code example why you have the lock at all, never mind why it's used in that specific way. It's possible the right solution is to not hold the lock at all while you call BeginReceive(). Use it only while processing the result from EndReceive().

Is it good practice to put try-catch in a loop until all statements in the try block is executed without any exceptions?

I was trying to develop a multicast receiver program and socket initialization was done as shown below:
public void initializeThread()
{
statuscheckthread = new Thread(SetSocketOptions);
statuscheckthread.IsBackground = true;
}
private void Form1_Load(object sender, EventArgs e)
{
rxsock = new Socket(AddressFamily.InterNetwork, SocketType.Dgram, ProtocolType.Udp);
iep = new IPEndPoint(IPAddress.Any, 9191);
rxsock.Bind(iep);
ep = (EndPoint)iep;
initializeThread();
statuscheckthread.Start();
}
public void SetSocketOptions()
{
initializeThread(); //re-initializes thread thus making it not alive
while (true)
{
if (NetworkInterface.GetIsNetworkAvailable())
{
bool sockOptnSet = false;
while (!sockOptnSet)
{
try
{
rxsock.SetSocketOption(SocketOptionLevel.IP, SocketOptionName.AddMembership, new MulticastOption(IPAddress.Parse("224.50.50.50")));
rxsock.SetSocketOption(SocketOptionLevel.IP, SocketOptionName.MulticastTimeToLive, 64);
sockOptnSet = true;
}
catch
{
//Catch exception here
}
}
}
break; // Break out from loop once socket options are set
}
}
When my PC is not connected to a network, SetSocketOption method was throwing exception and even after network is connected,
I was unable to receive data because socket options are not set.
To avoid this I used a thread which runs in the background checking
for network availability and once network is available, it sets the socket options.
It works properly in some PC's but in some others, NetworkInterface.GetIsNetworkAvailable()
returned true before network got connected
(while network was being identified) .
So, to make sure Socket options are set, I used a bool variable sockOptnSet
which is set as
true if all the statements in the try block is executed as shown inside the method public void SetSocketOptions()
This program works fine in all PC's I tried, but I am doubtful about how much I can rely on this to work.
My questions are:
1) Is this good practice?
2) If not, what are the possible errors or problems it may cause? And how can I implement it in a better way?

Is this a good practice?
No, not a good practice. The vast majority of exceptions, including your first one, fall in the category of vexing exceptions. Software is supposed to work, worked well when you tested it, but doesn't on the user's machine. Something went wrong but you do not know what and there isn't anything meaningful that you can do about it. Trying to keep your program going is not useful, it cannot do the job it is supposed to do. In your case, there's no hope that the socket is ever going to receive data when there is no network. And, as you found out, trying to work around the problem just begets more problems. That's normal.
If this is bad practice, how can I implement it in a better way?
You need help from a human. The user is going to have to setup the machine to provide a working network connection. This requires a user interface, you must have a way to tell a human what he needs to do to solve your problem. You can make that as intricate or as simple as you desire. Just an error message, a verbatim copy of the Exception.Message can be enough. Writing an event handler for the AppDomain.CurrentDomain.UnhandledException event is a very good (and required) strategy. Microsoft spent an enormous amount of effort to make exception messages as clear and helpful as possible, even localizing them for you in the user's native language, you want to take advantage of that. Even if the exception message is mystifying, a quick Google query on the message text returns hundreds of hits. With this event handler in place, you don't have to do anything special. Your program automatically terminates and your user knows what to do about it.
You can certainly make it more intricate, you discovered that SetSocketOption() is liable to fail right after the network becomes available but works when you wait long enough. So this is actually an error condition that you can work around, just by waiting long enough. Whether you should write the code to handle this is something that you have to decide for yourself. It is something you write when you have enough experience with the way your program behaves, you never write it up front. Usually as a result from feedback from the users of your program.

Some good advice in the comments, lets' expand on it.
Firstly, I would put all this socket code in to its' own class, outside of the form. This makes it its' own entity and semantically easier to understand. This class could have a property Initialised, which is initially set to false. The first thing you do in your form is call an Initialise method on this class which attempts to set socket options and catches the relevant exceptions if the network is not available. If it is available, we set our Initialised property to true.
If not available, we set a single timeout (see System.Threading.Timer) that calls this same function (potentially with a retry count) after 'x' seconds. Once again we'll find ourselves back in this Initialise function, perhaps with a retry count mentioned at the beginning. Once again, if it is available, we're good - if not, set the timer again. Eventually, after 'x' retries if we're not initialised we can throw an exception or set some other failure property to indicate that we can't proceed.
Your Form class can periodically check (or hook in to an event) to determine whether the socket is now ready for communication. In case of failure you can gracefully quit out, or because our class is nice and abstracted, attempt to start the whole process again.

Problem with sockets and OutOfMemory error

I have a huge problem. Trying to create an app that has to have two parts: server and client side. Those two parts have to communicate somehow and exchange objects. I have decides to use Sockets because i'm not familiar with WCF, and i can test both parts on same computer (just put them to listen at 127.0.0.1 address).
Now, when i try to send some "custom" serializable object from client i got "OutOfMemory" exception at server side! I read about Sockets, ways to send/receive objects, i have tried some code i found on net but no luck! I have no idea what's wrong with my code.
Here's my code:
This is test class defined in code of both sides:
[Serializable]
class MyClass
{
public string msg="default";
}
Client-side sending code (works fine):
private void cmdSendData_Click(object sender, System.EventArgs e)
{
try
{
MyClass test = new MyClass();
NetworkStream ns = new NetworkStream(m_socWorker); //m_socWorker is socket
BinaryWriter bw = new BinaryWriter(ns);
MemoryStream ms = new MemoryStream();
BinaryFormatter bf = new BinaryFormatter();
bf.Serialize(ms, test);
bw.Write(ms.ToArray());
MessageBox.Show("Length is: " + ms.ToArray().Length); //length is 152!
ns.Close();
}
catch(System.Net.Sockets.SocketException se)
{
MessageBox.Show (se.Message );
}
}
Server-side code (the one that cause problems):
public void OnDataReceived(IAsyncResult asyn)
{
try
{
CSocketPacket theSockId = (CSocketPacket)asyn.AsyncState ;
NetworkStream ns = new NetworkStream(m_socWorker);
byte[] buffer = new byte[1024];
ns.Read(buffer, 0, buffer.Length);
BinaryFormatter bin = new BinaryFormatter();
MemoryStream mem = new MemoryStream(buffer.Length);
mem.Write(buffer, 0, buffer.Length);
mem.Seek(0, SeekOrigin.Begin);
MyClass tst = (MyClass)bin.Deserialize(mem); //ERROR IS THROWN HERE!
MessageBox.Show(tst.msg);
theSockId.thisSocket.EndReceive(asyn);
WaitForData(m_socWorker);
}
catch (ObjectDisposedException )
{
System.Diagnostics.Debugger.Log(0,"1","\nOnDataReceived: Socket has been closed\n");
}
catch(SocketException se)
{
MessageBox.Show (se.Message );
}
}
So, i got exception when i try to deserialize. Have no idea what's wrong.
I have threatened my code "if you continue causing problems i'll report you to StackOverflow guys" so here i'm :)

There is some very odd code there that:
assumes we read 1024 bytes without checking
copies the 1024 buffer
assumes the serialized data is 1024 bytes, no more no less
deserializes from that
IMO there is your error; you should be reading the correct number of bytes from the stream (usually in a loop). Generally, you would be looping, checking the return value from Read until either we have read the amount of data we wanted, or we get EOF (return <= 0).
Or better; use serializers that do this for you... For example, protobuf-net has SerializeWithLengthPrefix and DeserializeWithLengthPrefix that handle all the length issues for you.
Since you mention "custom" serialization - if you are implementing ISerializable it is also possible that the problem is in there - but we can't see that without code. Besides, the current buffer/stream is so broken (sorry, but it is) that I doubt it is getting that far anyway.

First, while I'm not certain if this is the cause of your issue directly, you have a serious issue with your reading logic.
You create a 1024 byte buffer and read into it without checking to see how much was actually read; if the incoming buffer only has 56 bytes, you'll only read 56 bytes (unless you use a blocking read on the socket itself, which could time out). Likewise, your buffer could have 2000 bytes in it, which means you'd have 976 bytes left in the buffer that won't get processed until you receive more data. That could be an entire object, and the client could be waiting on a response to it before it sends any more.
You then take that buffer and copy it again into a MemoryStream. Rather than doing this, just take the overload of the MemoryStream constructor that takes a buffer. You won't be copying the data that way.
You call EndReceive after you've processed the incoming data; while this may not actually cause an error, it's not in the spirit of the Begin/End old-style async pattern. You should call EndXXX at the beginning of your callback to get the result.
I realize that this is not a direct answer to your question, but you really need to reconsider your decision not to use WCF.
I was in the same boat as you a couple of months ago; I had not used WCF before, and I hadn't bothered to look at how things work in it. It was a very large black box to me, and I had done socket-based communication on other platforms, so it was a known quantity. Looking back, my choice to take the plunge into WCF was the best decision I could have made. Once you've wrapped your head around some of the concepts (bindings, contracts, and how to use the various attributes), development of the service is simple and you don't have to spend your time writing plumbing.
NetTcpBinding provides a TCP-based binding that can support long-lived connections and sessions (which is how I'm using it), and even takes care of keep-alive messages to keep the connection open via Reliable Sessions. It's as simple as turning on a flag. If you need something more interopable (meaning cross-platform), you can write your own binding that does this and keep your code as-is.
Look at some of the TCP WCF examples; it won't take you terribly long to get something up and running, and once you've reached that point, modification is as simple as adding a function to your interface, then a corresponding function on your service class.

Is it necessary to wrap StreamWriter in a using block?

A few days ago I posted some code like this:
StreamWriter writer = new StreamWriter(Response.OutputStream);
writer.WriteLine("col1,col2,col3");
writer.WriteLine("1,2,3");
writer.Close();
Response.End();
I was told that instead I should wrap StreamWriter in a using block in case of exceptions. Such a change would make it look like this:
using(StreamWriter writer = new StreamWriter(Response.OutputStream))
{
writer.WriteLine("col1,col2,col3");
writer.WriteLine("1,2,3");
writer.Close(); //not necessary I think... end of using block should close writer
}
Response.End();
I am not sure why this is a valuable change. If an exception occurred without the using block, the writer and response would still be cleaned up, right? What does the using block gain me?

Nope the stream would stay open in the first example, since the error would negate the closing of it.
The using operator forces the calling of Dispose() which is supposed to clean the object up and close all open connections when it exits the block.

I'm going to give the dissenting opinion. The answer to the specific question "Is it necessary to wrap StreamWriter in a using block?" is actually No. In fact, you should not call Dispose on a StreamWriter, because its Dispose is badly designed and does the wrong thing.
The problem with StreamWriter is that, when you Dispose it, it Disposes the underlying stream. If you created the StreamWriter with a filename, and it created its own FileStream internally, then this behavior would be totally appropriate. But if, as here, you created the StreamWriter with an existing stream, then this behavior is absolutely The Wrong Thing(tm). But it does it anyway.
Code like this won't work:
var stream = new MemoryStream();
using (var writer = new StreamWriter(stream)) { ... }
stream.Position = 0;
using (var reader = new StreamReader(stream)) { ... }
because when the StreamWriter's using block Disposes the StreamWriter, that will in turn throw away the stream. So when you try to read from the stream, you get an ObjectDisposedException.
StreamWriter is a horrible violation of the "clean up your own mess" rule. It tries to clean up someone else's mess, whether they wanted it to or not.
(Imagine if you tried this in real life. Try explaining to the cops why you broke into someone else's house and started throwing all their stuff into the trash...)
For that reason, I consider StreamWriter (and StreamReader, which does the same thing) to be among the very few classes where "if it implements IDisposable, you should call Dispose" is wrong. Never call Dispose on a StreamWriter that was created on an existing stream. Call Flush() instead.
Then just make sure you clean up the Stream when you should. (As Joe pointed out, ASP.NET disposes the Response.OutputStream for you, so you don't need to worry about it here.)
Warning: if you don't Dispose the StreamWriter, then you do need to call Flush() when you're done writing. Otherwise you could have data still being buffered in memory that never makes it to the output stream.
My rule for StreamReader is, pretend it doesn't implement IDisposable. Just let it go when you're done.
My rule for StreamWriter is, call Flush where you otherwise would have called Dispose. (This means you have to use a try..finally instead of a using.)

If an exception occurs without the using block and kills the program, you will be left with openconnections. The using block will always close the connection for you, similiar to if you were to use try{}catch{}finally{}

Eventually, the writer will be cleaned up. When this happens is up to the garbage collector, who will notice that the Dispose for the command has not been called, and invoke it. Of course, the GC may not run for minutes, hours or days depending on the situation. If the writer is holding an exclusive lock on say, a file, no other process will be able to open it, even though you're long finished.
The using block ensures the Dispose call is always made, and hence that the Close is always called, regardless of what control flow occurs.

Wrapping the StreamWriter in a using block is pretty much equivalent of the following code:
StreamWriter writer;
try
{
writer = new StreamWriter(Response.OutputStream);
writer.WriteLine("col1,col2,col3");
writer.WriteLine("1,2,3");
}
catch
{
throw;
}
finally
{
if (writer != null)
{
writer.Close();
}
}
While you could very well write this code yourself, it is so much easier to just put it in a using block.

In my opinion it's necessary to wrap any class that implements IDisposable in a using block. The fact that a class implements IDisposable means that the class has resources that need to be cleaned up.

My rule of thumb is, if I see Dispose listed in intellisense, I wrap it in a using block.

the using block calls dispose() when it ends. It's just a handy way of ensuring that resources are cleaned up in a timely manner.

In almost every case, if a class implements IDisposable, and if you're creating an instance of that class, then you need the using block.

While it's good practice to always dipose disposable classes such as StreamWriter, as others point out, in this case it doesn't matter.
Response.OutputStream will be disposed by the ASP.NET infrastructure when it's finished processing your request.
StreamWriter assumes it "owns" a Stream passed to the constructor, and will therefore Close the stream when it's disposed. But in the sample you provide, the stream was instantiated outside your code, so there will be another owner who is responsible for the clean-up.