Develop Reference

Gracefully closing a named pipe and disposing of streams

I have a two-way named pipe. I'm not sure how to shut it down gracefully, though, once I'm done with it - if I close the connection from the client side, the server side throws an exception when it tries to dispose of the StreamReader and StreamWriter I'm using. I'm currently catching it, but that seems like a kludge job to me.
Server side code:
Thread pipeServer = new Thread(ServerThread);
pipeServer.Start();
private void ServerThread(object data)
{
int threadId = Thread.CurrentThread.ManagedThreadId;
log.Debug("Spawned thread " + threadId);
PipeSecurity ps = new PipeSecurity();
SecurityIdentifier sid = new SecurityIdentifier(WellKnownSidType.WorldSid, null);
ps.AddAccessRule(new PipeAccessRule(sid, PipeAccessRights.ReadWrite, System.Security.AccessControl.AccessControlType.Allow));
ps.AddAccessRule(new PipeAccessRule(WindowsIdentity.GetCurrent().Owner, PipeAccessRights.FullControl, System.Security.AccessControl.AccessControlType.Allow));
log.Debug("Pipe security settings set [Thread " + threadId + "]");
NamedPipeServerStream pipeServer =
new NamedPipeServerStream("RDPCommunicationPipe", PipeDirection.InOut, numThreads, PipeTransmissionMode.Message, PipeOptions.None, 0x1000, 0x1000, ps);
log.Debug("Pipe Servers created");
// Wait for a client to connect
log.Info("Pipe created on thread " + threadId + ". Listening for client connection.");
pipeServer.WaitForConnection();
log.Debug("Pipe server connection established [Thread " + threadId + "]");
Thread nextServer = new Thread(ServerThread);
nextServer.Start();
try
{
// Read the request from the client. Once the client has
// written to the pipe its security token will be available.
using (StreamReader sr = new StreamReader(pipeServer))
{
using (StreamWriter sw = new StreamWriter(pipeServer) { AutoFlush = true })
{
// Verify our identity to the connected client using a
// string that the client anticipates.
sw.WriteLine("I am the one true server!");
log.Debug("[Thread " + threadId + "]" + sr.ReadLine());
log.Info(string.Format("Client connected on thread {0}. Client ID: {1}", threadId, pipeServer.GetImpersonationUserName()));
while (!sr.EndOfStream)
{
log.Debug("[Thread " + threadId + "]" + sr.ReadLine());
}
}
}
}
// Catch the IOException that is raised if the pipe is broken
// or disconnected.
catch (IOException e)
{
log.Error("ERROR: " + e);
}
}
Client side code:
class Program
{
static void Main(string[] args)
{
Console.WriteLine("Starting...");
var client = new NamedPipeClientStream(".", "RDPCommunicationPipe", PipeDirection.InOut);
client.Connect();
Console.WriteLine("Pipe connected successfully");
using (StreamReader sr = new StreamReader(client))
{
using (StreamWriter sw = new StreamWriter(client) { AutoFlush = true })
{
string temp;
do
{
temp = sr.ReadLine();
Console.WriteLine(temp);
} while (temp.Trim() != "I am the one true server!");
sw.WriteLine("Message received and understood");
while (!string.IsNullOrEmpty(temp = Console.ReadLine()))
{
sw.WriteLine(temp);
}
}
}
client.Close();
}
}
It works perfectly until I hit enter on an empty line in the client app, which terminates it, closing the client. The server app then throws a System.IO.IOException: Pipe is broken. when it hits the end of the StreamWriter using block. How do I properly dispose of my stream handlers?
(Code based on ideas found here and here.)

I'm currently catching it, but that seems like a kludge job to me.
IMHO, it's about as good as you're going to get, if you want to be a good neighbor and dispose your owned StreamWriter object and still invest a minimum of effort.
That said, it seems to me that in this particular situation, it'd also be fine to comment out the call to Dispose() — or in your case, not use the using statement — and include another comment explaining that at that point in the sequence of execution of your code, you know that all that call is going to do is throw an exception, and so there's no point in making it.
Of course, if you just don't bother disposing the StreamWriter, then you'll want to explicitly dispose your pipe stream. You might also want to use the StreamWriter constructor that has the leaveOpen parameter, and pass true for that parameter, as a way of documenting your intent to not have the StreamWriter own the pipe stream object.
Either way, you're going to wind up leaving object in the finalizer queue, because the exception bypasses the call to GC.SuppressFinalize(), as does (of course) not bothering to call Dispose() at all. As long as you aren't dealing with a high-volume scenario (i.e. lots of these objects), that's probably fine. But it's certainly not ideal.
Unfortunately, named pipes themselves don't have semantics that provide for the kind of "graceful closure" that sockets do. That is, the only way for an endpoint to indicate they are done writing is to disconnect (for a server pipe) or close (for server or client pipes). Neither option leaves the pipe available for reading, so implementing a graceful closure on a pipe requires handshaking within the application protocol itself, rather than relying on the I/O object .
In addition to this inconvenience (which I admit, isn't really directly related to your question), the implementation of PipeStream.Flush() checks to see whether the pipe is writeable. Even though it has no intention of writing anything! It's that last part I find really annoying, and of course directly causes the issue you're asking about. It seems unreasonable to me for code in the .NET Framework to go out of its way to throw exceptions in scenarios where those exceptions cause more trouble than good.
All that said, you do have some other options:
Subclass the NamedPipeServerStream and NamedPipeClientStream types, and override the Flush() method so that it really does do nothing. Or rather, it would be nice if you could do this. But those types are sealed, so you can't.
Alternative to subclassing those types, you can wrap them in your own Stream implementation. This is a lot more of a hassle, especially since you are likely going to want to override all of the async members, at least if you intend to use these objects in any situation where I/O performance is of any interest.
Use separate one-directional pipes for reading and writing. In this implementation, you can close the StreamWriter itself as a way of closing the connection, which results in the correct order of things (i.e. the flush happens before the close on the pipe). This also addresses the graceful closure issue, because with two pipes for each connection, you can have the same basic "half-closed" semantics that sockets have. Of course, this option is significantly complicated by the challenge in identifying which pairs of pipe connections go with each other.
Both of these (that is, the second and third ones, i.e. the ones that are actually possible) have some obvious drawbacks. Having to have your own Stream class is a pain, due to all the duplication of code required. And doubling the pipe object count seems like a drastic way to address the exception (but it could be an acceptable and desirable implementation to support the graceful closure semantics, with the happy side-effect of eliminating the thrown-exception issue with StreamWriter.Dispose()).
Note that in a high-volume scenario (but then, why are you using pipes? 😊 ), throwing and catching exceptions with high frequency could be a problem (they are expensive). So one or the other of these two alternative options would probably be preferable in that case, over either catching the exception and just not bothering to close/dispose your StreamWriter (both of which add inefficiencies that would interfere with a high-volume scenario).

StreamWriter ReadLine method hangs program

I am trying to get some basic multi-threading working in C#. There's no clear tutorial that I've found so bear with my sub-par code:
class Program
{
private static TcpListener listener;
private static List<TcpClient> clients;
static void Main(string[] args)
{
listener = new TcpListener(IPAddress.Any, 1337);
clients = new List<TcpClient>();
StartListening();
Console.WriteLine("Accepting clients..");
Console.WriteLine("Press any key to exit");
Console.ReadKey();
}
static async void StartListening()
{
listener.Start();
while (true)
{
clients.Add(await listener.AcceptTcpClientAsync().ConfigureAwait(false));
Console.WriteLine("Client connected!");
HandleClient(clients[clients.Count - 1], clients.Count);
}
}
static async void HandleClient(TcpClient c, int number)
{
Console.WriteLine($"Getting client #{number}'s handshake..");
var ns = c.GetStream();
var sr = new StreamReader(ns);
var handshake = sr.ReadLine();
Console.WriteLine("Client {0}'s handshake: {1}", number, handshake);
}
}
Right, so what I'm trying to achieve in the server program is:
1. Accepting client
2. Receiving handshake, print it to console
3. Add to the clients list
The accepting client part works, however the program just stops just after the first line of HandleClient(). I've tried waiting for absurd amounts of time (1000ms) and even then it just doesn't receive anything, nor does it throw an exception (i.e. it stays connected). What am I doing wrong here?
Client code is here if you need it!

Your client code is broken here - you're never flushing it:
var sw = new StreamWriter(ns);
Thread.Sleep(1000);
sw.WriteLine(handshake);
There's no need for the Thread.Sleep call, but you should flush the StreamWriter if you want the data to actually be sent to the network layer immediately:
sw.Flush();
While (as Stephen Cleary notes) this wouldn't mean it necessarily got sent immediately, it's reasonable to expect it to be sent "reasonably soon" after flushing.

I am trying to get some basic multi-threading working in C#.
I'm always amazed at devs who try to learn concurrency by writing a bare-metal TCP/IP server (when they've never programmed TCP/IP previously). Concurrency and TCP/IP are two extremely difficult topics; trying to learn them at the same time is almost certainly going to end in disaster. It's just really odd that this is such a common approach.
If you want to learn asynchrony, I recommend my async intro and article on best practices (where, among other things, you'll learn the principle "avoid async void").
If you want to learn TCP/IP, I recommend my TCP/IP .NET FAQ blog series. However, writing TCP/IP code is almost never necessary, and since you can change both the client and server, I strongly recommend using SignalR instead.
I suspect that you actually want to learn async, and the TCP/IP is just getting in the way. So, drop the harder topic (TCP/IP) and use async with simpler communications such as WebAPI or SignalR.
(As far as the actual problem you saw, it is indeed caused by buffering, but it's important to note that flushing the buffer does not cause the data to be send over the network immediately; it is only sent to the networking layer immediately. A proper solution would need message framing.)

As Jon Skeet mentioned, to make sure the content is not just sitting in a buffer, you should Flush it.
Your client should look like so:
var sw = new StreamWriter(ns);
Thread.Sleep(1000); // let the server catch up
sw.WriteLine(handshake);
sw.Flush();
Console.WriteLine("Staying connected.. Press ESCAPE to exit.");
EDIT: Forgot to mention... That Thread.Sleep(1000); there... you can safely remove it.

ReaderWriterLockSlim throws LockRecursionException with Socket.BeginReceive

Given the simple socket client class below, connect it to a TCP server (I use SocketTest3, freely available online). Then disconnect the server and wait for a bit. You should get a LockRecursionException.
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Net;
using System.Net.Sockets;
using System.Threading;
namespace SocketRwlTest
{
public class SocketRwlTest
{
private Socket client = new Socket(AddressFamily.InterNetwork,
SocketType.Stream,
ProtocolType.Tcp);
private readonly ReaderWriterLockSlim rwl = new ReaderWriterLockSlim();
private const int maxLength = 200;
public SocketRwlTest(IPAddress address, ushort port)
{
client.Connect(new IPEndPoint(address, port));
ReceiveOne();
}
private void ReceiveOne()
{
rwl.EnterReadLock();
try
{
var inArray = new byte[maxLength];
client.BeginReceive(inArray, 0, maxLength, 0,
new AsyncCallback(ReceivedCallback),
inArray);
}
finally
{
rwl.ExitReadLock();
}
}
private void ReceivedCallback(IAsyncResult ar)
{
client.EndReceive(ar);
ReceiveOne();
}
}
}
I don't understand why it happens in the simplified example given. I know I should stop calling ReceiveOne as soon as I receive a zero-length message, but this is more of an exercise. I wondered if a similar bug could maintain a constant stream of callbacks running in the background and stealing resources without obviously bad things happening. I must admit I wasn't expecting this particular exception.
Question 1: Why does this happen? Are BeginXYZ methods perhaps allowed to execute callbacks instantly, on the same thread? If that's the case, who's to say this couldn't happen during normal runtime?
Question 2: Are there ways to avoid getting this exception while still maintaining the "desired" behaviour in this case? I mean fire a non-stopping stream of callbacks.
I'm using Visual Studio 2010 with .NET 4.

Question 1: Why does this happen? Are BeginXYZ methods perhaps allowed to execute callbacks instantly, on the same thread? If that's the case, who's to say this couldn't happen during normal runtime?
As described by mike z in the comments, the BeginReceive() method is not required to execute asynchronously. If data is available it will execute synchronously, calling the callback delegate in the same thread. This is by definition a recursive call, and so would not be compatible with the use of a non-recursive lock object (such as the ReaderWriterLockSlim you're using here).
This certainly can happen "during normal runtime". I'm not sure I understand the second part of your question. Who's to say it can't happen? No one. It can happen.
Question 2: Are there ways to avoid getting this exception while still maintaining the "desired" behaviour in this case? I mean fire a non-stopping stream of callbacks.
I'm afraid I also don't know what you mean by "fire a non-stopping stream of callbacks".
One obvious workaround is to enable recursion on the ReaderWriterLockSlim object by passing LockRecursionPolicy.SupportsRecursion to its constructor. Alternatively, you could check the IsReadLockHeld property before trying to take the lock.
It is not clear from your code example why you have the lock at all, never mind why it's used in that specific way. It's possible the right solution is to not hold the lock at all while you call BeginReceive(). Use it only while processing the result from EndReceive().

Multithreaded NamePipeServer in C#

Hi
I want to use NamedPipeServerStream which is new from .NET 3.5 for namedpipe communication.
I want to write multi-threaded pipe server. is it handled by default or I should write code for that. my pipe server should handle multiple request at a time
any solution or code ?

You can write a multi threaded pipe server by repeatedly creating a NamedPipeServerStream and waiting for one connection, then spawning a thread for that instance of NamedPipeServerStream.
You can only have 254 concurrent clients though according to the .NET MSDN documentation linked below. For Win32 APIs though you can pass a special value to get unlimited based on system resources. It seems the MSDN documentation is wrong as noted below.
The below code is not tested so please do not simply copy and paste for production use without testing:
public class PipeServer
{
bool running;
Thread runningThread;
EventWaitHandle terminateHandle = new EventWaitHandle(false, EventResetMode.AutoReset);
public string PipeName { get; set; }
void ServerLoop()
{
while (running)
{
ProcessNextClient();
}
terminateHandle.Set();
}
public void Run()
{
running = true;
runningThread = new Thread(ServerLoop);
runningThread.Start();
}
public void Stop()
{
running = false;
terminateHandle.WaitOne();
}
public virtual string ProcessRequest(string message)
{
return "";
}
public void ProcessClientThread(object o)
{
NamedPipeServerStream pipeStream = (NamedPipeServerStream)o;
//TODO FOR YOU: Write code for handling pipe client here
pipeStream.Close();
pipeStream.Dispose();
}
public void ProcessNextClient()
{
try
{
NamedPipeServerStream pipeStream = new NamedPipeServerStream(PipeName, PipeDirection.InOut, 254);
pipeStream.WaitForConnection();
//Spawn a new thread for each request and continue waiting
Thread t = new Thread(ProcessClientThread);
t.Start(pipeStream);
}
catch (Exception e)
{//If there are no more avail connections (254 is in use already) then just keep looping until one is avail
}
}

Each NamedPipeServerStream instance is a Stream implementation wrapping a handle to an instance of a named pipe. You can (and a multithreaded pipe server will) have multiple instances of NamedPipeServerStream for the same named pipe: each one wraps a handle to a different instance of the named pipe, servicing a different client. Named pipe instances (even for the same pipe) are kept separate by the operating system, so there is no need for any explicit coding to keep each client's communication with the server separate.
What you do need to code explicitly is the threading model for the server. The simplest approach to multithreading the server is explained in this SO answer, which includes a pseudo-code template. More scalable implementations, if large numbers of concurrent callers need to be supported, would use thread pooling and the asynchronous methods instead of creating a dedicated thread for each connection.

NamedPipeServerStream is a point to point connection. You have to synchronise the calls yourself - for example calls written to a queue and your server reads from the synchronised queue and makes the calls.

C# network programming and resource usage

I've been doing a lot of research on how best to write "correct" network code in C#.
I've seen a number of examples using the "using" statement of C#, and I think this is a good approach, however i've seen inconsistent use of it with various expressions.
For instance, suppose I have some code like this:
TcpClient tcpClient = new TcpClient("url.com", 80);
NetworkStream tcpStream = tcpClient.GetStream();
StreamReader tcpReader = new StreamReader(tcpStream);
StreamWriter tcpWriter = new StreamWriter(tcpStream);
Obviously, this code is going to be very flaky. So, i've seen some code that puts the using on the tcpClient, which seems good. However, doesn't NetworkStream also have resources that need cleaning up? What about StreamReader/Writer?
Do I need to wrap all 4 statements in nested using statements?
And if so, what happens when the time has come to dispose? Won't StreamWriter close the stream and consequently the socket? Then what happens when StreamReader, then NetworkStream, then TcpClient each go through their disposals?
Which brings up another question. WIth both StreamReader and StreamWriter composed of the same stream, who owns it? Don't they both think they own it, and will thus both try to destroy it? Or does the framework know that the stream has already been destroyed and just silently ignore it?
It almost seems like the using statement is only necessary for the last statement in the chain, but then what happens if an exception is thrown in GetStream()? I don't think it would properly clean up the socket then, so it seems redundant usings are necessary to ensure this doesn't happen.
Does anyone know of any good, recent books on network programming with .net, and prefeably c# that include chapters on exception handling and resource management? Or maybe any good articles online? All the books I can find are from the .NET 1.1 era (Network Programming for the Microsoft .NET Framework, Network Programming in.NET, etc..), so this seems like a topic that needs some good resources.
EDIT:
Please, don't let Marc's very good comment stop anyone else from commenting on this :)
I'd like to hear anyone elses book recommendations or opinions on resource management, especially in regard to asynchronous usage.

Generally, objects should internally handle multiple Dispose() calls, and only do the main code once; so a stream getting Dispose()d multiple times is not usually a problem. Personally, I would use lots of using there; note that you don't need to indent/nest, though (unless different levels have different life-times):
using(TcpClient tcpClient = new TcpClient("url.com", 80))
using(NetworkStream tcpStream = tcpClient.GetStream())
using(StreamReader tcpReader = new StreamReader(tcpStream))
using(StreamWriter tcpWriter = new StreamWriter(tcpStream))
{
...
}
As you say, this ensures that if an error happens during initialization, everything is still cleaned up correctly. This also ensures that every level gets a chance (in the right order) to deal correctly with any buffered data, etc.
Re ownership; NetworkStream is actually an oddity in the first place... most streams are either input xor output. NetworkStream bends a few rules and shims two directions into one API; so this is an exception... normally the ownership would be clearer. Additionally, many wrappers have a flag to determine whether they should close the wrapped stream. StreamReader doesn't, but some do (such as GZipStream, which has a leaveOpen ctor option). If you don't want to flow ownership, this is an option - or use a non-closing stream intermediary - one is here (NonClosingStream or similar).
Re books; I picked up a copy of "TCP/IP Sockets in C#: Practical Guide for Programmers" (here) - adequate, but not great.

If an object supports IDisposable, it's best to put it in a using {} block because the dispose method gets called automatically for you. This also makes for less code on your part. It is important to note the using a 'using' doesn't handle any exceptions. YOu still have to do that if you want to handle any errors. Once the using block goes out of scope, so does your object.
Old Style Code
object obj;
try
{
obj= new object();
//Do something with the object
}
catch
{
//Handle Exception
}
finally
{
if (obj != null)
{
obj.Dispose();
}
}
Newer Style Code
try
{
using (object obj = new object())
{
//Do something with the object
}
catch
{
//Handle Exception
}

What about sockets?
Is it OK to do:
serverSocket = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);
serverSocket.Connect(serverEndPoint, m_NegotiationPort);
.
.
.
serverSocket.Close();
or better
using (Socket serverSocket = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp)
{
.
.
.
}