I have a two-way named pipe. I'm not sure how to shut it down gracefully, though, once I'm done with it - if I close the connection from the client side, the server side throws an exception when it tries to dispose of the StreamReader and StreamWriter I'm using. I'm currently catching it, but that seems like a kludge job to me.
Server side code:
Thread pipeServer = new Thread(ServerThread);
pipeServer.Start();
private void ServerThread(object data)
{
int threadId = Thread.CurrentThread.ManagedThreadId;
log.Debug("Spawned thread " + threadId);
PipeSecurity ps = new PipeSecurity();
SecurityIdentifier sid = new SecurityIdentifier(WellKnownSidType.WorldSid, null);
ps.AddAccessRule(new PipeAccessRule(sid, PipeAccessRights.ReadWrite, System.Security.AccessControl.AccessControlType.Allow));
ps.AddAccessRule(new PipeAccessRule(WindowsIdentity.GetCurrent().Owner, PipeAccessRights.FullControl, System.Security.AccessControl.AccessControlType.Allow));
log.Debug("Pipe security settings set [Thread " + threadId + "]");
NamedPipeServerStream pipeServer =
new NamedPipeServerStream("RDPCommunicationPipe", PipeDirection.InOut, numThreads, PipeTransmissionMode.Message, PipeOptions.None, 0x1000, 0x1000, ps);
log.Debug("Pipe Servers created");
// Wait for a client to connect
log.Info("Pipe created on thread " + threadId + ". Listening for client connection.");
pipeServer.WaitForConnection();
log.Debug("Pipe server connection established [Thread " + threadId + "]");
Thread nextServer = new Thread(ServerThread);
nextServer.Start();
try
{
// Read the request from the client. Once the client has
// written to the pipe its security token will be available.
using (StreamReader sr = new StreamReader(pipeServer))
{
using (StreamWriter sw = new StreamWriter(pipeServer) { AutoFlush = true })
{
// Verify our identity to the connected client using a
// string that the client anticipates.
sw.WriteLine("I am the one true server!");
log.Debug("[Thread " + threadId + "]" + sr.ReadLine());
log.Info(string.Format("Client connected on thread {0}. Client ID: {1}", threadId, pipeServer.GetImpersonationUserName()));
while (!sr.EndOfStream)
{
log.Debug("[Thread " + threadId + "]" + sr.ReadLine());
}
}
}
}
// Catch the IOException that is raised if the pipe is broken
// or disconnected.
catch (IOException e)
{
log.Error("ERROR: " + e);
}
}
Client side code:
class Program
{
static void Main(string[] args)
{
Console.WriteLine("Starting...");
var client = new NamedPipeClientStream(".", "RDPCommunicationPipe", PipeDirection.InOut);
client.Connect();
Console.WriteLine("Pipe connected successfully");
using (StreamReader sr = new StreamReader(client))
{
using (StreamWriter sw = new StreamWriter(client) { AutoFlush = true })
{
string temp;
do
{
temp = sr.ReadLine();
Console.WriteLine(temp);
} while (temp.Trim() != "I am the one true server!");
sw.WriteLine("Message received and understood");
while (!string.IsNullOrEmpty(temp = Console.ReadLine()))
{
sw.WriteLine(temp);
}
}
}
client.Close();
}
}
It works perfectly until I hit enter on an empty line in the client app, which terminates it, closing the client. The server app then throws a System.IO.IOException: Pipe is broken. when it hits the end of the StreamWriter using block. How do I properly dispose of my stream handlers?
(Code based on ideas found here and here.)
I'm currently catching it, but that seems like a kludge job to me.
IMHO, it's about as good as you're going to get, if you want to be a good neighbor and dispose your owned StreamWriter object and still invest a minimum of effort.
That said, it seems to me that in this particular situation, it'd also be fine to comment out the call to Dispose() — or in your case, not use the using statement — and include another comment explaining that at that point in the sequence of execution of your code, you know that all that call is going to do is throw an exception, and so there's no point in making it.
Of course, if you just don't bother disposing the StreamWriter, then you'll want to explicitly dispose your pipe stream. You might also want to use the StreamWriter constructor that has the leaveOpen parameter, and pass true for that parameter, as a way of documenting your intent to not have the StreamWriter own the pipe stream object.
Either way, you're going to wind up leaving object in the finalizer queue, because the exception bypasses the call to GC.SuppressFinalize(), as does (of course) not bothering to call Dispose() at all. As long as you aren't dealing with a high-volume scenario (i.e. lots of these objects), that's probably fine. But it's certainly not ideal.
Unfortunately, named pipes themselves don't have semantics that provide for the kind of "graceful closure" that sockets do. That is, the only way for an endpoint to indicate they are done writing is to disconnect (for a server pipe) or close (for server or client pipes). Neither option leaves the pipe available for reading, so implementing a graceful closure on a pipe requires handshaking within the application protocol itself, rather than relying on the I/O object .
In addition to this inconvenience (which I admit, isn't really directly related to your question), the implementation of PipeStream.Flush() checks to see whether the pipe is writeable. Even though it has no intention of writing anything! It's that last part I find really annoying, and of course directly causes the issue you're asking about. It seems unreasonable to me for code in the .NET Framework to go out of its way to throw exceptions in scenarios where those exceptions cause more trouble than good.
All that said, you do have some other options:
Subclass the NamedPipeServerStream and NamedPipeClientStream types, and override the Flush() method so that it really does do nothing. Or rather, it would be nice if you could do this. But those types are sealed, so you can't.
Alternative to subclassing those types, you can wrap them in your own Stream implementation. This is a lot more of a hassle, especially since you are likely going to want to override all of the async members, at least if you intend to use these objects in any situation where I/O performance is of any interest.
Use separate one-directional pipes for reading and writing. In this implementation, you can close the StreamWriter itself as a way of closing the connection, which results in the correct order of things (i.e. the flush happens before the close on the pipe). This also addresses the graceful closure issue, because with two pipes for each connection, you can have the same basic "half-closed" semantics that sockets have. Of course, this option is significantly complicated by the challenge in identifying which pairs of pipe connections go with each other.
Both of these (that is, the second and third ones, i.e. the ones that are actually possible) have some obvious drawbacks. Having to have your own Stream class is a pain, due to all the duplication of code required. And doubling the pipe object count seems like a drastic way to address the exception (but it could be an acceptable and desirable implementation to support the graceful closure semantics, with the happy side-effect of eliminating the thrown-exception issue with StreamWriter.Dispose()).
Note that in a high-volume scenario (but then, why are you using pipes? 😊 ), throwing and catching exceptions with high frequency could be a problem (they are expensive). So one or the other of these two alternative options would probably be preferable in that case, over either catching the exception and just not bothering to close/dispose your StreamWriter (both of which add inefficiencies that would interfere with a high-volume scenario).
Related
Need: long-running program with TCP connections
A C# 4.0 (VS1010, XP) program needs to connect to a host using TCP, send and receive bytes, sometimes close the connection properly and reopen it later. Surrounding code is written using Rx.Net Observable style. The volume of data is low but the program should runs continuously (avoid memory leak by taking care of properly disposing resources).
The text below is long because I explain what I searched and found. It now appears to work.
Overall questions are: since Rx is sometime unintuitive, are the solutions good? Will that be reliable (say, may it run for years without trouble)?
Solution so far
Send
The program obtains a NetworkStream like this:
TcpClient tcpClient = new TcpClient();
LingerOption lingerOption = new LingerOption(false, 0); // Make sure that on call to Close(), connection is closed immediately even if some data is pending.
tcpClient.LingerState = lingerOption;
tcpClient.Connect(remoteHostPort);
return tcpClient.GetStream();
Asynchronous sending is easy enough. Rx.Net allows to handle this with much shorter and cleaner code than traditional solutions. I created a dedicated thread with an EventLoopScheduler. The operations needing a send are expressed using IObservable. Using ObserveOn(sendRecvThreadScheduler) guarantee that all send operations are done on that thread.
sendRecvThreadScheduler = new EventLoopScheduler(
ts =>
{
var thread = new System.Threading.Thread(ts) { Name = "my send+receive thread", IsBackground = true };
return thread;
});
// Loop code for sending not shown (too long and off-topic).
So far this is excellent and flawless.
Receive
It seems that to receive data, Rx.Net should also allow shorter and cleaner code that traditional solutions.
After reading several resources (e.g. http://www.introtorx.com/ ) and stackoverflow, it seems that a very simple solution is to bridge the Asynchronous Programming to Rx.Net like in https://stackoverflow.com/a/14464068/1429390 :
public static class Ext
{
public static IObservable<byte[]> ReadObservable(this Stream stream, int bufferSize)
{
// to hold read data
var buffer = new byte[bufferSize];
// Step 1: async signature => observable factory
var asyncRead = Observable.FromAsyncPattern<byte[], int, int, int>(
stream.BeginRead,
stream.EndRead);
return Observable.While(
// while there is data to be read
() => stream.CanRead,
// iteratively invoke the observable factory, which will
// "recreate" it such that it will start from the current
// stream position - hence "0" for offset
Observable.Defer(() => asyncRead(buffer, 0, bufferSize))
.Select(readBytes => buffer.Take(readBytes).ToArray()));
}
}
It mostly works. I can send and receive bytes.
Close time
This is when things start to go wrong.
Sometimes I need to close the stream and keep things clean. Basically this means: stop reading, end the byte-receiving observable, open a new connection with a new one.
For one thing, when connection is forcibly closed by remote host, BeginRead()/EndRead() immediately loop consuming all CPU returning zero bytes. I let higher level code notice this (with a Subscribe() to the ReadObservable in a context where high-level elements are available) and cleanup (including closing and disposing of the stream). This works well, too, and I take care of disposing of the object returned by Subscribe().
someobject.readOneStreamObservableSubscription = myobject.readOneStreamObservable.Subscribe(buf =>
{
if (buf.Length == 0)
{
MyLoggerLog("Read explicitly returned zero bytes. Closing stream.");
this.pscDestroyIfAny();
}
});
Sometimes, I just need to close the stream. But apparently this must cause exceptions to be thrown in the asynchronous read. c# - Proper way to prematurely abort BeginRead and BeginWrite? - Stack Overflow
I added a CancellationToken that causes Observable.While() to end the sequence. This does not help much to avoid these exceptions since BeginRead() can sleep for a long time.
Unhandled exception in the observable caused the program to exit. Searching provided .net - Continue using subscription after exception - Stack Overflow which suggested to add a Catch that resumes the broken Observable with an empty one, effectively.
Code looks like this:
public static IObservable<byte[]> ReadObservable(this Stream stream, int bufferSize, CancellationToken token)
{
// to hold read data
var buffer = new byte[bufferSize];
// Step 1: async signature => observable factory
var asyncRead = Observable.FromAsyncPattern<byte[], int, int, int>(
stream.BeginRead,
stream.EndRead);
return Observable.While(
// while there is data to be read
() =>
{
return (!token.IsCancellationRequested) && stream.CanRead;
},
// iteratively invoke the observable factory, which will
// "recreate" it such that it will start from the current
// stream position - hence "0" for offset
Observable.Defer(() =>
{
if ((!token.IsCancellationRequested) && stream.CanRead)
{
return asyncRead(buffer, 0, bufferSize);
}
else
{
return Observable.Empty<int>();
}
})
.Catch(Observable.Empty<int>()) // When BeginRead() or EndRead() causes an exception, don't choke but just end the Observable.
.Select(readBytes => buffer.Take(readBytes).ToArray()));
}
What now? Question
This appears to work well. Conditions where remote host forcibly closed the connection or is just no longer reachable are detected, causing higher level code to close the connection and retry. So far so good.
I'm unsure if things feel quite right.
For one thing, that line:
.Catch(Observable.Empty<int>()) // When BeginRead() or EndRead() causes an exception, don't choke but just end the Observable.
feels like the bad practice of empty catch block in imperative code. Actual code does log the exception, and higher level code detect the absence of reply and correctly handle, so it should be considered fairly okay (see below)?
.Catch((Func<Exception, IObservable<int>>)(ex =>
{
MyLoggerLogException("On asynchronous read from network.", ex);
return Observable.Empty<int>();
})) // When BeginRead() or EndRead() causes an exception, don't choke but just end the Observable.
Also, this is indeed shorter than most traditional solutions.
Are the solutions correct or did I miss some simpler/cleaner ways?
Are there some dreadful problems that would look obvious to wizards of Reactive Extensions?
Thank you for your attention.
I'm building an application that uses Named Pipes for IPC.
When started writing a stress test, I found an issue related to when a client does quickly connect-disconnect.
Server code:
static void ServerThread()
{
var serverPipe = new NamedPipeServerStream("myipc", PipeDirection.InOut, -1, PipeTransmissionMode.Message, PipeOptions.Asynchronous | PipeOptions.WriteThrough);
serverPipe.BeginWaitForConnection(
ar =>
{
var thisPipe = (NamedPipeServerStream)ar.AsyncState;
thisPipe.EndWaitForConnection(ar);
Task.Factory.StartNew(ServerThread);
thisPipe.Dispose();
},
serverPipe);
}
Client does nothing but connect-disconnect as follow:
static void RunClients()
{
for (int i = 0; i < 100; i++)
{
var clientPipe = new NamedPipeClientStream(".", "myipc", PipeDirection.InOut, PipeOptions.Asynchronous | PipeOptions.WriteThrough);
clientPipe.Connect(1000);
clientPipe.Dispose();
}
}
When this runs, one of the clients is failing in Connect() while the server fails in BeginWaitForConnection - saying Pipe is being closed.
If I add at least Thread.Sleep(100) before each client disposes - everything works just fine.
I'm sure what I'm doing is a corner case, but I believe the pipes should be able to handle this in greaceful way.
Any ideas on what could be wrong?
Thanks!
one of the clients is failing in Connect()
Because the server immediately disposes the pipe after connecting.
the server fails in BeginWaitForConnection
Because the client immediately disposes the pipe after connecting.
I believe the pipes should be able to handle this in greaceful way.
It does, it gracefully throws an exception to let your code know that something exceptional happened. You seem to assume it is normal that code closes a pipe without doing anything to let the other end know that the pipe is about to disappear. That is not normal, it is exceptional. So you get an exceptional notification for it.
You catch exceptions with try/catch. There are two things you can do in your catch handler. You can assume that it is okay for code to close a pipe willy-nilly, in which case you do nothing beyond closing your end of the pipe and get out. Or you can assume that something Really Bad happened because the other end of the pipe didn't say goodbye nicely. Which is rather important to distinguish the oh-crap kind of mishaps, like a pipe client or the server crashing. It is up to you to choose your preferred way, but I strongly recommend to not ignore the oh-crap scenarios, it does and will happen. You just created a good simulation of such a mishap.
I am having stability issues using a named pipe for communication between a C# and Java app.
Here is the code that sets up the named pipe in C# and reads lines of XML strings.
try
{
NamedPipeServerStream inStream = new NamedPipeServerStream(inName, PipeDirection.In);
inStream.WaitForConnection();
reader = new StreamReader(inStream);
while (!Stopped && !reader.EndOfStream)
{
string xml = reader.ReadLine();
processXml(xml);
}
}
catch (Exception e)
{
log.Error("Error in receiver", e);
}
finally
{
log.Info("Receiver ended");
}
And here is the connection and write code in Java
public void connect() throws TransportUnavailableException
{
try
{
File inPipe = new File(inName);
os = new FileOutputStream(inPipe);
// Uses JAXB for XML serialization
marshaller = context.createMarshaller();
}
catch (FileNotFoundException e)
{
throw new TransportUnavailableException("Named pipe not found: " + inName);
}
}
public void send(Message message)
{
marshaller.marshal(message, os);
os.write('\n');
os.flush();
}
Everything works fine normally. But many users are reporting crashes. I don't see any exceptions in logs that suggest a reason for the pipe dying. I just see that the receiving thread in C# ends (i.e. 'Receiver ended' in the logs) and after this I get an IO exception on the next attempted send from Java with a message 'The handle is invalid'. This seems to happen randomly, but usually within the 1st minute or 2 after the connection was established. The pipe ending message also happens when the application is not doing anything, it could have been minutes since the last user operation. Then it could be a few more minutes before the next write is attempted from Java.
All reasons for my app to bring down the pipe on purpose (e.g. a crash elsewhere in system) are logged and I never see that as a reason for the pipe ended, I just get the message that the reader has given up reading.
Could there be any external reason for the pipe being killed, anti-virus, firewall etc?
I noticed I didn't use a RandomAccessFile from Java like most examples seem to use. Could this be a reason?
Any help/suggestion appreciated
Thanks!
Your server side code only processes one connection, then it exits when it reads to EOS. You need to create the named pipe, loop accepting connections, and spin up a new thread to handle each connection. You also need to close each connection when you're finished with it.
However I would use TCP rather than named pipes for this, for several reasons.
I'm trying to create a chat with file transfer application using TCPSocket and here is my code..
SENDER:
public void sendData(string message)
{
StreamWriter streamWriter = new StreamWriter(netStream); // netStream is
// connected
streamWriter.WriteLine(message);
streamWriter.WriteLine(message);
logs.Add(string.Format("Message Sent! :{0}", message));
//netStream.Flush();
streamWriter.Flush();
}
RECEIVER:
private void ReceiveData()
{
StreamReader streamReader = new StreamReader(ChatNetStream);
StringBuilder dataAppends = new StringBuilder();
bool doneTransfer = false;
string data;
while (!doneTransfer)
{
while ((data = streamReader.ReadLine()) != null)
{
dataAppends.Append(data);
}
doneTransfer = true;
//ChatNetStream.Close();
//streamReader
}
//do whatever i want with dataAppends.ToString() here..
ReceiveData()
}
the problem is i always turn into infinite loop inside this statement
while ((data = streamReader.ReadLine()) != null)
{
dataAppends.Append(data);
}
even if i put streamWriter.Flush() on my sender..
do i need to close/dispose the netStream/NetworkStream?
anyway, can i use only 1 socket or connection to send a File and send a chat at the same time..? or do i need to use a new socket connection everytime i send a file..
You get an infinite loop because StreamReader.ReadLine will only return null when the end of the stream is reached. For a network stream, "end of stream" means "the other side has closed its half of the connection". Since the other side is your client, and it keeps the connection open while waiting for the user to type in more data, you will end up with an infinite loop.
What you want to do instead is fire off an operation that only completes if there is more data to read. There are two ways to go about this: either use a blocking read operation (on a dedicated thread, so that you don't block your application's other processing while waiting for messages), or use an async (event- or callback-based) approach.
For the synchronous (blocking) approach, see the documentation on NetworkStream.Read which includes example code that shows how to check if there is incoming data and how you can read it. The one point you absolutely need to know here is that when Read returns zero, it means that all data has been read and the connection has been closed from the other side (so you should close your end as well and not loop; the client has disconnected).
For low-level async network reads, the relevant operation is NetworkStream.BeginRead, which comes with its own example.
Both approaches are lower-level than what you currently have and will require you to manually assemble data inside a buffer and decide when "enough data" (i.e. a full line) has accumulated for you to process. You will then have to carefully pull that data out of the buffer and continue.
For a higher-level approach that still allows you some degree of orchestrating things, look into using client sockets (and in particular the two sync and async options there). This functionality is introduced by the TcpClient (and server-side the corresponding TcpListener) classes.
Finally, as jValdron's comment says, you will either need a separate connection for transferring file data or engineer some custom protocol that allows you to interleave multiple kinds of data over the same network stream. The second solution is has generally more technical merit, but it will also be harder for you to implement correctly.
Checkout the BasicSend example in networkComms.net which demonstrates a simple chat application using an open source library.
I've been doing a lot of research on how best to write "correct" network code in C#.
I've seen a number of examples using the "using" statement of C#, and I think this is a good approach, however i've seen inconsistent use of it with various expressions.
For instance, suppose I have some code like this:
TcpClient tcpClient = new TcpClient("url.com", 80);
NetworkStream tcpStream = tcpClient.GetStream();
StreamReader tcpReader = new StreamReader(tcpStream);
StreamWriter tcpWriter = new StreamWriter(tcpStream);
Obviously, this code is going to be very flaky. So, i've seen some code that puts the using on the tcpClient, which seems good. However, doesn't NetworkStream also have resources that need cleaning up? What about StreamReader/Writer?
Do I need to wrap all 4 statements in nested using statements?
And if so, what happens when the time has come to dispose? Won't StreamWriter close the stream and consequently the socket? Then what happens when StreamReader, then NetworkStream, then TcpClient each go through their disposals?
Which brings up another question. WIth both StreamReader and StreamWriter composed of the same stream, who owns it? Don't they both think they own it, and will thus both try to destroy it? Or does the framework know that the stream has already been destroyed and just silently ignore it?
It almost seems like the using statement is only necessary for the last statement in the chain, but then what happens if an exception is thrown in GetStream()? I don't think it would properly clean up the socket then, so it seems redundant usings are necessary to ensure this doesn't happen.
Does anyone know of any good, recent books on network programming with .net, and prefeably c# that include chapters on exception handling and resource management? Or maybe any good articles online? All the books I can find are from the .NET 1.1 era (Network Programming for the Microsoft .NET Framework, Network Programming in.NET, etc..), so this seems like a topic that needs some good resources.
EDIT:
Please, don't let Marc's very good comment stop anyone else from commenting on this :)
I'd like to hear anyone elses book recommendations or opinions on resource management, especially in regard to asynchronous usage.
Generally, objects should internally handle multiple Dispose() calls, and only do the main code once; so a stream getting Dispose()d multiple times is not usually a problem. Personally, I would use lots of using there; note that you don't need to indent/nest, though (unless different levels have different life-times):
using(TcpClient tcpClient = new TcpClient("url.com", 80))
using(NetworkStream tcpStream = tcpClient.GetStream())
using(StreamReader tcpReader = new StreamReader(tcpStream))
using(StreamWriter tcpWriter = new StreamWriter(tcpStream))
{
...
}
As you say, this ensures that if an error happens during initialization, everything is still cleaned up correctly. This also ensures that every level gets a chance (in the right order) to deal correctly with any buffered data, etc.
Re ownership; NetworkStream is actually an oddity in the first place... most streams are either input xor output. NetworkStream bends a few rules and shims two directions into one API; so this is an exception... normally the ownership would be clearer. Additionally, many wrappers have a flag to determine whether they should close the wrapped stream. StreamReader doesn't, but some do (such as GZipStream, which has a leaveOpen ctor option). If you don't want to flow ownership, this is an option - or use a non-closing stream intermediary - one is here (NonClosingStream or similar).
Re books; I picked up a copy of "TCP/IP Sockets in C#: Practical Guide for Programmers" (here) - adequate, but not great.
If an object supports IDisposable, it's best to put it in a using {} block because the dispose method gets called automatically for you. This also makes for less code on your part. It is important to note the using a 'using' doesn't handle any exceptions. YOu still have to do that if you want to handle any errors. Once the using block goes out of scope, so does your object.
Old Style Code
object obj;
try
{
obj= new object();
//Do something with the object
}
catch
{
//Handle Exception
}
finally
{
if (obj != null)
{
obj.Dispose();
}
}
Newer Style Code
try
{
using (object obj = new object())
{
//Do something with the object
}
catch
{
//Handle Exception
}
What about sockets?
Is it OK to do:
serverSocket = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);
serverSocket.Connect(serverEndPoint, m_NegotiationPort);
.
.
.
serverSocket.Close();
or better
using (Socket serverSocket = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp)
{
.
.
.
}