Before the introduction of async-await programming into C#, how was one able to put a network request into another thread and yield execution time back to the CPU until a response is received so that this thread will not waste CPU time?
Because when CPU allocates time to this thread and thread sits idle waiting for a response, that would be a waste of CPU time, right?
In several ways, however Asynchronous Programming Model (APM) was the go-to for this type of Asynchrony
An asynchronous operation that uses the IAsyncResult design pattern is
implemented as two methods named BeginOperationName and
EndOperationName that begin and end the Asynchronous Operation
OperationName respectively. For example, the FileStream class provides
the BeginRead and EndRead methods to Asynchronously read bytes from a
file. These methods implement the asynchronous version of the Read
method.
To answer your question
Because when CPU allocates time to this thread and thread sits idle
waiting for a response, that would be a waste of CPU time, right?
No blocking a thread and waiting for a completion port to call back doesn't cause CPU cycles to run away, however polling on a thread will.
There is a lot to how this works, however an example use can be seen here
Example of usage
private static void TestWrite()
{
// Must specify FileOptions.Asynchronous otherwise the BeginXxx/EndXxx methods are
// handled synchronously.
FileStream fs = new FileStream(Program.FilePath, FileMode.OpenOrCreate,
FileAccess.Write, FileShare.None, 8, FileOptions.Asynchronous);
string content = "A quick brown fox jumps over the lazy dog";
byte[] data = Encoding.Unicode.GetBytes(content);
// Begins to write content to the file stream.
Console.WriteLine("Begin to write");
fs.BeginWrite(data, 0, data.Length, Program.OnWriteCompleted, fs);
Console.WriteLine("Write queued");
}
private static void OnWriteCompleted(IAsyncResult asyncResult)
{
// End the async operation.
FileStream fs = (FileStream)asyncResult.AsyncState;
fs.EndWrite(asyncResult);
// Close the file stream.
fs.Close();
Console.WriteLine("Write completed");
// Test async read bytes from the file stream.
Program.TestRead();
}
Related
I know something about the IOCP, but I'm a little confused with APM.
static FileStream fs;
static void Main(string[] args)
{
fs = new FileStream(#"c:\bigfile.txt", FileMode.Open);
var buffer = new byte[10000000];
IAsyncResult asyncResult = fs.BeginRead(buffer, 0, 10000000, OnCompletedRead, null);
Console.WriteLine("async...");
int bytesRead = fs.EndRead(asyncResult);
Console.WriteLine("async... over");
}
static void OnCompletedRead(IAsyncResult ar)
{
Console.WriteLine("finished");
}
I wonder, is the read action executed by an IO thread asynchronously? Or a worker thread in a thread pool?
And the callback function OnCompletedRead, is it also executed by an IO thread in CLR thread pool?
Are these two threads the same one? If not, there are two threads generated, one executes the read action and another does the callback.
If you don't use an AsyncCallback argument with BeginRead then there is only one thread that runs code in your program. This uses IO completion ports to signal when the IO is complete by running a small amount of code on a thread in the IO thread pool to update the status of the operation as being complete. When you call EndRead it will block the current thread until the IO operation is complete. It is asynchronous in that when you start the read operation the current thread does not need to do anything other than wait for the IO hardware to perform the read operation, so you can do other things in the meantime and then decide when you want to stop and wait for the IO to finish.
If you do pass in an AsyncCallback then when the IO operation is complete it will execute a small amount of code on an IO thread pool thread which will trigger your callback method to be executed on a thread from the .NET thread pool.
Usually, mclaassen is right about the nature of IO bound work, IOCP and the APM. When BeginRead executes, it does so asynchronously all the way down to kernel mode. But, there is one caveat specifically in your example that he didn't mention in his answer.
In your example, you use the FileStream class. One important thing to note is that if you dont use the FileStream overload that accepts a useAsync boolean, when you invoke a BeginWrite / EndWrite operation, it will queue work on a new ThreadPool thread.
This is the proper overload:
public FileStream(
string path,
FileMode mode,
FileAccess access,
FileShare share,
int bufferSize,
bool useAsync
)
From MSDN:
useAsync:
Type: System.Boolean
Specifies whether to use asynchronous
I/O or synchronous I/O. However, note that the underlying operating
system might not support asynchronous I/O, so when specifying true,
the handle might be opened synchronously depending on the platform.
When opened asynchronously, the BeginRead and BeginWrite methods
perform better on large reads or writes, but they might be much slower
for small reads or writes. If the application is designed to take
advantage of asynchronous I/O, set the useAsync parameter to true.
Using asynchronous I/O correctly can speed up applications by as much
as a factor of 10, but using it without redesigning the application
for asynchronous I/O can decrease performance by as much as a factor
of 10.
You have to make sure each specific method implementing the APM pattern truly uses true asynchronous work all the way down.
I'm writing a class that exposes a subsection of a stream for reading. Since data may be read from multiple different subsections at the same time, only one operation may be active at any one time.
I had the idea of locking the underlying stream before every operation. Is locking the stream around the BeginRead call sufficient to ensure that concurrent asynchronous reads from different positions in the underlying stream happen correctly?
public sealed class SubStream : Stream
{
// ...
public override IAsyncResult BeginRead(byte[] buffer, int offset, int count,
AsyncCallback callback, object state)
{
lock (this.baseStream)
{
this.baseStream.Seek(this.offset + this.position, SeekOrigin.Begin);
return this.baseStream.BeginRead(buffer, offset, count,
callback, state);
}
}
public override int EndRead(IAsyncResult asyncResult)
{
int read;
lock (this.baseStream)
{
read = baseStream.EndRead(asyncResult);
this.position += read;
}
return read;
}
// Read() and ReadByte() also lock on this.baseStream (not shown).
// ...
}
For example, if thread A calls BeginRead, a lock on the base stream is acquired. Now thread B calls BeginRead and has to wait for the lock to be released. Thread A sets the position of the base stream and starts an asynchronous read operation. Then releases the lock. Thread B then acquires the lock and changes the position of the base stream and starts another asynchronous read operation. Then, sometime later, the asynchronous read from thread A completes. Can I be sure that this reads from the original position in the base stream? If not, how do I fix it?
Here you might end up with multiple threads calling BeginRead on the same instance of resource (baseStream). As per MSDN, the "EndRead must be called exactly once for every call to BeginRead. Failing to end a read process before beginning another read can cause undesirable behavior such as deadlock." In you case, I recon a trouble 'if Thread B is on Seek (on baseStream) while Thread A in the middle of executing their EndRead(callback)'.
Due to the nature of requirement, you are better off with wrapping multi-threaded access with synchronous I/O. This means, the current implementation can be amended with synchronous I/O instead of asynchronous I/O. Also, you may want to consider informing queuing threads about the completion of former threads using Monitor.WaitOne (baseStream) and Monitor.Pulse(baseStream) or Monitor.PulseAll(baseStream).
Alternately, I would like to throw another idea of Memory-Mapped file for segmented style.
In the given code snippet you'll read multiple times from the same position. Move the position update to the BeginRead function. Apart from that you are honoring the contract of the FileStream class by never concurrently calling its methods.
Quick preface of what I'm trying to do. I want to start a process and start up two threads to monitor the stderr and stdin. Each thread chews off bits of the stream and then fires it out to a NetworkStream. If there is an error in either thread, both threads need to die immediately.
Each of these processes with stdout and stdin monitoring threads are spun off by a main server process. The reason this becomes tricky is because there can easily be 40 or 50 of these processes at any given time. Only during morning restart bursts are there ever more than 50 connections, but it really needs to be able to handle 100 or more. I test with 100 simultaneous connections.
try
{
StreamReader reader = this.myProcess.StandardOutput;
char[] buffer = new char[4096];
byte[] data;
int read;
while (reader.Peek() > -1 ) // This can block before stream is streamed to
{
read = reader.Read(buffer, 0, 4096);
data = Server.ClientEncoding.GetBytes(buffer, 0, read);
this.clientStream.Write(data, 0, data.Length); //ClientStream is a NetworkStream
}
}
catch (Exception err)
{
Utilities.ConsoleOut(string.Format("StdOut err for client {0} -- {1}", this.clientID, err));
this.ShutdownClient(true);
}
This code block is run in one Thread which is right now not Background. There is a similar thread for the StandardError stream. I am using this method instead of listening to OutputDataReceived and ErrorDataReceived because there was an issue in Mono that caused these events to not always fire properly and even though it appears to be fixed now I like that this method ensures I'm reading and writing everything sequentially.
ShutdownClient with True simply tries to kill both threads. Unfortunately the only way I have found to make this work is to use an interrupt on the stdErrThread and stdOutThread objects. Ideally peek would not block and I could just use a manual reset event to keep checking for new data on stdOut or stdIn and then just die when the event is flipped.
I doubt this is the best way to do it. Is there a way to execute this without using an Interrupt?
I'd like to change, because I just saw in my logs that I missed a ThreadInterruptException thrown inside Utlities.ConsoleOut. This just does a System.Console.Write if a static variable is true, but I guess this blocks somewhere.
Edits:
These threads are part of a parent Thread that is launched en masse by a server upon a request. Therefore I cannot set the StdOut and StdErr threads to background and kill the application. I could kill the parent thread from the main server, but this again would get sticky with Peek blocking.
Added info about this being a server.
Also I'm starting to realize a better Queuing method for queries might be the ultimate solution.
I can tell this whole mess stems from the fact that Peek blocks. You're really trying to fix something that is fundamentally broken in the framework and that is never easy (i.e. not a dirty hack). Personally, I would fix the root of the problem, which is the blocking Peek. Mono would've followed Microsoft's implementation and thus ends up with the same problem.
While I know exactly how to fix the problem should I be allowed to change the framework source code, the workaround is lengthy and time consuming.
But here goes.
Essentially, what Microsoft needs to do is change Process.StartWithCreateProcess such that standardOutput and standardError are both assigned a specialised type of StreamReader (e.g. PipeStreamReader).
In this PipeStreamReader, they need to override both ReadBuffer overloads (i.e. need to change both overloads to virtual in StreamReader first) such that prior to a read, PeekNamedPipe is called to do the actual peek. As it is at the moment, FileStream.Read() (called by Peek()) will block on pipe reads when no data is available for read. While a FileStream.Read() with 0 bytes works well on files, it doesn't work all that well on pipes. In fact, the .NET team missed an important part of the pipe documentation - PeekNamedPipe WinAPI.
The PeekNamedPipe function is similar to the ReadFile function with the following exceptions:
...
The function always returns immediately in a single-threaded application, even if there is no data in the pipe. The wait mode of a named pipe handle (blocking or nonblocking) has no effect on the function.
The best thing at this moment without this issue solved in the framework would be to roll out your own Process class (a thin wrapper around WinAPI would suffice).
Why dont you just set both Threads to be backround and then kill the app? It would cause an immediate closing of both threads.
You're building a server. You want to avoid blocking. The obvious solution is to use the asynchronous APIs:
var myProcess = Process.GetCurrentProcess();
StreamReader reader = myProcess.StandardOutput;
char[] buffer = new char[4096];
byte[] data;
int read;
while (!myProcess.HasExited)
{
read = await reader.ReadAsync(buffer, 0, 4096);
data = Server.ClientEncoding.GetBytes(buffer, 0, read);
await this.clientStream.WriteAsync(data, 0, data.Length);
}
No need to waste threads doing I/O work :)
Get rid of peek and use the method below to read from the process output streams. ReadLine() returns null when the process ends. To join this thread with your calling thread either wait for the process to end or kill the process yourself. ShutdownClient() should just Kill() the process which will cause the other thread reading the StdOut or StdErr to also exit.
private void ReadToEnd()
{
string nextLine;
while ((nextLine = stream.ReadLine()) != null)
{
output.WriteLine(nextLine);
}
}
I need to read from NetworkStream which would send data randomly and the size of data packets also keep varying. I am implementing a multi-threaded application where each thread would have its own stream to read from. If there is no data on the stream, the application should keep waiting for the data to arrive. However, if the server is done sending data and has terminated the session, then it should exit out.
Initially I had utilised the Read method to obtain the data from the stream, but it used to block the thread and kept waiting until data appeared on the stream.
The documentation on MSDN suggests,
If no data is available for reading,
the Read method returns 0. If the
remote host shuts down the connection,
and all available data has been
received, the Read method completes
immediately and return zero bytes.
But in my case, I have never got the Read method to return 0 and exit gracefully. It just waits indefinitely.
In my further investigation, I came across BeginRead which watches the stream and invokes a callback method asynchronously, as soon as it receives the data. I have tried to look for various implementations using this approach as well, however, I was unable to identify when would using BeginRead be beneficial as opposed to Read.
As I look at it, BeginRead has just the advantage of having the async call, which would not block the current thread. But in my application, I already have a separate thread to read and process the data from stream, so that wouldn't make much difference for me.
Can anyone please help me understand the Wait and Exit mechanism for
BeginRead and how is it different from Read?
What would be the best way to implement the desired functionality?
I use BeginRead, but continue blocking the thread using a WaitHandle:
byte[] readBuffer = new byte[32];
var asyncReader = stream.BeginRead(readBuffer, 0, readBuffer.Length,
null, null);
WaitHandle handle = asyncReader.AsyncWaitHandle;
// Give the reader 2seconds to respond with a value
bool completed = handle.WaitOne(2000, false);
if (completed)
{
int bytesRead = stream.EndRead(asyncReader);
StringBuilder message = new StringBuilder();
message.Append(Encoding.ASCII.GetString(readBuffer, 0, bytesRead));
}
Basically it allows a timeout of the async reads using the WaitHandle and gives you a boolean value (completed) if the read was completed in the set time (2000 in this case).
Here's my full stream reading code copied and pasted from one of my Windows Mobile projects:
private static bool GetResponse(NetworkStream stream, out string response)
{
byte[] readBuffer = new byte[32];
var asyncReader = stream.BeginRead(readBuffer, 0, readBuffer.Length, null, null);
WaitHandle handle = asyncReader.AsyncWaitHandle;
// Give the reader 2seconds to respond with a value
bool completed = handle.WaitOne(2000, false);
if (completed)
{
int bytesRead = stream.EndRead(asyncReader);
StringBuilder message = new StringBuilder();
message.Append(Encoding.ASCII.GetString(readBuffer, 0, bytesRead));
if (bytesRead == readBuffer.Length)
{
// There's possibly more than 32 bytes to read, so get the next
// section of the response
string continuedResponse;
if (GetResponse(stream, out continuedResponse))
{
message.Append(continuedResponse);
}
}
response = message.ToString();
return true;
}
else
{
int bytesRead = stream.EndRead(asyncReader);
if (bytesRead == 0)
{
// 0 bytes were returned, so the read has finished
response = string.Empty;
return true;
}
else
{
throw new TimeoutException(
"The device failed to read in an appropriate amount of time.");
}
}
}
Async I/O can be used to achieve the same amount of I/O in less threads.
As you note, right now your app has one thread per Stream. This is OK with small numbers of connections, but what if you need to support 10000 at once? With async I/O, this is no longer necessary because the read completion callback allows context to be passed identifying the relevant stream. Your reads no longer block, so you don't need one thread per Stream.
Whether you use sync or async I/O, there is a way to detect and handle stream closedown on the relevant API return codes. BeginRead should fail with IOException if the socket has already been closed. A closedown while your async read is pending will trigger a callback, and EndRead will then tell you the state of play.
When your application calls BeginRead,
the system will wait until data is
received or an error occurs, and then
the system will use a separate thread
to execute the specified callback
method, and blocks on EndRead until
the provided NetworkStream reads data
or throws an exception.
Did you try server.ReceiveTimeout? You can set the time which Read() functon will wait for incomming data before returning zero. In your case, this property is probably set to infinite somewhere.
BeginRead is an async process which means your main thread will start execute Read in another process. So now we have 2 parallel processes. if u want to get the result, u have to call EndRead, which will gives the result.
some psudo
BeginRead()
//...do something in main object while result is fetching in another thread
var result = EndRead();
but if your main thread doesn't have anything else to do and u have to need the result, u should call Read.
I have to be able to save a file, unfortunatly it can potentially be very large so saving it can potentially take minutes. As I need to do this from a GUI thread I don't want to block the GUI from executing. I was thinking about attempting the save operation on a seperate thread to allow the primary GUI thread to continue executing.
Is there a nice (easy) way to spawn a new thread, save the file, and destroy the thread without any nasty side effects?!
It must be said that I have NEVER had to use threads before so I am a complete novice! Any and all help would be greatly appreciated!
BackgroundWorker (as suggested by Frederik) is a good choice, particularly if you want to report progress to the UI while you're saving. A search for BackgroundWorker tutorial gets a lot of hits, so you should be able to follow one of those to get you started.
One thing to be careful of: would there be any way of changing the data structure that you'll be trying to save from the UI thread? If so, you should disable those aspects of the UI while you're saving - it would (probably!) be bad to be half way through saving the data, then allow the user to change some of it. If you can get away with effectively handing off the data to the background thread and then not touching it from the UI thread, that will make your life a lot easier.
You could maybe use the BackGroundWorker component, as it will abstract a bit the Threading part for you.
Your problem might be that there are several nice and easy ways of doing it. If you just want to set off the file save and not worry about knowing when it has completed, then having a method
void SaveMyFile(object state)
{
// SaveTheFile
}
and calling it with
ThreadPool.QueueUserWorkItem( SaveMyFile );
will do what you want.
I would recommend doing Asynchronous I/O. It's a little bit easier to set up and doesn't require you to create new threads yourself.
Asynchronous programming is where you have, for example, a file stream you want to write to but does not want to wait for it to finish. You might want to be notified when it's finished but you don't want to wait.
What you do is using the BeginWrite/BeginRead and EndWrite/EndRead functions that are available on the Stream class.
In your method you start by calling BeginWrite with all the data you want to write and also pass in a callback function. This function will be called when BeginWrite has finished.
Inside the callback function you call EndWrite and clean up the stream and check for errors.
BeginWrite will not block which means that if it's called from within an event handler that thread can finish that handler and continue processing more event (such as other GUI events).
using System;
using System.IO;
using System.Text;
class Program
{
private static FileStream stream;
static void Main(string[] args)
{
stream = new FileStream("foo.txt",
FileMode.Create,
FileAccess.Write);
const string mystring = "Foobarlalala";
ASCIIEncoding encoding = new ASCIIEncoding();
byte[] data = encoding.GetBytes(mystring);
Console.WriteLine("Started writing");
stream.BeginWrite(data, 0, data.Length, callback, null);
Console.WriteLine("Writing dispatched, sleeping 5 secs");
System.Threading.Thread.Sleep(5000);
}
public static void callback(IAsyncResult ia)
{
stream.EndWrite(ia);
Console.WriteLine("Finished writing");
}
}
}
The sleeping is pretty important because the thread that's writing stuff will be killed if the main thread is killed off. This is not an issue in a GUI application, only here in this small example.
MSDN has a pretty good overview on how to write this stuff, and also some good articles on Asynch programming in general in case you go for the backgroundworker or ThreadPool.
or u could use old friends delegates.