I just need to read up to N bytes from a SslStream but if no byte has been received before a timeout, cancel, while leaving the stream in a valid state in order to try again later. (*)
This can be done easily for non-SSL streams i.e. NetworkStream simply by using its ReadTimeout property which will make the stream throw an exception on timeout. Unfortunately this approach doesn't work on SslStream per the official docs:
SslStream assumes that a timeout along with any other IOException when one is thrown from the inner stream will be treated as fatal by its caller. Reusing a SslStream instance after a timeout will return garbage. An application should Close the SslStream and throw an exception in these cases.
[Updated 1] I tried a different approach like this:
task = stream->ReadAsync(buffer, 0, buffer->Length);
if (task->Wait(timeout_ms)) {
count = task->Result;
...
}
But this doesn't work if Wait() returned false: when calling ReadAsync() again later it throws an exception:
Exception thrown: 'System.NotSupportedException' in System.dll
Tests.exe Warning: 0 : Failed reading from socket: System.NotSupportedException: The BeginRead method cannot be called when another read operation is pending.
[Update 2] I tried yet another approach to implement timeouts by calling Poll(timeout, ...READ) on the underlying TcpClient socket: if it returns true, then call Read() on the SSlStream, or if it returns false then we have a timeout. This doesn't work either: because SslStream presumably uses its own internal intermediary buffers, Poll() can return false even if there's data left to be read in the SslStream.
[Update 3] Another possibility would be to write a custom Stream subclass that would sit between NetworkStream and SslStream and capture the timeout exception and return 0 bytes instead to SslStream. I'm not sure how to do this, and more importantly, I have no idea if returning a 0 bytes read to SslStream would still not corrupt it somehow.
(*) The reason I'm trying to do this is that reading synchronously with a timeout from a non-secure or secure socket is the pattern I'm already using on iOS, OS X, Linux and Android for some cross-platform code. It works for non-secure sockets in .NET so the only case remaining is SslStream.
You can certainly make approach #1 work. You simply need to keep track of the Task and continue waiting without calling ReadAsync again. So, very roughly:
private Task readTask; // class level variable
...
if (readTask == null) readTask = stream->ReadAsync(buffer, 0, buffer->Length);
if (task->Wait(timeout_ms)) {
try {
count = task->Result;
...
}
finally {
task = null;
}
}
Needs to be fleshed-out a bit so the caller can see that the read isn't completed yet but the snippet is too small to give concrete advice.
I also encountered this problem with an SslStream returning five bytes of garbage data on the read after a timeout, and I separately came up a solution that is similar to OP's Update #3.
I created a wrapper class which wraps the Tcp NetworkStream object as it is passed into the SslStream constructor. The wrapper class passes all calls onto to the underlying NetworkStream except that the Read() method includes an extra try...catch to suppress the Timeout exception and return 0 bytes instead.
SslStream works correctly in this instance, including raising the appropriate IOException if the socket is closed. Note that our Stream returning 0 from a Read() is different from a TcpClient or Socket returning 0 from a Read() (which typically means a socket disconnect).
class SocketTimeoutSuppressedStream : Stream
{
NetworkStream mStream;
public SocketTimeoutSuppressedStream(NetworkStream pStream)
{
mStream = pStream;
}
public override int Read(byte[] buffer, int offset, int count)
{
try
{
return mStream.Read(buffer, offset, count);
}
catch (IOException lException)
{
SocketException lInnerException = lException.InnerException as SocketException;
if (lInnerException != null && lInnerException.SocketErrorCode == SocketError.TimedOut)
{
// Normally, a simple TimeOut on the read will cause SslStream to flip its lid
// However, if we suppress the IOException and just return 0 bytes read, this is ok.
// Note that this is not a "Socket.Read() returning 0 means the socket closed",
// this is a "Stream.Read() returning 0 means that no data is available"
return 0;
}
throw;
}
}
public override bool CanRead => mStream.CanRead;
public override bool CanSeek => mStream.CanSeek;
public override bool CanTimeout => mStream.CanTimeout;
public override bool CanWrite => mStream.CanWrite;
public virtual bool DataAvailable => mStream.DataAvailable;
public override long Length => mStream.Length;
public override IAsyncResult BeginRead(byte[] buffer, int offset, int size, AsyncCallback callback, object state) => mStream.BeginRead(buffer, offset, size, callback, state);
public override IAsyncResult BeginWrite(byte[] buffer, int offset, int size, AsyncCallback callback, object state) => mStream.BeginWrite(buffer, offset, size, callback, state);
public void Close(int timeout) => mStream.Close(timeout);
public override int EndRead(IAsyncResult asyncResult) => mStream.EndRead(asyncResult);
public override void EndWrite(IAsyncResult asyncResult) => mStream.EndWrite(asyncResult);
public override void Flush() => mStream.Flush();
public override Task FlushAsync(CancellationToken cancellationToken) => mStream.FlushAsync(cancellationToken);
public override long Seek(long offset, SeekOrigin origin) => mStream.Seek(offset, origin);
public override void SetLength(long value) => mStream.SetLength(value);
public override void Write(byte[] buffer, int offset, int count) => mStream.Write(buffer, offset, count);
public override long Position
{
get { return mStream.Position; }
set { mStream.Position = value; }
}
public override int ReadTimeout
{
get { return mStream.ReadTimeout; }
set { mStream.ReadTimeout = value; }
}
public override int WriteTimeout
{
get { return mStream.WriteTimeout; }
set { mStream.WriteTimeout = value; }
}
}
This can then be used by wrapping the TcpClient NetworkStream object before it's passed to the SslStream, as follows:
NetworkStream lTcpStream = lTcpClient.GetStream();
SocketTimeoutSuppressedStream lSuppressedStream = new SocketTimeoutSuppressedStream(lTcpStream);
using (lSslStream = new SslStream(lSuppressedStream, true, ServerCertificateValidation, SelectLocalCertificate, EncryptionPolicy.RequireEncryption))
The problem comes down to SslStream corrupting its internal state on any exception from the underlying stream, even a harmless timeout. Oddly, the five (or so) bytes of data that the next read() returns are actually the start of the TLS encrypted payload data from the wire.
Hope this helps
Related
I'm trying to convert a class which uses multiple threads to use overlapping I/O. It's almost working, but it seems to randomly hit a threading issue, and I'm not sure why.
Too much code to post directly, but here's the basic pattern. The goal is to sit there reading data off the connection until the connection get disposed, so when each EndRead() completes, it should start a new BeginRead().
public enum State
{
Idle,
BeforeRead,
PendingRead,
FinishingRead,
Died,
}
private int state;
private IAsyncResult asyncResult;
private byte[] readBuffer = new byte[4096];
private System.Net.Sockets.NetworkStream stream;
public void Connect(System.Net.Sockets.TcpClient client, string host, int port)
{
client.Connect(host, port);
this.stream = client.GetStream();
}
private bool SetState(State expectedState, State newState)
{
return Interlocked.CompareExchange(ref this.state, (int)newState, (int)expectedState) == expectedState;
}
public void BeginRead()
{
try
{
while (true)
{
if (!SetState(State.Idle, State.BeforeRead))
return;
IAsyncResult async;
async = stream.BeginRead(readBuffer, 0, readBuffer.Length, x => EndRead(true), null);
if (async == null)
return;
SetState(State.BeforeRead, State.PendingRead);
lock (this)
this.asyncResult = async;
if (async.AsyncWaitHandle.WaitOne(0))
EndRead(false);
}
}
catch { this.state = State.Died; }
}
private void EndRead(bool asynchronousCallback)
{
try
{
if (!SetState(State.PendingRead, State.FinishingRead))
return;
IAsyncResult async;
lock (this)
{
async = this.asyncResult;
this.asyncResult = null;
}
if (async == null)
return;
int bytesRead = stream.EndRead(async);
HandleData(bytesRead, readBuffer);
SetState(State.FinishingRead, State.Idle);
if (asynchronousCallback)
BeginRead();
}
catch { this.state = State.Died; }
}
Most of the time it works, but occasionally it does one of several things:
Stops receiving messages
Throws an exception that I the asyncResult has already been handled: "EndReceive can only be called once for each asynchronous operation".
I should also mention that there is synchronous writing going on from another thread (stream.Write, not stream.BeginWrite). I think reading and writing should be independent of each other, so it shouldn't affect the behavior.
Is my design fundamentally flawed? This is a stripped down example, so it's possible the stuff I stripped out could be causing the problem, but I need to know if my basic design is valid or not. What is the proper way to chain read asynchronously?
(And in case the suggestion is to use async/await, this code needs to run on Windows XP, so that's not an option.)
You have a race condition:
IAsyncResult async;
async = stream.BeginRead(readBuffer, 0, readBuffer.Length, x => EndRead(true), null);
/* Race condition here */
if (async == null)
return;
SetState(State.BeforeRead, State.PendingRead);
lock (this)
this.asyncResult = async;
Your EndRead can execute before the SetState and/or before the this.asyncResult = async executes. You cannot do this. The state must be set before the BeginRead is issued, and reset in case of failure. Do not retain and use a member asyncResult, but instead pass the callback to BeginRead and get the async result on the callback:
SetState(State.BeforeRead, State.PendingRead);
stream.BeginRead(readBuffer, 0, readBuffer.Length, EndRead);
...
private void EndRead(IAsyncResult asyncResult) {
int bytesRead = stream.EndRead(asyncResult);
...
}
I'm fixing a bug : the app will hang in download process for 5 minutes while losing network connection. What I need to do is reduce the hanging time to like 20 seconds.
log snippet:
Severe 2016-01-27 , 11:03:14
在 HttpsDownload.doDownloadByHttps(FileDownloadInfo fileInfo, downloadingDelegate downDelegate)
Message: Init method:GET
Info 2016-01-27 , 11:08:29
在 HttpsDownload.writeToFile(FileDownloadInfo fileInfo, Stream stream, downloadingDelegate downDelegate, Boolean Clear)
Message: Exception while reading from the response stream
Exception:
Message: time out
Type: System.Net.WebException
在 System.Net.ConnectStream.Read(Byte[] buffer, Int32 offset, Int32 size)
在 HttpsDownload.writeToFile(FileDownloadInfo fileInfo, Stream stream, downloadingDelegate downDelegate, Boolean Clear)
code snippet:
private void doDownloadByHttps(FileDownloadInfo fileInfo, downloadingDelegate downDelegate)
{
//code
webResponse = (HttpWebResponse)webRequest.GetResponse();
responseStream = webResponse.GetResponseStream();
writeToFile(fileInfo, responseStream, downDelegate, bClear);
}
private void writeToFile(FileDownloadInfo fileInfo, Stream stream, downloadingDelegate downDelegate, bool Clear)
{
int count = stream.Read(readBuffer, 0, BlockSize);
while (count > 0)
{
//code
count = stream.Read(readBuffer, 0, BlockSize);
}
}
I believe that timeout occurs in stream.Read according to the log. So the easiest way to do the trick should be stream.ReadTimeout = 20000, but obviously I can't set the ReadTimeout property.
Another way that can think of is to write a class override ReadTimeout like:
public class MyStream : Stream {
//code
public override int ReadTimeout { get; set; }
//code
}
But it's not preferred. Any better way to fix the bug?
The timeout needs to be set in the webRequest object, of type HttpWebRequest.
Check this out: https://msdn.microsoft.com/en-US/library/system.net.httpwebrequest.readwritetimeout(v=vs.110).aspx
I am developing a Windows application that uploads a file to a webserver, IIS. My code is working just fine when I run the app on a 64Bit Machine. Upload to the IIS is working. But when I run it on a 32Bit machine, the upload is not working.
I think it has something to do with IIS. But I don´t know what it could be. Does someone has experienced same issues?
UPDATE: This has nothing to with the server side. I tested several endpoints, but nothing worked.
This must be related to my upload code. This code is working from 64Bit Apps but not on 32Bit:
try
{
System.Net.Http.HttpClient hc = new System.Net.Http.HttpClient();
hc.DefaultRequestHeaders.TryAddWithoutValidation("Accept", "text/html,application/xhtml+xml,application/xml");
hc.DefaultRequestHeaders.TryAddWithoutValidation("Accept-Encoding", "gzip, deflate");
hc.DefaultRequestHeaders.TryAddWithoutValidation("User-Agent", "Mozilla/5.0 (Windows NT 6.2; WOW64; rv:19.0) Gecko/20100101 Firefox/19.0");
hc.DefaultRequestHeaders.TryAddWithoutValidation("Accept-Charset", "ISO-8859-1");
using (VirtualStream ms = new VirtualStream() { Size = UploadSize })
{
StreamContent content = new StreamContent(ms, BufferSize);
// time for the calculation of the total average throughput
var overallStart = DateTime.Now;
var start = DateTime.Now;
var responseTask = hc.PostAsync(URL, content);
while (!responseTask.IsCompleted)
{
// Check the Exit and Abort Constraints
if ((DateTime.Now - overallStart).TotalMilliseconds > MaxTestLength || _cancelationRequested)
{
System.Diagnostics.Debug.WriteLine("Bytes sent " + bytesSent);
hc.CancelPendingRequests();
IsRunning = false;
return;
}
try
{
bytesSent = ms.Position - bytesOfCalibrationPhase;
}
catch (Exception)
{
// The Upload is an async process which dispses the underlying stream when the upload finishes
// In some cases this could lead to ObjectDiposed Exceptions when accessing the current stream position
// If it is the case, the upload has finished....
break;
}
}
BytesSent is always "0" on 32Bit machines...Why is that?
It seems to me that the loop you have is there for three reasons:
1) You want to cancel the upload if cancellation is requested
2) You want to cancel the upload if there is a timeout
3) you want to know the progress of the upload
I suggest that you remove the loop completely, and achieve these tree goals in a different way. Here is how you can do this:
For (1), use the other overload of the PostAsync method that has a CancellationToken parameter. This allows you to provide a token that you can use from somewhere else to cancel the upload operation.
For (2), you can use the CancellationTokenSource (that you used to create the CancellationToken), to request that the upload operation be canceled after some time (if the task was not already completed). See the CancelAfter method.
Here is some code sample for (1) and (2):
Put the following two lines in some place (probably as fields) so that these variables are available from both your upload code and the code that might wish to cancel the upload:
CancellationTokenSource cancellation_token_source = new CancellationTokenSource();
CancellationToken cancellation_token = cancellation_token_source.Token;
The following line of code will setup automatic cancellation after 10 seconds:
cancellation_token_source.CancelAfter(TimeSpan.FromSeconds(10));
In the following line, we pass the cancellation_token to the PostAsync method:
var responseTask = hc.PostAsync(URL, content, cancellation_token);
I noticed you were waiting for responseTask.IsCompleted to become true. This means that you don't want your method to return until the upload is complete. In this case, use the following to wait for the upload to complete.
responseTask.Wait();
In case you want to convert your method to become asynchronous, mark your method as async, and instead use the following:
await responseTask;
You can use the following to cancel the upload:
cancellation_token_source.Cancel();
For (3), first look at the answers in this question.
If this does not work for you, I have the following suggestion:
You can create a decorator for the Stream class, that informs you when the stream was read, like this (please note the Read method):
public class ReadNotifierStreamWrapper : Stream
{
private readonly Stream m_Stream;
private readonly Action<int> m_ReadNotifier;
public ReadNotifierStreamWrapper(Stream stream, Action<int> read_notifier)
{
m_Stream = stream;
m_ReadNotifier = read_notifier;
}
public override void Flush()
{
m_Stream.Flush();
}
public override long Seek(long offset, SeekOrigin origin)
{
return m_Stream.Seek(offset, origin);
}
public override void SetLength(long value)
{
m_Stream.SetLength(value);
}
public override int Read(byte[] buffer, int offset, int count)
{
var bytes_read = m_Stream.Read(buffer, offset, count);
m_ReadNotifier(bytes_read);
return bytes_read;
}
public override void Write(byte[] buffer, int offset, int count)
{
m_Stream.Write(buffer, offset, count);
}
public override bool CanRead
{
get { return m_Stream.CanRead; }
}
public override bool CanSeek
{
get { return m_Stream.CanSeek; }
}
public override bool CanWrite
{
get { return m_Stream.CanWrite; }
}
public override long Length
{
get { return m_Stream.Length; }
}
public override long Position
{
get { return m_Stream.Position; }
set { m_Stream.Position = value; }
}
}
And then you can use it to wrap your ms stream like this:
int total_bytes = 0;
var stream_wrapper = new ReadNotifierStreamWrapper(ms , bytes =>
{
total_bytes += bytes;
Debug.WriteLine("Bytes sent " + total_bytes);
});
HttpContent content = new StreamContent(stream_wrapper); //Here we are creating the StreamContent from stream_wrapper instead of ms
This way you will get a notification when the stream is being read. And you will know how much bytes were read.
Please note that you might need to work more on the ReadNotifierStreamWrapper class to make it better. For example, maybe the HttpClient decides for some reason that it wants to seek the stream via the Seek method. You might need to take that into account. Although, I don't think that HttpClient will do this since it needs to read the whole file to upload it all, it does not make sense to skip parts of the file. You can put some breakpoints on the Seek method of ReadNotifierStreamWrapper to see if it will get called.
The exception you get;
EventSourceException: No Free Buffers available from the operating
system
is caused by the Debug.WriteLine, which uses ETW (Event Tracing for Windows). ETW is a Windows kernel-level tracing facility that relies on a number of buffers to cache data before writing it to disk. ETW uses the buffer size and the size of physical memory to calculate the maximum number of buffers allocated for the event tracing session's buffer pool. So, if your application is 64-bit ETW will allocate more buffers due to there being more addressable memory available.
The reason you are hitting the limit is that you are writing debugging events faster than ETW can process the buffers.
I believe it is possible to increase the maximum number of buffers using registry entries, but this will increase memory consumption and you have the potential to hit the limit again with bigger files, slower connection, etc.
So your best bet is to either sleep the thread on each loop to write tracing events at a lower rate or replace the Debug.WriteLine with some other form of logging.
I'm getting a stream from HttpWebResponse.GetResponseStream() where I'm reading data from.
Now I want to implement a Timeout property. The easiest way to do it would be stream.ReadTimeout = timeout but this throws an InvalidOperationException -> Timeouts are not supported on this stream.
Given this, I'm trying to implement the timeout property myself but got stuck on a dispose. This is what I got so far:
public class MyStream : Stream {
private readonly Stream _src;
public override int ReadTimeout { get; set; }
public MyStream (Stream src, int timeout) {
ReadTimeout = timeout;
_src = src;
}
public override int Read(byte[] buffer, int offset, int count) {
var timer = new AutoResetEvent(false);
int read = 0;
ThreadPool.QueueUserWorkItem(
_ => {
read = _src.Read(buffer, offset, count);
timer.Set();
});
bool completed = timer.WaitOne(ReadTimeout);
if (completed) {
return read;
}
throw new TimeoutException(string.Format("waited {0} miliseconds", ReadTimeout));
}
The problem with this code is after is throws a TimeoutException that is being properly handled somewhere. It throws an Exception on _src.Read(buffer, offset, count) saying that the _src stream was disposed.
Is there a way to cancel the ThreadPool method or should I use a better approach and which one?
Thanks
EDIT
As asked by #JotaBe, were's the code where I get the stream from HttpWebResponse:
_httpRequest = WebRequest.CreateHttp(url);
_httpRequest.AllowReadStreamBuffering = false;
_httpRequest.BeginGetResponse(
result =>
{
try {
_httpResponse = (HttpWebResponse)_httpRequest.EndGetResponse(result);
stream = _httpResponse.GetResponseStream();
}
catch (WebException) {
downloadCompleted.Set();
Abort();
}
finally {
downloadCompleted.Set();
}
},
null);
bool completed = downloadCompleted.WaitOne(15 * 1000);
if (completed) {
return new MyStream(stream, 10000);
}
If you're trying to get a timeout if you don't receive and answer form a web server, you're trying to do it in the wrong place.
To get a Response, you usually make a Request:
HttpWebResponse response = (HttpWebResponse)request.GetResponse ();
This is the operation which can timeout. You have to call it in a differente way, using the Begin/End asynchronous pattern.
There is a full example of this in
MSDN doc for HttpWebRequest.BeginGetResponse Method
This example uses a callback function. However, there are many different ways to use Begin/End. For example, you can use a WaitHandle available in IAsyncResult like this:
IAsyncResult ar = req.BeginGetResponse(yourCallback, null);
bool completed = ar.AsyncWaitHandle.WaitOne(15000 /*your timeout in miliseconds*/);
This will wait 15 seconds. If the response arrives before this, completed will be true. If not, completed will be false. You can then use the HttpWebRequest.Abort() method to abort the request.
The Begin/End pattern takes charge of managing the neccesary threads.
I ended up using Nomad101 suggestion and surround the read with a try/catch.
I have a c# network application where alot of anonymous users connect to (game service).
Now I check the logs and occasionally I see this exception:
[10:30:18.21352] System.Int32 Read(Byte[], Int32, Int32): The stream does not support reading.
at System.Net.Sockets.NetworkStream.Read(Byte[] buffer, Int32 offset, Int32 size)
at BusinessLayer.Listener.ListenerWorker.ProcessClient(Object obj) in File.cs:line 141
This error comes from a NetworkStream object, now I am trying to reproduce the problem, but how? How can I get this exception?
I tried disconnecting myself, but that just gives a timeout, tried other things, but cannot get it to work.
Maybe somebody has an idea?
Contents of the file is:
private static void ProcessClient(
Object obj)
{
ISession session = (ISession)obj;
NetworkStream networkStream = null;
try
{
DebugUtility.SetThreadName("Worker: {0}", session.Name);
networkStream = session.TcpClient.GetStream();
networkStream.ReadTimeout = Config.ReadTimeout;
// Loop received packets (blocks untill next packet)
Int32 packetSize;
Byte[] buffer = new Byte[session.PacketSize];
while ((packetSize = networkStream.Read(buffer, 0, buffer.Length)) != 0)
{
// Get String from packet bytes
String packet = Encoding.UTF8.GetString(buffer, 0, packetSize);
// Check if packet has data
if (String.IsNullOrEmpty(packet))
continue;
// Log biggest received package
DebugUtility.CheckMaxPacketSize(session.Name, packet.Length);
// Handle packet (in new thread)
Logger.DebugLog("Received: {0}", packet);
ThreadPool.QueueUserWorkItem(session.HandlePacket, packet);
}
}
catch (Exception ex)
{
Logger.LogException(ex);
}
finally
{
if (networkStream != null)
networkStream.Close();
if (session != null)
session.Disconnect();
}
}
What arguments are you passing in the
System.Net.Sockets.NetworkStream.Read(Byte[] buffer, Int32 offset, Int32 size)
method. Are you using any of NetworkStream.Length or NetworkStream.Position properties.
i.e is it somthing like (not exactly)
System.Net.Sockets.NetworkStream.Read(buffer, stream.Position, stream.Length)
then as explained in MSDN documentation use of NetworkStream.Length and NetworkStream.Position properties will always throw a NotSupportedException as its not currently Supported.