I have a non-closing stream class that's wrapped in a using block with a binary reader, but for some reason when the block ends, my non-closing stream still closes.
The stream is defined as:
internal class NonClosingStream : Stream, IDisposable
{
private Stream baseStream;
public NonClosingStream(Stream baseStream)
{
this.baseStream = baseStream;
}
public override bool CanRead{ get { return baseStream.CanRead; } }
public override bool CanSeek{ get { return baseStream.CanSeek; } }
public override bool CanWrite { get { return baseStream.CanWrite; } }
public override void Flush()
{
baseStream.Flush();
}
public override long Length { get { return baseStream.Length; } }
public override long Position
{
get { return baseStream.Position; }
set { baseStream.Position = value; }
}
public override int Read(byte[] buffer, int offset, int count)
{
return baseStream.Read(buffer, offset, count);
}
public override long Seek(long offset, SeekOrigin origin)
{
return baseStream.Seek(offset, origin);
}
public override void SetLength(long value)
{
baseStream.SetLength(value);
}
public override void Write(byte[] buffer, int offset, int count)
{
baseStream.Write(buffer, offset, count);
}
public override void Close()
{
// Disconnects from base stream, but does not close it
this.baseStream = null;
}
void IDisposable.Dispose()
{
// Disconnects from base stream, but does not close it
this.baseStream = null;
}
}
and the read block looks like this:
public T Deserialize<T>(Stream stream)
{
using (NonClosingStream nonClosingStream = new NonClosingStream(stream))
using (BinaryReader reader = new BinaryReader(nonClosingStream, Encoding.ASCII, true))
{
// Read the type name, then convert it to an actual type
String typeName = reader.ReadString();
Type graphType = AvailableTypes.GetType(typeName);
// If a deserializer for this type already exists, use it.
if (deserializerFunctions.ContainsKey(graphType))
{
return (T)deserializerFunctions[graphType](reader);
}
// Otherwise, create one and use it
T graph = (T)FormatterServices.GetUninitializedObject(graphType);
typeof(ServiceSerializer).GetMethod("DeserializeObject",
BindingFlags.NonPublic | BindingFlags.Static)
.MakeGenericMethod(graphType)
.Invoke(this, new Object[] { reader, graph });
return graph;
}
}
what am I doing wrong?
Update
So I wrote this little diddy:
static void Main()
{
MemoryStream stream = new MemoryStream();
using (NonClosingStream nonCloser = new NonClosingStream(stream))
using (BinaryWriter writer = new BinaryWriter(nonCloser))
using (BinaryReader reader= new BinaryReader(nonCloser))
{
writer.Write("Lorem ipsum");
stream.Seek(0, SeekOrigin.Begin);
String data = reader.ReadString();
Console.WriteLine(data);
}
stream.Seek(0, SeekOrigin.Begin);
using (NonClosingStream nonCloser = new NonClosingStream(stream))
using (BinaryWriter writer = new BinaryWriter(nonCloser))
using (BinaryReader reader = new BinaryReader(nonCloser))
{
writer.Write("Lorem ipsum");
stream.Seek(0, SeekOrigin.Begin);
String data = reader.ReadString();
Console.WriteLine(data);
}
Console.ReadLine();
}
and it seems to work fine, the stream stays open like it's supposed to. So I guess the consensus is right. Somehow I'm closing the stream elsewhere. When I figure out what I'll post the results. Thanks all.
Update
Gaaaahhh, I figured out the problem. So the way the code works is that while it serializes/deserializes an object, it then builds a customized serializer out of expression trees and then compiles it so that future serializations are more fluid. That means my code is littered with stuff like this:
Action<BinaryReader, Object> assignmentAction = delegate(BinaryReader bReader, Object oGraph)
{
bReader.ReadByte(); // Read the next action
bReader.ReadString(); // Read the field name
bReader.ReadByte(); // Read the field type
// Call the assignment lambda
assignmentLambda(reader, deserializerFunctions[primitiveType], (T)oGraph);
};
did you catch that? No? Neither did I apparently. Let's add some context:
private static void DeserializeObject<T>(BinaryReader reader, T graph)
{
...
Action<BinaryReader, Object> assignmentAction = delegate(BinaryReader bReader, Object oGraph)
{
bReader.ReadByte(); // Read the next action
bReader.ReadString(); // Read the field name
bReader.ReadByte(); // Read the field type
// Call the assignment lambda
assignmentLambda(reader, deserializerFunctions[primitiveType], (T)oGraph);
};
...
}
The lambda is closing over reader from the outside block, instead of using the bReader provided when the cached deserializer runs. Hence, when the deserializer runs, it's using an already discarded Binary reader object rather than the fresh one being provided to it. I guess the issue wasn't that I was closing the stream, but that I was using a disposed reader. At least that explains why it would work once, and then fail the second time, since the second time it relies on the cached deserializer. Oops!
Thanks all.
Since your stream does not create inner stream most likely outer code closes your inner stream. Chances are your code look like:
NonClosingStream nonClosing;
using(var stream = new FileStream(...))
{
nonClosing = new NonClosingStream(stream );
....
}
// inner stream now closed and nonClosing will fail all operations.
void IDisposable.Dispose()
Your class has two Dispose() methods. The one you explicitly implemented. And the one that you inherited from the Stream class. Problem is, BinaryStream doesn't know beans about yours. It only knows about the one that Stream implements. Furthermore, when you use the BinaryStream(Stream) constructor, the BinaryStream object assumes ownership of the passed Stream object. Which means it will dispose that stream when it gets disposed itself. Perhaps you see the problem now, the inherited Dispose() method will be called, not yours. And it closes the base stream.
This is why Stream implements the Dispose pattern. You'll need to make it look like this instead:
internal class NonClosingStream : Stream {
protected override Dispose(bool disposing) {}
}
It depends on whether the stream which is being wrapped by your NonClosingStream class is referenced elsewhere. If not, then the underlying stream will have no references, so at some point afterwards its finalizer will close the stream.
Related
Often, the IDisposable object of a using block is itself constructed from another IDisposable object, e.g.
using (FileStream stream = File.Open(path, FileMode.Open))
using (MyObject obj = new MyObject(stream))
{
// do something with obj
}
Unfortunately the code above keeps the file stream open until the MyObject object is disposed.
To dispose of the file stream as soon as the MyObject constructor has completed, I could instead write:
MyObject CreateMyObject(string path)
{
using (FileStream stream = File.Open(path, FileMode.Open))
{
return new MyObject(stream);
}
}
using (MyObject obj = CreateMyObject(path))
{
// do something with obj
}
But I don't like the verbosity of this solution. I tried replacing CreateMyObject() with a lambda but I failed to find a legal syntax. Is there a way to do this without calling a custom creator function?
Edit: Bearing in mind some of the comments, I should point out that I'm trying to avoid try...finally - kind of the main reason for a using block in the first place.
Additional clarification: The MyObject object is constructed from information in the stream - i.e. its constructor reads the content of the stream in its entirity. No other method in MyObject references the stream. The content of the stream could come from anywhere - a file, a resource, an Internet socket, etc.
You could invoke some magic like so:
TResult CreateUsingDisposable<TDisposable, TResult>(TDisposable disposable, Func<TDisposable, TResult> getResult)
where TDisposable : IDisposable
{
using (disposable)
{
return getResult(disposable);
}
}
using (var obj = CreateUsingDisposable(new FileStream(path, FileMode.Open), stream => new MyObject(stream)))
{
}
But why? There's a super easy to read no-nonsense way of doing that:
MyObject obj;
using (var stream = new FileStream(path, FileMode.Open))
{
obj = new MyObject(stream);
}
using (obj)
{
}
While I don't see a way to avoid the creator function, you can make it generic enough to define once and use for any class:
static T WithStream<T>(string path, Func<FileStream, T> getter)
{
using (FileStream stream = File.Open(path, FileMode.Open))
{
return getter(stream);
}
}
class MyObject : IDisposable
{
public MyObject (Stream stream){ /* Work with stream */}
public void Dispose(){}
}
static void Main()
{
using (MyObject obj = WithStream("path", fs => new MyObject(fs)))
{
// do something with obj
}
}
I have many files on disk that I need to read and deserialize, If i try to access these files using a streamreader from multiple tasks I get the following exception:
System.ObjectDisposedException: Cannot access a disposed object
System.Runtime.InteropServices.SafeHandle.DangerousReleaseInternal(bool dispose)SafeHandle.cs:193
System.Runtime.InteropServices.SafeHandle.DangerousRelease()SafeHandle.cs:142
System.IO.FileStream.Dispose(bool disposing)FileStream.cs:913
System.IO.IsolatedStorage.IsolatedStorageFileStream.Dispose(bool disposing)IsolatedStorageFileStream.cs:250
System.IO.Stream.Close()stream.cs:248
System.IO.StreamReader.Dispose(bool disposing)streamreader.cs:296
System.IO.TextReader.Dispose()textreader.cs:78
System.IO.TextReader.Dispose()(wrapper remoting-invoke-with-check)
.FileServices.StreamReaderWrapper.Dispose()
.FileServices.IsolatedSerializer.<DeserializeAsync>d__9<T>.MoveNext()
Here is the code:
public async Task<T> DeserializeAsync<T>(string path)
{
T result = default(T);
if (!_fileRepository.FileManager.FileExists(path)) { return result; }
using (var streamReader = _streamReader.GetNew(_fileRepository.FileManager.OpenFile(
path, FileMode.Open, FileAccess.Read, FileShare.Read)))
{
var contents = await streamReader.ReadToEndAsync();
result = _jsonSerializer.DeserializeFromString<T>(contents, Encoding.UTF8);
streamReader.Dispose();
}
return result;
}
What am I doing wrong here?
/// <summary>
/// Wrapper to allow testing with StreamReader
/// </summary>
public class StreamReaderWrapper : IStreamReader
{
private StreamReader _streamReader;
public void Dispose()
{
if (_streamReader != null)
{
_streamReader.Dispose();
_streamReader = null;
}
}
public IStreamReader GetNew(Stream stream)
{
Dispose();
_streamReader = new StreamReader(stream);
return this;
}
public string ReadToEnd()
{
return _streamReader.ReadToEnd();
}
public Task<string> ReadToEndAsync()
{
return _streamReader.ReadToEndAsync();
}
}
You are calling Dispose twice. The using statement will dispose your streamReader variable automatically after exiting the code block. But you are also explicitly calling streamReader.Dispose();
Also, your StreamReaderWrapper.GetNew() should just return a new instance of the wrapper, otherwise threads will step on each other's toes
I think you can't access the "getNew" function, as you already disposed the object..
from your code:
using (var streamReader = _streamReader.GetNew(_fileRepository.FileManager.OpenFile(
who's "_streamReader"? an instance of the wrapper or the wrapped reader?
Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 7 years ago.
Improve this question
I have an input stream and a stream reader component. This works fine but now I want to log all the traffic (save a copy in a file).
So I need to spy on a stream. A solution I am thinking of is a stream pipe (pipeline) or a stream wrapper that takes a stream as input and then gives me a first look at the traffic. Something like this:
void Init(System.Net.Sockets.NetworkStream stream)
{
System.IO.Stream wrappedStream = new MyWrapper(stream);
wrappedStream.ReadSpy = MyMethod;
XmlReader reader = XmlReader.Create(wrappedStream);
}
// This will get called after some bytes have been read from a stream,
// but before they get passed to the XmlReader
byte[] MyMethod(byte[] buffer)
{
m_Writer.Write(buffer); // write to a file
return buffer; // Give to XmlReader
}
What you want is called the Decorator Pattern. It's a technique for dynamically adding/modifying behavior:
http://sourcemaking.com/design_patterns/decorator
http://www.oodesign.com/decorator-pattern.html
To do this, you want to implement the abstract class Stream, with a constructor of factory that accepts another Stream instance. You provide an implementation for every method/overload of the abstract class that invokes the same method/overload on the decorated Stream, plus doing whatever additional work your needs require.
Once you've done that and decorated a Stream with your new decorator, It can be used interchangeably by anything else that accepts a Stream, including other similar decorators: decorators can even be nested, like layers of an onion to compose the behaviors you need.
Something like this:
class StreamInterceptor : Stream
{
public Stream DecoratedInstance { get; set; }
public event Action<byte[]> BytesRead;
public event Action<byte[]> BytesWritten;
public StreamInterceptor( Stream instance )
{
if ( instance == null ) throw new ArgumentNullException("instance");
this.DecoratedInstance = instance ;
return ;
}
public override bool CanRead
{
get { return DecoratedInstance.CanRead; }
}
public override bool CanSeek
{
get { return DecoratedInstance.CanSeek; }
}
public override bool CanWrite
{
get { return DecoratedInstance.CanWrite; }
}
public override void Flush()
{
DecoratedInstance.Flush();
return;
}
public override long Length
{
get { return DecoratedInstance.Length; }
}
public override long Position
{
get { return DecoratedInstance.Position; }
set { DecoratedInstance.Position = value; }
}
public override int Read( byte[] buffer , int offset , int count )
{
int bytesRead = DecoratedInstance.Read(buffer, offset, count);
// raise the bytes read event
byte[] temp = new byte[bytesRead];
Array.Copy(buffer,offset,temp,0,bytesRead);
BytesRead(temp);
return bytesRead;
}
public override long Seek( long offset , SeekOrigin origin )
{
return DecoratedInstance.Seek(offset, origin);
}
public override void SetLength( long value )
{
DecoratedInstance.SetLength(value);
return;
}
public override void Write( byte[] buffer , int offset , int count )
{
// raise the bytes written event
byte[] temp = new byte[count];
Array.Copy(buffer,offset,temp,0,count);
BytesWritten(temp);
DecoratedInstance.Write(buffer, offset, count);
return;
}
}
Once you have that, you can say something like this:
static void Main()
{
StreamInterceptor si = new StreamInterceptor(File.Open("foo.bar.txt",FileMode.Open,FileAccess.ReadWrite,FileShare.Read));
si.BytesRead += (bytes) => { Console.WriteLine("{0} bytes read", bytes.Length); } ;
si.BytesWritten += (bytes) => { Console.WriteLine("{0} bytes written", bytes.Length); } ;
Stream s = (Stream) si ;
DoSomethingUseful(s);
}
And your event handler will be invoked whenever somebody reads or writes from the stream.
I have a few different methods which connect to a remote host, send a message, get a reply, and use the information. this has worked fine until i use two methods in the same instance of my connection class. in the example where i'm getting the error, i run the following method;
public string sendRequestAccountID(string siteID)
{
//build message
String response = String.Empty;
TcpClient client = new TcpClient();
client.Connect(detailsHere);
NetworkStream stream = client.GetStream();
StreamWriter writer = new StreamWriter(stream);
writer.AutoFlush = false;
writer.WriteLine(sb.ToString());
writer.Flush();
StreamReader reader = new StreamReader(stream);
List<XmlNode> nodeList = new List<XmlNode>();
byte[] responseBytes = new byte[4096];
int bytesRead = 0;
while (true)
{
bytesRead = stream.Read(responseBytes, 0, responseBytes.Length);
if (bytesRead > 0)
{
//handle message
}
if (bytesRead == 0)
{
stream.Flush();
stream.Close();
client.Close();
string finalResponse = stuffHereToSend;
return finalResponse;
}
}
}
This sends fine, and returns a message as expected. However, if i then use the same instance of my connection class and use the following method;
public bool sendNewDevice(IDeviceInterface device)
{
NetworkStream stream;
sb = new StringBuilder();
//build message
String response = String.Empty;
TcpClient client = new TcpClient();
client.Connect(detailsHere);
stream = client.GetStream();
StreamWriter writer = new StreamWriter(stream);
writer.AutoFlush = false;
writer.WriteLine(sb.ToString());
writer.Flush();
StreamReader reader = new StreamReader(stream);
List<XmlNode> nodeList = new List<XmlNode>();
byte[] responseBytes = new byte[4096];
int bytesRead = 0;
while (true)
{
bytesRead = stream.Read(responseBytes, 0, responseBytes.Length);
if (bytesRead > 0)
{
//handle message
}
}
}
I'm getting an error which reads "cannot access a disposed object" on;
bytesRead = stream.Read(responseBytes, 0, responseBytes.Length);
Although i thought that i'd newly just assigned the stream in the latest method. Is it trying to use the previously closed one? is there a way around this or something silly that i'm missing?
edit: is it something to with the clients not disposing correctly? the two methods are ran within a second of each other, maybe the second is trying to open before the first has closed?
When a StreamWriter (and reader) is closed or its Dispose method is called, it disposes the underlying stream. Prior to .net 4.5 there really wasn't anything you could do about it, other than use something other than StreamWriter or write a class to wrap the Stream given to StreamWriter and ignore the call to Dispose. in .NET 4.5, there is an overload you can use to tell the StreamWriter not to dispose the stream you give it. eg: new StreamWriter(stream, StreamWriter.UTF8NoBOM, 1024, false)
You could try using a wrapped stream to ignore the close (calling new StreamWriter(new NonDisposableStreamWrapper(stream))):
public class NonDisposableStreamWrapper : Stream
{
private Stream wrappedStream;
public NonDisposableStreamWrapper(Stream wrappedStream)
{
this.wrappedStream = wrappedStream;
}
public override void Flush()
{
wrappedStream.Flush();
}
public override long Seek(long offset, SeekOrigin origin)
{
return wrappedStream.Seek(offset, origin);
}
public override void SetLength(long value)
{
wrappedStream.SetLength(value);
}
public override int Read(byte[] buffer, int offset, int count)
{
return wrappedStream.Read(buffer, offset, count);
}
public override void Write(byte[] buffer, int offset, int count)
{
wrappedStream.Write(buffer, offset, count);
}
public override bool CanRead
{
get { return wrappedStream.CanRead; }
}
public override bool CanSeek
{
get { return wrappedStream.CanSeek; }
}
public override bool CanWrite
{
get { return wrappedStream.CanWrite; }
}
public override long Length
{
get { return wrappedStream.Length; }
}
public override long Position
{
get { return wrappedStream.Position; }
set { wrappedStream.Position = value; }
}
}
I'm using the Amazon SDK and I have a method that returns a Stream for an object stored in Amazon S3 service.
It contains something like this:
var request = new GetObjectRequest().WithBucketName(bucketName).WithKey(keyName);
using (var response = client.GetObject(request))
{
return response.ResponseStream;
}
Obviously, when doing this, the stream is not readable from the calling method because the request object has been disposed and when this is done, it closes the stream.
I don't want to download the file to a MemoryStream or a FileStream.
If I don't use the using clause, the garbage collector will dispose the request object at some point so I can't just not use it.
What I'm asking is, is there a way to wrap or copy the Stream into another Stream and then return it without having to download the file?
I'm using .NET 3.5.
Edit: The method is inherited from an abstract class and the caller method doesn't know it is working with Amazon. So it HAS to return a Stream.
You can't work with the stream after it's disposed, but you can postpone disposing the response object until after the response stream has been used. There are three options I can suggest.
Return Response. One option is to return the response object to the caller. The caller can access the contained response stream and then dispose the response when done. This is the easiest change but requires the caller to change as well.
Wrap the stream. Instead of returning the response stream directly, create a new object that extends Stream and wraps the response stream but also has a reference to the response itself. Then when your wrapper is disposed, you can internally dispose the response object. This requires only changing your called code as long as the type being returned is just Stream.
Callback. Don't return anything from the method and use a callback mechanism instead. Accept an Action<Stream> as a parameter and call this callback with the stream. That way the callers code is called and you still have control over disposing the response and stream. This is the safest mechanism since you're not relying on the caller to dispose anything, but requires the most changes.
There is a method OpenStream in TransferUtility class which returns a stream object.
public Stream OpenStream(
String bucketName,
String key
)
I looked through the source code of AWSSDK, and found in the SDK the OpenStream just returns the response stream directly. (It doesn't use "using" statement on response object.)
If your function returns the response object (without using the using statement), and the caller assigns it to a variable, there will still be a reference to the response object. Thus it won't be eligible garbage collected.
Had the same issue (I thought). But have you tried to not use "using". This will not use the stream and send it to the caller who will have the responsibility to dispose it. As simple as that.
var request = new GetObjectRequest { BucketName = containerName, Key = blobName };
GetObjectResponse response = null;
try
{
response = client.GetObject(request));
}
catch (AmazonS3Exception ex)
{
if ((ex.ErrorCode == "NoSuchBucket") || (ex.ErrorCode == "AccessDenied") || (ex.ErrorCode == "InvalidBucketName") || (ex.ErrorCode == "NoSuchKey"))
{
return null;
}
throw;
}
return response.ResponseStream;
For completeness, and since I went with #Samuel option number 2 (Wrap the stream) which #spakinz commented he did too, I include here my implementation of what I called AmazonS3Stream
public class AmazonS3Stream : Stream
{
private Stream stream;
private GetObjectResponse response;
public AmazonS3Stream(GetObjectResponse response)
{
this.stream = response.ResponseStream;
this.response = response;
}
// The whole purpose of this class
protected override void Dispose(bool disposing)
{
// base.Dispose(disposing); // Do we really need this line? Probably not since I tested it and I can see that the stream is disposed when Response.Dispose is called by itself. And that makes sense because we know that this.stream is pointing to response.ResponseStream (that's what we declared in the constructor: this.stream = response.ResponseStream; ) So, what do we expect from response.Dispose() ? Obviously the first thing it would do is call ResponseStream.Dispose()
response.Dispose();
}
public override long Position
{
get { return stream.Position; }
set { stream.Position = Position; }
}
public override long Length
{
get { return stream.Length; }
}
public override bool CanRead
{
get { return stream.CanRead; }
}
public override bool CanSeek
{
get { return stream.CanSeek; }
}
public override bool CanWrite
{
get { return stream.CanWrite; }
}
public override void Flush()
{
stream.Flush();
}
public override void Write(byte[] buffer, int offset, int count)
{
stream.Write(buffer, offset, count);
}
public override void SetLength(long value)
{
stream.SetLength(value);
}
public override long Seek(long offset, SeekOrigin origin)
{
return stream.Seek(offset, origin);
}
public override int Read(byte[] buffer, int offset, int count)
{
return stream.Read(buffer, offset, count);
}
}
To add a code example to #Jun Y.'s answer. This is what I did and it was the cleanest solution by far.
using (var client = new AmazonS3Client(Amazon.RegionEndpoint.USEast2))
{
var transferUtility = new TransferUtility(client);
return await transferUtility.OpenStreamAsync(S3BucketName, key);
}