Call to MemoryStream.GetBuffer() succeeds even after MemoryStream.Close(); Why? - c#

I have found the following construct in some open-source code:
var mstream = new MemoryStream();
// ... write some data to mstream
mstream.Close();
byte[] b = mstream.GetBuffer();
I thought this code would have "unexpected" behavior and maybe throw an exception, since the call to Close should effectively be a call to Dispose according to the MSDN documentation.
However, as far as I have been able to tell from experimenting, the call to GetBuffer() always succeeds and returns a valid result, even if I Thread.Sleep for 20 seconds or enforce garbage collection via GC.Collect().
Should the call to GetBuffer() succeed even after Close/Dispose? In that case, why is not the underlying buffer released in the MemoryStream disposal?

It doesn't need to. The buffer is managed memory, so normal garbage collection will deal with it, without needing to be included in the disposal.
It's useful to be able to get the bytes of the memory stream, even after the stream has been closed (which may have happened automatically after the steam was passed to a method that writes something to a stream and then closes said stream). And for that to work, the object needs to hold onto the buffer along with a record of how much had been written to it.
In considering the second point, it actually makes more sense in a lot of cases to call ToArray() (which as said, requires the in-memory store that GetBuffer() returns to still be alive) after you've closed the stream, because having closed the stream guarantees that any further attempt to write to the stream will fail. Hence if you have a bug where you obtain the array too early, it will throw an exception rather than just give you incorrect data. (Obviously if you explicitly want to get the current array part-way through the stream operations, that's another matter). It also guarantees that all streams are fully flushed rather than having part of their data in a temporary buffer (MemoryStream isn't buffered because MemoryStream essentially is a buffer, but you may have been using it with chained streams or writers that had their own separate buffer).

Technically there's nothing to dispose in MemoryStream. Literally nothing, it doesn't have operating system handles, unmanaged resources, nothing. it is just a wrapper around byte[]. All what you can do is set the buffer(internal array) to null, that BCL team hasn't done for some reason.
As #mike noted in comments BCL team wanted GetBuffer and ToArray to work, even after disposed, though we're not sure why?. Reference source.
Here's how Dispose implemented.
protected override void Dispose(bool disposing)
{
try {
if (disposing) {
_isOpen = false;
_writable = false;
_expandable = false;
// Don't set buffer to null - allow GetBuffer & ToArray to work.
#if FEATURE_ASYNC_IO
_lastReadTask = null;
#endif
}
}
finally {
// Call base.Close() to cleanup async IO resources
base.Dispose(disposing);
}
}
and GetBuffer is below
public virtual byte[] GetBuffer()
{
if (!this._exposable)
{
throw new UnauthorizedAccessException(Environment.GetResourceString("UnauthorizedAccess_MemStreamBuffer"));
}
return this._buffer;
}
As you can see in Dispose _buffer is untouched, and in GetBuffer no disposed checks.

Since the GC is non-deterministic you cannot force it to immediately dispose the MemoryStream, thus the instance will not be marked as disposed immediately, instead it will just be marked for disposal. This means that for some time, until it is really disposed you can use some of its functions. Since it keeps a strong reference to its buffer you can get it, here is what the GetBuffer method looks like:
public virtual byte[] GetBuffer()
{
if (!this._exposable)
{
throw new UnauthorizedAccessException(Environment.GetResourceString("UnauthorizedAccess_MemStreamBuffer"));
}
return this._buffer;
}

Unlike most interface methods, the IDisposable.Dispose does not promise to do anything. Instead, it provides a standard means by which the owner of an object can let that object know that its services are no longer required, in case the object might need to make use of that information. If an object has asked outside entities to do something on its behalf, and has promised those outside entities that it will let them know when their services are no longer required, its Dispose method can relay the notification to those entities.
If an object has a method which can only be performed while the object has outside entities acting on its behalf, an attempt to call that method after those entities have been dismissed should throw an ObjectDisposedException rather than failing in some other way. Further, if there is a method which cannot possibly be useful after the entity is dismissed, it should often throw ObjectDisposedException even if a particular didn't actually need to use the entity. On the other hand, if a particular call would have a sensible meaning after an object has dismissed all entities that were acting on its behalf, there's no particular reason why such a call shouldn't be allowed to succeed.
I would view the ObjectDisposedException much like I view the collection-modified InvalidOperationException of IEnumerator<T>.MoveNext(): if some condition (either Dispose, or modification of a collection, respectively) would prevent a method from behaving "normally", the method is allowed to throw the indicated exception, and is not allowed to behave in some other erroneous fashion. On the other hand, if the method is capable of achieving its objectives without difficulty, and if doing so would make sense, such behavior should be considered just as acceptable as would be throwing an exception. In general, objects are not required to operate under such adverse conditions, but sometimes it may be helpful for them to do so [e.g. enumeration of a ConcurrentDictionary will not be invalidated by changes to the collection, since such invalidation would make concurrent enumeration useless].

Related

Should Dispose() ever create new instances of objects?

Using C#.NET 4.0
My company's application makes use of a resource locker to keep records from being edited simultaneously. We use the database to store the start time of a lock as well as the user who acquired the lock. This has led to the following (strange?) implementation of dispose on the resource locker, which happens to be called from a destructor:
protected virtual void Dispose(bool disposing)
{
lock (this)
{
if (lockid.HasValue)
{
this.RefreshDataButtonAction = null;
this.ReadOnlyButtonAction = null;
try
{
**Dictionary<string, object> parameters = new Dictionary<string, object>();
parameters.Add("#lockID", lockid.Value);
parameters.Add("#readsToDelete", null);
Object returnObject = dbio2.ExecuteScalar("usp_DeleteResourceLockReads", parameters);**
lockid = null;
}
catch (Exception ex)
{
Logger.WriteError("ResourceLockingController", "DeleteResourceLocks", ex);
}
finally
{
((IDisposable)_staleResourcesForm).Dispose();
_staleResourcesForm = null;
}
}
}
}
I am concerned about the bolded section we because have been logging strange "Handle is not initialized" exceptions from the database call. I read elsewhere that it is not safe to create new objects during Finalize(), but does the same rule apply to dispose()? Are there any possible side effects that accompany creating new objects during Dispose()?
which happens to be called from a destructor
That's the real problem. You cannot assume that the *dbio2" object hasn't been finalized itself. Finalization order is not deterministic in .NET. The outcome would look much like you describe, an internal handle used by the dbase provider will have been released so a "Handle is not initialized" exception is expected. Or the dbio2 object was simply already disposed.
This is especially likely to go wrong at program exit. You'll then also have problem when the 2 second timeout for the finalizer thread, a dbase operation can easily take more.
You simply cannot rely on a finalizer to do this for you. You must check the disposing argument and not call the dbio2.ExecuteScalar() method when it is false. Which probably ends the usefulness of the destructor as well.
Dispose is just a method, like any other method. There are some conventions about things that it should/shouldn't do, but there's nothing from the system's perspective that is wrong with creating objects in a Dispose call.
Making a DB call is a bit concerning to be personally; I wouldn't expect such an expensive and error prone activity to be called in a Dispose method, but that's more of a convention/expectation. The system won't have a problem with that.
Yes, but I would not do so unless the object created is within the local scope of the method. IDisposable is an advertisement that this class has some resource (often an unmanaged resource) that should be freed when the object is no longer used. If your Dispose is being called by your finializer (ie you are not calling the destructor directly, but waiting for the GC to do it) it can be an indication that you should be calling it earlier. You never know when the C# destructor will run, so you may be unnecessarily tying up that resource. It could also be in indication that your class doesn't need to implement IDisposable.
In your case your are using object dbio2 which I assume represents your DB connection. However, since this is called from the destructor how do you know if your connection is still valid? You destructor could an hour after your connection has been lost. You should try to ensure that this Dispose is called while you know the dbio2 object is still in scope.

Returning disposable object from within using block

I'm noticing a lot of byte[] getting stuck in memory when profiling my program. I did some digging and found the majority of the instances created in some manner like this:
public byte[] CreateBytes(byte[] bytes)
{
using (var start = new MemoryStream())
{
using (var memStr = new MemoryStream(bytes))
{
//do stuff
return start.ToArray();
}
}
}
The returned byte[] is then passed to other methods and is used in creating another MemoryStream from within another using block:
using (var uncompressedStream = new MemoryStream(uncompressedData))
{
using (var compressedStream = new MemoryStream())
{
//Do some compression
}
}
myObject.Bytes = uncompressedData;
uncompressedData = null;
return myObject;
(uncompressedData is the value returned from CreateBytes()).
My question is, when does the byte[] get cleaned up? Do I specifically need to set it to null, and if so, where? After the second using block I no longer need it, but if I simply put uncompressedData = null; I'm not sure that's going to reclaim the memory.
I would've thought that the using statement in CreateBytes(byte[] bytes) would've disposed of the bytes, but since it's returning a reference does that postpone and/or forego the disposal?
EDIT: I added another line of code. Since I'm storing the uncompressedBtyes in another object, setting uncompressedData to null is pointless and the byte[] will live as long as myObject (or until myObject.Bytes is set to null), correct?
The byte[] gets cleaned up when two conditions are both satisfied:
There are no references to that memory block
The garbage collector decides to collect that memory block
The GC runs at non-deterministic times based on a variety of factors.
The byte array, just like any other managed object in memory, is eligible for garbage collection as soon as it is no longer accessible from any root reference. You can trust the GC to know when that's happened, and to intelligently schedule times to actually do the clean up of eligible objects. Try not to worry about when an object that's eligible for cleanup is actually cleaned up, chances are the GC knows better than you.
In addition to Erics answer:
When you are sure you do not more need that byte[], assign to it null.
This will not guarantee that memory will be reclaimed now, neither that it will be reclimed after, this is just a way to help GC to identify it like a subject for collection.
The using statement really has nothing to do with the byte array you are returning.
The MemoryStream is disposable, and disposal of that object is what the using statement is managing (note that disposal is not the same as garbage collection, but is typically a precursor thereto).
When you call ToArray(), you are creating something new that is not disposable; once it is out of scope, it will be cleaned up by the garbage collector at an indeterminate time (this is typically a well optimized process).

For Microsoft built classes that inherit IDisposable, do I explicitly have to call Dispose?

Regarding the Microsoft built classes that inherit IDisposable, do I explicitly have to call Dispose to prevent memory leaks?
I understand that it is best practice to call Dispose (or better yet use a using block), however when programming, typically I don't always immediately realise that a class inherits from IDisposable.
I also understand that Microsoft implementation of IDisposable is a bit borked, which is why they created the article explaining the correct usage of IDisposable.
Long story short, in which instances is it okay to forget to call Dispose?
There are a couple of issues in the primary question
Do I explicitly have to call Dispose to prevent memory leaks?
Calling Dispose on any type which implements IDisposable is highly recomended and may even be a fundamental part of the types contract. There is almost no good reason to not call Dispose when you are done with the object. An IDisposable object is meant to be disposed.
But will failing to call Dispose create a memory leak? Possibly. It's very dependent on what exactly that object does in it's Dispose method. Many free memory, some unhook from events, others free handles, etc ... It may not leak memory but it will almost certainly have a negative effect on your program
In which instances is it okay to forget to call Dispose?
I'd start with none. The vast majority of objects out there implement IDisposable for good reason. Failing to call Dispose will hurt your program.
It depends on two things:
What happens in the Dispose method
Does the finalizer call Dispose
Dispose functionlity
Dispose can do several type of actions, like closing a handle to a resource (like file stream), change the class state and release other components the class itself uses.
In case of resource being released (like file) there's a functionality difference between calling it explicitly and waiting for it to be called during garbage collection (assuming the finalizer calls dispose).
In case there's no state change and only components are released there'll be no memory leak since the object will be freed by the GC later.
Finalizer
In most cases, disposable types call the Dispose method from the finalizer. If this is the case, and assuming the context in which the dispose is called doesn't matter, then there's a high chance that you'll notice no difference if the object will not be disposed explicitly. But, if the Dispose is not called from the finalizer then your code will behave differently.
Bottom line - in most cases, it's better to dispose the object explicitly when you're done with it.
A simple example to where it's better to call Dispose explicitly: Assuming you're using a FileStream to write some content and enable no sharing, then the file is locked by the process until the GC will get the object. The file may also not flush all the content to the file so if the process crashes in some point after the write was over it's not guaranteed that it will actually be saved.
It can be safe to not call Dispose, but the problem is knowing when this is the case.
A good 95% of IEnumerator<T> implementations have a Dispose that's safe to ignore, but the 5% is not just 5% that'll cause a bug, but 5% that'll cause a nasty hard to trace bug. More to the point, code that gets passed an IEnumerator<T> will see both the 95% and the 5% and won't be able to dynamically tell them apart (it's possible to implement the non-generic IEnumerable without implementing IDisposable, and how well that turned out can be guessed at by MS deciding to make IEnumerator<T> inherit from IDisposable!).
Of the rest, maybe there's 3 or 4% of the time it's safe. For now. You don't know which 3% without looking at the code, and even then the contract says you have to call it, so the developer can depend on you doing so if they release a new version where it is important.
In summary, always call Dispose(). (I can think of an exception, but it's frankly too weird to even go into the details of, and it's still safe to call it in that case, just not vital).
On the question of implementing IDisposable yourself, avoid the pattern in that accursed document.
I consider that pattern an anti-pattern. It is a good pattern for implementing both IDisposable.Dispose and a finaliser in a class that holds both managed and unmanaged resources. However holding both managed IDisposable and unmanaged resources is a bad idea in the first place.
Instead:
If you have an unmanaged resource, then don't have any unmanaged resources that implement IDisposable. Now the Dispose(true) and Dispose(false) code paths are the same, so really they can become:
public class HasUnmanaged : IDisposable
{
IntPtr unmanagedGoo;
private void CleanUp()
{
if(unmanagedGoo != IntPtr.Zero)
{
SomeReleasingMethod(unmanagedGoo);
unmanagedGoo = IntPtr.Zero;
}
}
public void Dispose()
{
CleanUp();
GC.SuppressFinalize(this);
}
~HasUnmanaged()
{
CleanUp();
}
}
If you have managed resources that need to be disposed, then just do that:
public class HasUnmanaged : IDisposable
{
IDisposable managedGoo;
public void Dispose()
{
if(managedGoo != null)
managedGoo.Dispose();
}
}
There, no cryptic "disposing" bool (how can something be called Dispose and take false for something called disposing?) No worrying about finalisers for the 99.99% of the time you won't need them (the second pattern is way more common than the first). All good.
Really need something that has both a managed and an unmanaged resource? No, you don't really, wrap the unmanaged resource in a class of your own that works as a handle to it, and then that handle fits the first pattern above and the main class fits the second.
Only implement the CA10634 pattern when you're forced to because you inherited from a class that did so. Thankfully, most people aren't creating new ones like that any more.
It is never OK to forget to call Dispose (or, as you say, better yet use using).
I guess if the goal of your program is to cause unmanaged resource leaks. Then maybe it would be OK.
The implementation of IDisposable indicates that a class uses un-managed resources. You should always call Dispose() (or use a using block when possible) when you're sure you're done with the class. Otherwise you are unnecessarily keeping un-managed resources allocated.
In other words, never forget to call Dispose().
Yes, always call dispose. Either explicitly or implicitly (via using). Take, for example, the Timer class. If you do not explicitly stop a timer, and do not dispose it, then it will keep firing until the garbage collector gets around to collecting it. This could actually cause crashes or unexpected behavior.
It's always best to make sure Dispose is called as soon as you are done with it.
Microsoft (probably not officially) says it is ok to not call Dispose in some cases.
Stephen Toub from Microsoft writes (about calling Dispose on Task):
In short, as is typically the case in .NET, dispose aggressively if
it's easy and correct to do based on the structure of your code. If
you start having to do strange gyrations in order to Dispose (or in
the case of Tasks, use additional synchronization to ensure it's safe
to dispose, since Dispose may only be used once a task has completed),
it's likely better to rely on finalization to take care of things. In
the end, it's best to measure, measure, measure to see if you actually
have a problem before you go out of your way to make the code less
sightly in order to implement clean-up functionality.
[bold emphasize is mine]
Another case is base streams
var inner = new FileStrem(...);
var outer = new StreamReader(inner, Encoding.GetEncoding(1252));
...
outer.Dispose();
inner.Dispose(); -- this will trigger a FxCop performance warning about calling Dispose twice.
(I have turned off this rule)

Should a .Net/C# object call Dispose() on itself?

Below is some sample code written by a colleague. This seems obviously wrong to me but I wanted to check. Should an object call its own Dispose() method from within one of its own methods? It seems to me that only the owner/creator of the object should call Dispose() when it's done with the object and not the object itself.
It's an .asmx web method that calls Dispose() on itself when it's done. (The fact that it's a web method is probably incidental to the question in general.) In our code base we sometimes instantiate web service classes within methods of other web services and then call methods on them. If my code does that to call this method, the object is toast when the method returns and I can't really use the object any more.
[WebMethod]
public string MyWebMethod()
{
try
{
return doSomething();
}
catch(Exception exception)
{
return string.Empty;
}
finally
{
Dispose(true);
}
}
UPDATE:
Found a few links that are related:
Do I need to dispose a web service reference in ASP.NET?
Dispose a Web Service Proxy class?
For sure it's not a good prartice. The caller should decide when he is finished using the IDisposable object, not an object itself.
There are very few valid reasons to perform a "Self Disposing" action.
Threading is the one I use more than not.
I have several applications that "fire and forget" threads.
Using this methodology allow the object to self dispose.
This helps keep the environment clean without a threading manager process.
if I ever see that in one of my projects, I would ask why and I'm 99.9999% sure that i would remove it anyway
for me this is a kind of red flag / code smells
There are no technical restrictions on what a Dispose method is allowed to do. The only thing special about it is that Dispose gets called in certain constructs (foreach, using). Because of that, Dispose might reasonably be used to flag an object as no-longer-useable, especially if the call is idempotent.
I would not use it for this purpose however, because of the accepted semantics of Dispose. If I wanted to mark an object as no-longer-useable from within the class itself, then I would create a MarkUnuseable() method that could be called by Dispose or any other place.
By restricting the calls to Dispose to the commonly accepted patterns, you buy the ability to make changes to the Dispose methods in all of your classes with confidence that you will not unexpectedly break any code that deviates from the common pattern.
Just remove it, but take care to dispose it in all object that call it.
Technically yes, if that "method" is the finaliser and you are implementing the Finalise and IDisposable pattern as specified by Microsoft.
While a .Net object would not normally call Dispose on itself, there are times when code running within an object may be the last thing that expects to be using it. As a simple example, if a Dispose method can handle the cleanup of a partially-constructed object, it may be useful to have a constructor coded something like the following:
Sub New()
Dim OK As Boolean = False
Try
... do Stuff
OK = True
Finally
If Not OK Then Me.Dispose
End Try
End Sub
If the constructor is going to throw an exception without returning, then the partially-constructed object, which is slated for abandonment, will be the only thing that will ever have the information and impetus to do the necessary cleanup. If it doesn't take care of ensuring a timely Dispose, nothing else will.
With regard to your particular piece of code, the pattern is somewhat unusual, but it looks somewhat like the way a socket can get passed from one thread to another. There's a call which returns an array of bytes and invalidates a Socket; that array of bytes can be used in another thread to create a new Socket instance which takes over the communications stream established by the other Socket. Note that the data regarding the open socket is effectively an unmanaged resource, but it can't very well be wrapped in an object with a finalizer because it's often going to be handed off to something the garbage-collector can't see.
No! It is unexpected behavior and breaks the best practices guidelines. Never do anything that is unexpeceted. Your object should only do what is needed to maintain it's state while protecting the integrity of the object for the caller. The caller will decide when it's done (or the GC if nothing else).

Method returns an IDisposable - Should I dispose of the result, even if it's not assigned to anything?

This seems like a fairly straightforward question, but I couldn't find this particular use-case after some searching around.
Suppose I have a simple method that, say, determines if a file is opened by some process. I can do this (not 100% correctly, but fairly well) with this:
public bool IsOpen(string fileName)
{
try
{
File.Open(fileName, FileMode.Open, FileAccess.Read, FileShare.None);
}
catch
{
// if an exception is thrown, the file must be opened by some other process
return true;
}
}
(obviously this isn't the best or even correct way to determine this - File.Open throws a number of different exceptions, all with different meanings, but it works for this example)
Now the File.Open call returns a FileStream, and FileStream implements IDisposable. Normally we'd want to wrap the usage of any FileStream instantiations in a using block to make sure they're disposed of properly. But what happens in the case where we don't actually assign the return value to anything? Is it still necessary to dispose of the FileStream, like so:
try
{
using (File.Open(fileName, FileMode.Open, FileAccess.Read, FileShare.None));
{ /* nop */ }
}
catch
{
return true;
}
Should I create a FileStream instance and dispose of that?
try
{
using (FileStream fs = File.Open(fileName, FileMode.Open, FileAccess.Read, FileShare.None));
}
...
Or are these totally unnecessary? Can we simply call File.Open and not assign it to anything (first code example), and let the GC dispose of it right away?
Yes, you should definitely dispose of the FileStream. Otherwise the stream will remain open and the file won't be usable until a finalizer happens to clean it up.
The important thing here is ownership: for File.Open, the caller is assumed to "own" the stream returned to it - and if you own something which implements IDisposable, it's your responsibility to dispose of it.
Compare this with the situation of Image.FromStream: in that case, you pass in a stream and the Image then assumes that it owns that stream. You mustn't close the stream yourself, in that case - you have to dispose of the image when you're done, and it will dispose of the stream.
Calling a static method which returns something disposable almost always assumes that the caller takes ownership of the resource. Ditto constructors (which are effectively static methods.)
The irony in this case is that if you don't dispose of the stream returned by File.Open, you'll have found that the file is usable - at the same time as making it unusable until some indeterminate time.
If a method returns an IDisposable, I personally would always put it in a using block. Even if I don't assign the return value to anything.
Even if you don't assign it to a variable, the disposable object is still created. Dispose is not going to be called automatically. The only difference will be that the returned object will become immediately eligible for garbage collection, because there are no (strong) references to it.
The garbage collector does not call Dispose automatically when it reclaims an object. However, most IDisposable types provide a finalizer (which will be called just before the GC reclaims an object) that invokes Dispose as a fallback strategy (safety net) — study the IDisposable pattern to see how this is done:
~SomeClass // <-- the finalizer method will usually call Dispose;
{ // but you have no control over when it will be called!
Dispose(false);
}
Remember that you don't know when the garbage collector will run (because it's non-deterministic). Therefore, you also don't know when the finalizer method will be called. And because of that -- if you haven't called Dispose explicitly (either yourself, or with a using block) -- you don't know when it will be called by the finalizer.
That's the advantage of calling Dispose explicitly: You can free resources -- or at least allow the GC to free managed resources -- as soon as you're done with them, instead of holding on to resources until the finalizer gets called sometime in the future.
Yes, you don't want to leave the FileStream opened. For one, you won't even be able to open the file yourself after that. Calling Close() is good enough, but using using is probably the preferred pattern.
There's a much bigger problem with your code however. It cannot possibly work reliably on Windows. A typical scenario:
The File.Open() call succeeds. You close it
Your thread gets pre-empted by the Windows scheduler
Another thread in another process gets a chance to run, it opens the file
Your thread regains the CPU and continues after the File.Open() call
You open the file, trusting that it will work since IsOpen() returned false.
Kaboom.
Never write code like this, failure is extremely hard to diagnose. Only ever open a file when you are ready to start reading or writing to it. And don't close it until you are done.
Extra bonus: it is now obvious that you want to use a using statement.
When you call any method that returns something, an instance of that something is being created. Just because you're not actually capturing it, doesn't make it any less "there". Therefore, in the case of an IDisposable object, the object is being created by the method you're calling in spite of the fact that you're doing nothing with it. So yes, you still need to dispose of it somehow. The first approach with the using statement seems like it should work.

Categories

Resources