Uses of "using" in C# has a nice explanation of the utilities of the using feature.
.Net has its garbage collector. How does it handle the lack of a dipose()?
Specifically for DB connections, statements and resultsets, is it required a using() for each of them? What happens if they are left behind with no using(), dispose() and neither close()?
Update: the context is web applications, therefore there may be thousands of simultaneous users, each with his own connection/stmt/rs and the app will never be closed.
Since using is a shorthand for calling Dispose, you could imitate it with try/finally. So the real question is what's the consequence of not calling Dispose at all.
Although C# has garbage collection which would eventually release resources most of the time, you want the release of critical resources to happen as soon as you are done with them. If you use using or the equivalent try/finally, the resources would be released quickly. If you let the garbage collector to release the resources for you, your program may be starved of resources while they are "in custody" of GC (i.e. your program is no longer using them, but GC has not released them yet). Moreover, since GC offers no hard guarantee of running finalizers, some resources may not get released explicitly until your program ends, which may cause resource starvation of other processes.
You don't know when .net's garbage collector is called and run, so it's allowing you to do it yourself when you don't need it. So, when your code gets out of using() it's dispose object used in using() instead of waiting for GC to run on its own schedule.
If you don't use with DB connection, then GC will dispose it on its own way based on criteria of algorithm it's implementing. It might get too late(in terms of computer clock) to sweep it.
Garbage collector is a background thread which doesn't run every millisecond. It has specific schedule and its own algorithm which tends it to work on a specific time. E.g., some GC algorithms check for objects having no references then they sweep those objects when GC runs.
Specifically for DB connections, statements and resultsets, is it
required a using() for each of them? What happens if they are left
behind with no using(), dispose() and neither close()?
Actually the worst consequence of a memory leak is holding some memory until you restart the PC. However in this case probably the worst consequence is leaking memory until you restart the application.
If memory growth increases up to where GC cannot clean any longer, in fact if Gen 2 of Small Object Heap is overflow (Large object heap also can overflow), it will throw out of memory exception and close the application.
.Net has its garbage collector. How does it handle the lack of a
dipose()?
All the standard database connection related classes have implement Dispose and Finalize methods properly. Generally there are unmanaged resources in those classes. Unmanaged resources are the resouces (eg: file handlers, database connection handlers and etc) which could cause worse memory leaks that may hold memory until you restart the PC. However, that's where GC's finalization comes handy. If you don't call the Dispose for such Disposable object, garbage collector will execute the Finalize method(if there is a ~destructor) and clear unmanaged resources.
That's the reason why it is required to implement IDispose Pattern properly Dispose and Finalization as required. Finalization is required only if it has Unamanged resources.
The most likely consequence of failing to promptly Dispose database objects is that the program will ask a database server to open a database connections on its behalf with a promise that it will tell the server when they are no longer needed (i.e. close them), but may leave the connections open for quite awhile after they're no longer needed. Such behavior may increase the number of connections the database server will need to keep open simultaneously. Depending upon the server, there may be no consequences, or the extra connections may impair performance, or they may cause some connection requests to get needlessly denied.
Although .NET will try to ensure that database servers will get notified when database objects are abandoned, even if Dispose is not called, the code which uses database objects will generally know when it will no longer need them, long before .NET can determine that they're abandoned. Note also that while some .NET database-related libraries may keep connections open for a little while after a Dispose (so that if code needs the database again it can resume using the earlier connection) such libraries may use timers to limit how long connections are maintained in expectation of further use, rather than depend upon the garbage-collector (which might go a very long time without noticing that an object has been abandoned).
Related
With resource leaks I mean Streams, StreamWriter (I suppose they are consuming File descriptors), Handles (GDI or user also Graphics fonts). Shortly all Closable objects can count as a resource!
If there are some resource leaks in the application. Say some InputStreams are not getting closed, are they potential memory leaks too because Garbage Collector is not going to remove them from memory?
Another question: Are resource leaks affecting the performance?
If there are some resource leaks in the application. Say some InputStreams are not getting closed, are they potential memory leaks too because Garbage Collector is not going to remove them from memory?
The GC will clean up resources whether they are closed or not. The only resources which cannot be cleaned up if not shutdown are threads.
When a Stream is discarded without closing, it is closed by the finalizer on a collection. This has two problems
exactly when or even if this happens in unpredictable meaning the file might not be flushed or a lock is retained on the file preventing it being deleted.
the finalizer is a single thread and closing a resources can take time. If you have enough of these the finalizer won't keep up and you will get an OutOfMemoryError because you have a large number of resources waiting to be cleaned.
It is always better to clean up resources when you have finished with them.
Another question: Are resource leaks affecting the performance?
They can, it depends on how much is leaked. If you don't have much idea you have to assume it a problem unless you are confident it is not. e.g. a program could leak one 24 bytes object once per day, or it could leak 100 MB per seconds. Not all leaks are the same.
It depends on what you call performance. I'll assume you're speaking of overall performance, meaning that memory consumption, speed, and the like are all important.
It depends, too, on the resource used. Some resources (e.g. file handles) are recovered when the process exits, so the leak will only be a problem when executing. Others (like server or database connections) could remain leaking even after your application execution. Others (like mutexes, etc.) should be released as soon as possible.
Now, the consequences depend on the resource. If the resource is a native object in the same process, then leaking it will probably leak the associated memory. If the resource is a mutex you locked but failed to unlock, then you are probably about to deadlock your application. If the resource is a connection, the server will keep that connection open even after you stopped using it. If the resource is a file, it could stop other applications (or even your own application) to access it again.
In the end, while some resources could be leaked, other shouldn't. As far as I am concerned, no resource should be leaked, ever, but YMMV.
So you should make an habit of always correctly releasing the resources you acquired (memory, files, connections, mutexes, etc.), no matter the perceived importance of that resource. Doing so will train you in the right coding patterns (and mindset).
RAII and Exception Safety are the keyword you're searching for if you want to explore the notions.
For C#, the Dispose pattern (IDisposable interface, and the finalizer) and the using keyword will be needed. Another solution is to use the finally class of a try/finally to free your resource, but this is difficult to maintain correctly.
In Java, you'll need Java 7 (IIRC), and use the AutoCloseable interface and the "try-with-resources" statement. As in C#, you can use the finally class of a try/finally to free your resource, with the same problems.
Memory leak, by definition is a memory that is useless, but still allocated in your proess space. Considering that CLR process on 32bit machine has approximately 1.2GB of possible memory, I would say it's extremely dangerous to have a memory leaks in your application.
Naturally, everything depends on how big, mission critical + other factors yuor application is. But, in any case, always try to avoid them, especially if you already know that they are exist and especially if you already know where they are.
EDIT
The resource leaks are the same story actually. Resource allocates the memory, so the leak of it creates a memory leak, by definition.
Hope this helps.
Yes, memory leaks means that the app needs to run the Garbage collector more often and is able yto recover less memory on each run. When the memory is exausted, the application will crash.
Files not getting closed will lead to a the app being unable to do anything related to files or sockets when the maximum number of open files is reached, which usually makes the application unusable.
Leak can occur when you keep a rooted reference to an unused object. The GC is unable to collect it as it is still reachable. Pay special attention to your static instances and event handlers attached to static events.
When you leave a disposable object undiposed, in most case it will defer the release of unmanaged resources and may cause bug (Stream not flushed, ...). When releasing memory, the garbage collector calls a Finalizer on objects containing unmanaged resources. This is more expensive than a staight call to Dispose that will lessen GC's job. It is unpredictable, the finalizer may be called really lately. So if you do not call Dispose it can lead to temporary resource starvation (no remaining file handle, ...).
So there is no memory leak for not calling Dispose on this Stream but you shouldn't rely on Finalization as it is costly and unpredictable. Moreover Dispose can do more than releasing resources, it can also properly clear a managed object (flush buffered stream, ...).
Do I have to use a using statement to dispose immediately even in a method? Or will the ending of the method cause an automatic Dispose of all local variables including graphics?
(I’m asking this because I’ve seen examples that have Dispose calls at the end of methods, and wanted to know if that’s really necessary.)
Thanks.
Yes, you do. Going out of scope does not do anything; it does not call Dispose(), it does not garbage-collect, it does not call a finalizer.
If the type is IDisposable, then yes. It is your job to tidy up after yourself (assuming that the object is actually "done with" at this point).
Possible side-effects of not doing this:
files get left open and cause access-exceptions (FileStream)
connections get left open and cause pool saturation (DbConnection)
unmanaged handles get saturated and cause resource starvation (any winforms/etc)
transactions get left open and cause blocking (TransactionScope / DbTransaction)
etc
basically, bad things.
Furthermore, in most cases where you see Dispose() at the bottom of the method, a using would be preferable. There are some cases where that isn't possible (fields on an object, for example), but the point remains: it sounds like those are bad examples.
You don't ever have to dispose of an object. But if you want to release unmanaged resources earlier rather than whenever the GC gets around to it (which is when it processes the fReachable queue) then yes you should since the scope of of a method does not determine when a disposable object's finalize will be called.
For more you might want to read Jeffrey Richter's Garbage Collection: Automatic Memory Management in the Microsoft .NET Framework
As delnan noted if there's a hard limit to the resources you're using you're much more likely to impacted if you let it get reclaimed by the GC rather than calling Dispose. DB connections are a good example of this kind of reasource and is certainly the reason why the NumberOfReclaimedConnections performance counter got created.
Yes, use using. It is a great statement.
Actually if you don't call Dispose (either manually or by using using), you can end up in a situation where your managed memory is stil quite empty, but unmanaged resources are depleted. Eventually it can cause a deadlock because other objects or other processes will wait for your unmanaged resources and you will wait for them. (Garbage collector won't run, because managed memory still won't be full.)
So please call Dispose as soon as you can. Once you don't need an object which has got that method defined, call it.
From what I can tell, it is an accepted rule that if you have a class A that has a member m that is IDisposable, A should implement IDisposable and it should call m.Dispose() inside of it.
I can't find a satisfying reason why this is the case.
I understand the rule that if you have unmanaged resources, you should provide a finalizer along with IDisposable so that if the user doesn't explicitly call Dispose, the finalizer will still clean up during GC.
However, with that rule in place, it seems like you shouldn't need to have the rule that this question is about. For instance...
If I have a class:
class MyImage{
private Image _img;
... }
Conventions states that I should have MyImage : IDisposable. But if Image has followed conventions and implemented a finalizer and I don't care about the timely release of resources, what's the point?
UPDATE
Found a good discussion on what I was trying to get at here.
But if Image has followed conventions and implemented a finalizer and I don't care about the timely release of resources, what's the point?
You've missed the point of Dispose entirely. It's not about your convenience. It's about the convenience of other components that might want to use those unmanaged resources. Unless you can guarantee that no other code in the system cares about the timely release of resources, and the user doesn't care about timely release of resources, you should release your resources as soon as possible. That's the polite thing to do.
In the classic Prisoner's Dilemma, a lone defector in a world of cooperators gains a huge benefit. But in your case, being a lone defector produces only the tiny benefit of you personally saving a few minutes by writing low-quality, best-practice-ignoring code. It's your users and all the programs they use that suffer, and you gain practically nothing. Your code takes advantage of the fact that other programs unlock files and release mutexes and all that stuff. Be a good citizen and do the same for them. It's not hard to do, and it makes the whole software ecosystem better.
UPDATE: Here is an example of a real-world situation that my team is dealing with right now.
We have a test utility. It has a "handle leak" in that a bunch of unmanaged resources aren't aggressively disposed; it's leaking maybe half a dozen handles per "task". It maintains a list of "tasks to do" when it discovers disabled tests, and so on. We have ten or twenty thousand tasks in this list, so we very quickly end up with so many outstanding handles -- handles that should be dead and released back into the operating system -- that soon none of the code in the system that is not related to testing can run. The test code doesn't care. It works just fine. But eventually the code being tested can't make message boxes or other UI and the entire system either hangs or crashes.
The garbage collector has no reason to know that it needs to run finalizers more aggressively to release those handles sooner; why should it? Its job is to manage memory. Your job is to manage handles, so you've got to do that job.
But if Image has followed conventions
and implemented a finalizer and I
don't care about the timely release of
resources, what's the point?
Then there isn't one, if you don't care about timely release, and you can ensure that the disposable object is written correct (in truth I never make an assumption like that, not even with MSs code. You never know when something accidentally slipped by). The point is that you should care, as you never know when it will cause a problem. Think about an open database connection. Leaving it hanging around, means that it isn't replaced in the pool. You can run out if you have several requests come in for one.
Nothing says you have to do it if you don't care. Think of it this way, it's like releasing variables in an unmanaged program. You don't have to, but it is highly advisable. If for no other reason the person inheriting from the program doesn't have to wonder why it wasn't taken care of and then try and clear it up.
Firstly, there's no guaranteeing when an object will be cleaned up by the finalizer thread - think about the case where a class has a reference to a sql connection. Unless you make sure this is disposed of promptly, you'll have a connection open for an unknown period of time - and you won't be able to reuse it.
Secondly, finalization is not a cheap process - you should be making sure that if your objects are disposed of properly you're calling GC.SuppressFinalize(this) to prevent finalization happening.
Expanding on the "not cheap" aspect, the finalizer thread is a high-priority thread. It will take resources away from your main application if you give it too much to do.
Edit: Ok, here's a blog article by Chris Brummie about Finalization, including why it is expensive. (I knew I'd read loads about this somewhere)
If you don't care about the timely release of resources, then indeed there is no point. If you can be sure that the code is only for your consumption and you've got plenty of free memory/resources why not let GC hoover it up when it chooses to. OTOH, if someone else is using your code and creating many instances of (e.g.) MyImage, it's going to be pretty difficult to control memory/resource usage unless it disposes nicely.
Many classes require that Dispose be called to ensure correctness. If some C# code uses an iterator with a "finally" block, for example, the code in that block will not run if an enumerator is created with that iterator and not disposed. While there a few cases where it would be impractical to ensure objects were cleaned up without finalizers, for the most part code which relies upon finalizers for correct operation or to avoid memory leaks is bad code.
If your code acquires ownership of an IDisposable object, then unless either the object's cleass is sealed or your code creates the object by calling a constructor (as opposed to a factory method) you have no way of knowing what the real type of the object is, and whether it can be safely abandoned. Microsoft may have originally intended that it should be safe to abandon any type of object, but that is unrealistic, and the belief that it should be safe to abandon any type of object is unhelpful. If an object subscribes to events, allowing for safe abandonment will require either adding a level of weak indirection to all events, or a level of (non-weak) indirection to all other accesses. In many cases, it's better to require that a caller Dispose an object correctly than to add significant overhead and complexity to allow for abandonment.
Note also, btw, that even when objects try to accommodate abandonment it can still be very expensive. Create a Microsoft.VisualBasic.Collection (or whatever it's called), add a few objects, and create and Dispose a million enumerators. No problem--executes very quickly. Now create and abandon a million enumeartors. Major snooze fest unless you force a GC every few thousand enumerators. The Collection object is written to allow for abandonment, but that doesn't mean it doesn't have a major cost.
If an object you're using implements IDisposable, it's telling you it has something important to do when you're finished with it. That important thing may be to release unmanaged resources, or unhook from events so that it doesn't handle events after you think you're done with it, etc, etc. By not calling the Dispose, you're saying that you know better about how that object operates than the original author. In some tiny edge cases, this may actually be true, if you authored the IDisposable class yourself, or you know of a bug or performance problem related to calling Dispose. In general, it's very unlikely that ignoring a class requesting you to dispose it when you're done is a good idea.
Talking about finalizers - as has been pointed out, they have a cost, which can be avoided by Disposing the object (if it uses SuppressFinalize). Not just the cost of running the finalizer itself, and not just the cost of having to wait till that finalizer is done before the GC can collect the object. An object with a finalizer survives the collection in which it is identified as being unused and needing finalization. So it will be promoted (if it's not already in gen 2). This has several knock on effects:
The next higher generation will be collected less frequently, so after the finalizer runs, you may be waiting a long time before the GC comes around to that generation and sweeps your object away. So it can take a lot longer to free memory.
This adds unnecessary pressure to the collection the object is promoted to. If it's promoted from gen 0 to gen 1, then now gen 1 will fill up earlier than it needs to.
This can lead to more frequent garbage collections at higher generations, which is another performance hit.
If the object's finalizer isn't completed by the time the GC comes around to the higher generation, the object can be promoted again. Hence in a bad case you can cause an object to be promoted from gen 0 to gen 2 without good reason.
Obviously if you're only doing this on one object it's not likely to cost you anything noticeable. If you're doing it as general practice because you find calling Dispose on objects you're using tiresome, then it can lead to all of the problems above.
Dispose is like a lock on a front door. It's probably there for a reason, and if you're leaving the building, you should probably lock the door. If it wasn't a good idea to lock it, there wouldn't be a lock.
Even if you don't care in this particular case, you should still follow the standard because you will care in some cases. It's much easier to set a standard and follow it always based on specific guidelines than have a standard that you sometimes disregard. This is especially true as your team grows and your product ages.
I am comfortable with how to implement IDisposable..
However I'm not clear as to the exact consequences of not implementing this properly. Is it a case that unmanaged resources will never be cleared thus causing memory leaks or will the memory eventually get recycled anyway, just not in a timely manner?
It depends.
If the object in question fails to implement IDisposable, but does still implement a finalizer correctly, the resources will eventually get cleaned up.
However, if it doesn't properly implement a finalizer, AND it's wrapping an unmanaged resource, that resource will be leaked. This is more often a resource leak (ie: leaking a HANDLE stored as IntPtr) more than a traditional "memory leak" (though, of course, the object being pointed to does still use memory).
will the memory eventually get recycled anyway?
One point here... IDisposable has nothing to do with memory - it's about resources. These resources often use memory of their own, but implementing IDisposable does not cause your garbage collector to reclaim memory more quickly.
It depends entirely on what te implementation is, of course.
An incorrect dispose on a SqlTransaction could lead to excessive long blocking, for example - impacting multiple machines and users. An incorrect dispose on a SqlConnection could cause connection saturation and inability to connect. These are not just memory (etc) issues.
IIRC a missing graphic (pen/brush/whatever) dispose was responsible for a VS crash bug - GC didn't happen because there was no memory pressure, so instead it simply saturated the GDI handles and broke.
An incorrect file/stream dispose could cause exceptions due to a file bring unavailable for reading/writing - potentially losing data.
With unmanaged resources not properly handled, anything is possible.
Instead of thinking in terms of some vaguely-defined "resources", think of IDisposable as meaning "This object manipulates something outside itself (and possibly even outside the computer!) in a way that needs to be cleaned up, while the information necessary for such cleanup still exists." A finalizer which fires essentially says "Nobody else is using this stuff anymore, and I'm the only one who knows about it; I'd better clean it up because nobody else is going to."
It is dangerous to assume that finalizers will magically take care of things; while many objects can be safely abandoned, many others cannot. In some cases, it will be easy for an object to detect that it has been abandoned and clean up appropriately. In some other cases, it may be so difficult as to be impractical. Consider an object which is supposed to count how many times the word "quack" appears in a long-lived data stream. If the creator of the object has forgotten about it, and nobody is every going to query the count, the object may as well go away. Unfortunately, since the data stream holds a reference to the counting object (so as to let it know when data comes in), the counter object won't go away. If code creates a counter, then forgets about it, then creates another one, forgets about it, etc. this could become an unbounded memory leak.
A class that holds unmanaged resources should ensure that they are cleaned up during finalization. However, this only happens when the garbage collector gets around to it.
Not calling Dispose means waiting for the garbage collector to need memory before the resources are released.
The latter of the two. Just not in a timely manner.
The importance of this depends on the resources you are allocating. This is why you are more likely to see IDisposable interfaces on objects that open files and reference other, more crticital, resources.
Yes, unmanaged resources will never be released. (This assumes that the IDisposable types in question don't implement a "fallback" finaliser. If they do then the finaliser should sort things out... eventually.)
However, it's not just unmanaged memory allocations that you need to worry about; it's any unmanaged resources. For example, if you don't properly dispose of your database connections then you might find that the connection pool quickly gets exhausted.
Another potential problem with not implementing the disposable pattern properly - it is very easy to accidentally call .Dispose() more than once. If your Dispose method does not check for successive calls to it, and (eg.) attempts to free up an unmanaged resource more than once, you might end up with some unrecoverable error, or unexpected behaviour.
SO I already know about memory management in objective C, and I never had to know about it while programming in .net (C#). But i still have some questions about how everything is done.
-Why does the code leak in objective c if we allocate an object and not release it?
-Why doesn't this leak in C#?
-What are some advantages and disadvantages of automatic-garbage-collecting?
-Why not use autorelease on every allocated object (Objective C)?
-Is it possible to take care of the memory manually (C#)? so let's say i instantiate an object, and when I'm done i want to release it, and i don't want to wait for the garbage collector to do it?
It leaks in Objective-C because Objective-C doesn’t take any action on it. It relies on you doing all the work. It doesn’t leak in C# (more precisely, in .NET) because it employs a garbage collector which cleans up objects that are no longer used.
The main advantage of garbage collection is the above: you have far fewer memory leaks. (It’s still possible to have a memory leak, e.g. by filling a list indefinitely, but that’s harder to do accidentally.) It used to be thought that garbage collection has a disadvantage in that it could slow down the program because it keeps doing the garbage collection in the background and you have little control over it. In reality, however, the difference is negligible: there are other background tasks on your computer (e.g. device drivers) running all the time, the garbage collector doesn’t break the camel’s back either.
Auto-deallocation (as it is employed in C++ when a non-pointer variable goes out of scope) is dangerous because it opens the possibility to have a reference to it still in existence even after the object has been disposed. If your code then tries to access the object, the process goes kaboom big time.
Yes, it is possible to tell C# to release memory by invoking the garbage collector directly (GC.Collect()). However, I have yet to see a case where this is at all necessary. If you actually run out of memory, the garbage collector will already kick in automatically and free as much as it can.
Objective-C isn't a garbage-collected language, so it has no way of knowing that an object won't be used anymore unless you tell it. That's the purpose of the .NET garbage collector: it checks to see which objects can no longer be used by the program, and- at some point- gets rid of them to free up memory. There are no guarantees as to when, or if, it will ever free any given abandoned object; it's just trying to keep memory usage from going out of control.
C# can't release an object without the garbage collector. If you release an object that's still being referenced, your program is going to crash when you try to use that. Which is always the risk of manual memory management, but like all "memory-managed languages", it is trying to prevent you from making exactly that mistake. If you want to explicitly shut down an object's operation, implement the interface IDisposable for that object's type, and use the Dispose() method on that object- essentially a destructor. Be sure you're done with it, of course, and that the object will behave correctly (by throwing exceptions) if something tries to use it after it's been Dispose()d of.
Objective-C is reference-counted. When an object is out of references, it deletes itself. It's not a bad solution to the "is someone still using this object?" problem, except for data structures that refer to themselves; circular data structures will hang around forever unless carefully handled. .NET isn't a reference counter, so it will get rid of circular data structures that can't be reached from running code.
Autorelease is just a "release later", for returning a value that should self-destruct if the code that grabs it doesn't immediately want to hold onto it, as far as I understand. (I'm not an Objective-C programmer, though.) It gets around the "who releases this object?" problem for calls that return an object, without destroying it before the function is finished. It's a special use case, though, and it doesn't make sense in most other cases.
The advantage of automatic garbage collection is that you do not have to explicitly free/release your objects as you said. The disadvantage is you cannot be sure when (or even if) any given object instance will be released.
C# has other mechanisms to release other resources like files, or db connections that have to be released detirminitely. For example, using allows you to make sure that IDispose is called on an object for sure.
Garbage collected systems tend to have more memory in use at any given time than a well tuned manual implementation. Of course, you do not have memory leaks.
The quality and performance of garbage collectors can vary quite a bit which is something you may not have a lot of control over. For example, there may be a noticable lag when the GC runs. In .NET, you can call GC.Collect() to tell the GC that now would be a good time but it may or may not listen.
These days Objective-C is also garbage collected on the Mac. On the iPhone it is reference counted though so I assume that is where you are running into the issue.
On garbage collected systems, you can still run into the issue where a an object hangs onto a reference to an object that you would expect to be garbage collected. The garbage collector will not clean-up this up and so the memory is essentially leaked.
In reference counted systems, you do not have to keep track of every object that points at your object instances but you do have to specify that you want them released.
EDIT: I guess I did not say this explicitly - you cannot manually control memory allocation in C#.