With resource leaks I mean Streams, StreamWriter (I suppose they are consuming File descriptors), Handles (GDI or user also Graphics fonts). Shortly all Closable objects can count as a resource!
If there are some resource leaks in the application. Say some InputStreams are not getting closed, are they potential memory leaks too because Garbage Collector is not going to remove them from memory?
Another question: Are resource leaks affecting the performance?
If there are some resource leaks in the application. Say some InputStreams are not getting closed, are they potential memory leaks too because Garbage Collector is not going to remove them from memory?
The GC will clean up resources whether they are closed or not. The only resources which cannot be cleaned up if not shutdown are threads.
When a Stream is discarded without closing, it is closed by the finalizer on a collection. This has two problems
exactly when or even if this happens in unpredictable meaning the file might not be flushed or a lock is retained on the file preventing it being deleted.
the finalizer is a single thread and closing a resources can take time. If you have enough of these the finalizer won't keep up and you will get an OutOfMemoryError because you have a large number of resources waiting to be cleaned.
It is always better to clean up resources when you have finished with them.
Another question: Are resource leaks affecting the performance?
They can, it depends on how much is leaked. If you don't have much idea you have to assume it a problem unless you are confident it is not. e.g. a program could leak one 24 bytes object once per day, or it could leak 100 MB per seconds. Not all leaks are the same.
It depends on what you call performance. I'll assume you're speaking of overall performance, meaning that memory consumption, speed, and the like are all important.
It depends, too, on the resource used. Some resources (e.g. file handles) are recovered when the process exits, so the leak will only be a problem when executing. Others (like server or database connections) could remain leaking even after your application execution. Others (like mutexes, etc.) should be released as soon as possible.
Now, the consequences depend on the resource. If the resource is a native object in the same process, then leaking it will probably leak the associated memory. If the resource is a mutex you locked but failed to unlock, then you are probably about to deadlock your application. If the resource is a connection, the server will keep that connection open even after you stopped using it. If the resource is a file, it could stop other applications (or even your own application) to access it again.
In the end, while some resources could be leaked, other shouldn't. As far as I am concerned, no resource should be leaked, ever, but YMMV.
So you should make an habit of always correctly releasing the resources you acquired (memory, files, connections, mutexes, etc.), no matter the perceived importance of that resource. Doing so will train you in the right coding patterns (and mindset).
RAII and Exception Safety are the keyword you're searching for if you want to explore the notions.
For C#, the Dispose pattern (IDisposable interface, and the finalizer) and the using keyword will be needed. Another solution is to use the finally class of a try/finally to free your resource, but this is difficult to maintain correctly.
In Java, you'll need Java 7 (IIRC), and use the AutoCloseable interface and the "try-with-resources" statement. As in C#, you can use the finally class of a try/finally to free your resource, with the same problems.
Memory leak, by definition is a memory that is useless, but still allocated in your proess space. Considering that CLR process on 32bit machine has approximately 1.2GB of possible memory, I would say it's extremely dangerous to have a memory leaks in your application.
Naturally, everything depends on how big, mission critical + other factors yuor application is. But, in any case, always try to avoid them, especially if you already know that they are exist and especially if you already know where they are.
EDIT
The resource leaks are the same story actually. Resource allocates the memory, so the leak of it creates a memory leak, by definition.
Hope this helps.
Yes, memory leaks means that the app needs to run the Garbage collector more often and is able yto recover less memory on each run. When the memory is exausted, the application will crash.
Files not getting closed will lead to a the app being unable to do anything related to files or sockets when the maximum number of open files is reached, which usually makes the application unusable.
Leak can occur when you keep a rooted reference to an unused object. The GC is unable to collect it as it is still reachable. Pay special attention to your static instances and event handlers attached to static events.
When you leave a disposable object undiposed, in most case it will defer the release of unmanaged resources and may cause bug (Stream not flushed, ...). When releasing memory, the garbage collector calls a Finalizer on objects containing unmanaged resources. This is more expensive than a staight call to Dispose that will lessen GC's job. It is unpredictable, the finalizer may be called really lately. So if you do not call Dispose it can lead to temporary resource starvation (no remaining file handle, ...).
So there is no memory leak for not calling Dispose on this Stream but you shouldn't rely on Finalization as it is costly and unpredictable. Moreover Dispose can do more than releasing resources, it can also properly clear a managed object (flush buffered stream, ...).
Related
I've read many questions on this network in regards to when one should use GC.Collect(), but none address this concern.
If we somehow manage to use it, as such as it frees a large number of memory, is this necessarily a good thing, rather than simply relying on the Garbage Collector to manage the memory?
Regardless of how idle my ASP.NET application stays, or how much time passes, or operations are done, the memory that gets freed when explicitly calling GC.Collect() is never addressed (only memory on top of it is, like it's some kind of a cache.)
One theory I have in mind, is that perhaps the reason those 3GB+ are not freed (but rather the application frees memory above that threshold) might be due to reusing parts of it, or maybe some other helpful optimization, at the cost of, of course, using that memory throughout the run-time of the application.
Could anyone share some insight on these concerns? I'm following best practices and disposing objects whenever given the chance. No unmanaged resource is left to be finalized - it's always disposed by me.
EDIT: The Garbage Collector used is the workstation one, not the server.
The garbage collector's job is to simulate a machine with infinite memory. If your system has enough unallocated memory to continue functioning, why should the collector spend CPU cycles performing the hard work of identifying unreferenced objects? That's a performance cost with zero benefit.
If left alone, the collector will run and free those 3GB when the system starts to run low on memory. Until that point it is dormant.
I have windows form application in this I used Infragistics controls. I analyze the memory process after opening the form memory usage goes 27 MB but after closing it memory should release but it not working. While closing I called Dispose method also.
Some thoughts.
Don't use Task Manager to diagnose memory problems. Use a memory profiler. .Net manages its memory in a way which is not entirely obvious and Task Manager gives false assumptions about that nature of this.
Just because you dispose something, it doesn't mean the garbage collector is going to race to clean up and aggressively give memory back to the operating system, it will do it when it feels like it based on memory pressure and heuristics .
If you create a form (and its modal) you might as well just put it in a using statement, then you have peace-of-mind that you are doing everything correctly
Additional Resources
Fundamentals of Garbage Collection
Uses of "using" in C# has a nice explanation of the utilities of the using feature.
.Net has its garbage collector. How does it handle the lack of a dipose()?
Specifically for DB connections, statements and resultsets, is it required a using() for each of them? What happens if they are left behind with no using(), dispose() and neither close()?
Update: the context is web applications, therefore there may be thousands of simultaneous users, each with his own connection/stmt/rs and the app will never be closed.
Since using is a shorthand for calling Dispose, you could imitate it with try/finally. So the real question is what's the consequence of not calling Dispose at all.
Although C# has garbage collection which would eventually release resources most of the time, you want the release of critical resources to happen as soon as you are done with them. If you use using or the equivalent try/finally, the resources would be released quickly. If you let the garbage collector to release the resources for you, your program may be starved of resources while they are "in custody" of GC (i.e. your program is no longer using them, but GC has not released them yet). Moreover, since GC offers no hard guarantee of running finalizers, some resources may not get released explicitly until your program ends, which may cause resource starvation of other processes.
You don't know when .net's garbage collector is called and run, so it's allowing you to do it yourself when you don't need it. So, when your code gets out of using() it's dispose object used in using() instead of waiting for GC to run on its own schedule.
If you don't use with DB connection, then GC will dispose it on its own way based on criteria of algorithm it's implementing. It might get too late(in terms of computer clock) to sweep it.
Garbage collector is a background thread which doesn't run every millisecond. It has specific schedule and its own algorithm which tends it to work on a specific time. E.g., some GC algorithms check for objects having no references then they sweep those objects when GC runs.
Specifically for DB connections, statements and resultsets, is it
required a using() for each of them? What happens if they are left
behind with no using(), dispose() and neither close()?
Actually the worst consequence of a memory leak is holding some memory until you restart the PC. However in this case probably the worst consequence is leaking memory until you restart the application.
If memory growth increases up to where GC cannot clean any longer, in fact if Gen 2 of Small Object Heap is overflow (Large object heap also can overflow), it will throw out of memory exception and close the application.
.Net has its garbage collector. How does it handle the lack of a
dipose()?
All the standard database connection related classes have implement Dispose and Finalize methods properly. Generally there are unmanaged resources in those classes. Unmanaged resources are the resouces (eg: file handlers, database connection handlers and etc) which could cause worse memory leaks that may hold memory until you restart the PC. However, that's where GC's finalization comes handy. If you don't call the Dispose for such Disposable object, garbage collector will execute the Finalize method(if there is a ~destructor) and clear unmanaged resources.
That's the reason why it is required to implement IDispose Pattern properly Dispose and Finalization as required. Finalization is required only if it has Unamanged resources.
The most likely consequence of failing to promptly Dispose database objects is that the program will ask a database server to open a database connections on its behalf with a promise that it will tell the server when they are no longer needed (i.e. close them), but may leave the connections open for quite awhile after they're no longer needed. Such behavior may increase the number of connections the database server will need to keep open simultaneously. Depending upon the server, there may be no consequences, or the extra connections may impair performance, or they may cause some connection requests to get needlessly denied.
Although .NET will try to ensure that database servers will get notified when database objects are abandoned, even if Dispose is not called, the code which uses database objects will generally know when it will no longer need them, long before .NET can determine that they're abandoned. Note also that while some .NET database-related libraries may keep connections open for a little while after a Dispose (so that if code needs the database again it can resume using the earlier connection) such libraries may use timers to limit how long connections are maintained in expectation of further use, rather than depend upon the garbage-collector (which might go a very long time without noticing that an object has been abandoned).
In C++ it is easily possible to have a permanent memory leak - just allocate memory and don't release it:
new char; //permanent memory leak guaranteed
and that memory stays allocated for the lifetime of the heap (usually the same as program runtime duration).
Is the same (a case that will lead to a specific unreferenced object never been released while memory management mechanisms are working properly) possible in a C# program?
I've carefully read this question and answers to it and it mentions some cases which lead to getting higher memory consumption than expected or IMO rather extreme cases like deadlocking the finalizer thread, but can a permanent leak be formed in a C# program with normally functioning memory management?
It depends on how you define a memory leak. In an unmanaged language, we typically think of a memory leak as a situation where memory has been allocated, and no references to it exist, so we are unable to free it.
That kind of leaks are pretty much impossible to create in .NET (unless you call out into unmanaged code, or unless there's a bug in the runtime).
However, you can get another "weaker" form of leaks: when a reference to the memory does exist (so it is still possible to find and reset the reference, allowing the GC to free the memory normally), but you thought it didn't, so you assumed the object being referenced would get GC'ed. That can easily lead to unbounded growth in memory consumption, as you're piling up references to objects that are no longer used, but which can't be garbage collected because they're still referenced somewhere in your app.
So what is typically considered a memory leak in .NET is simply a situation where you forgot that you have a reference to an object (for example because you failed to unsubscribe from an event). But the reference exists, and if you remember about it, you can clear it and the leak will go away.
You can write unmanaged code in .NET if you wish, you have enclose your block of code with unsafe keyword, so if you are writing unsafe code are you not back to the problem of managing memory by yourself and if not get a memory leak?
It's not exactly a memory leak, but if you're communicating with hardware drivers directly (i.e. not through a properly-written .net extension of a set of drivers) then it's fairly possible to put the hardware into a state where, although there may or may not be an actual memory leak in your code, you can no longer access the hardware without rebooting it or the PC...
Not sure if this is a useful answer to your question, but I felt it was worth mentioning.
GC usually delay the collection of unreachable memory to some later time when an analysis of the references show that the memory is unreachable. (In some restricted cases, the compiler may help the GC and warn it that a memory zone is unreachable when it become so.)
Depending on the GC algorithm, unreachable memory is detected as soon as a collection cycle is ran, or it may stay undetected for a certain number of collection cycles (generational GC show this behavior for instance). Some techniques even have blind spots which are never collected (use of reference counted pointer for instance) -- some deny them the name of GC algorithm, they are probably unsuitable in general purpose context.
Proving that a specific zone will be reclaimed will depend on the algorithm and on the memory allocation pattern. For simple algorithm like mark and sweep, it is easy to give a bound (says till the next collection cycle), for more complex algorithms the matter is more complex (under a scheme which use a dynamic number of generations, the conditions in which a full collection is done are not meaningful to someone not familiar with the detail of the algorithm and the precise heuristics used)
A simple answer is that classic memory leaks are impossible in GC environments, as classically a memory leak is leaked because, as an unreferenced block theres no way for the software to find it to clean it up.
On the other hand, a memory leak is any situation where the memory usage of a program has unbounded growth. This definition is useful when analyzing how software might fail when run as a service (where services are expected to run, perhaps for months at a time).
As such, any growable data structure that continues to hold onto references onto unneeded objects could cause service software to effectively fail because of address space exhaustion.
Easiest memory leak:
public static class StaticStuff
{
public static event Action SomeStaticEvent;
}
public class Listener
{
public Listener() {
StaticStuff.SomeStaticEvent+=DoSomething;
}
void DoSomething() {}
}
instances of Listener will never be collected.
If we define memory leak as a condition where a memory that can be used for creating objects, cannot be used or a memory that can be released does not then
Memory leaks can happen in:
Events in WPF where weak events need to be used. This especially can happens in Attached Properties.
Large objects
Large Object Heap Fragmentation
http://msdn.microsoft.com/en-us/magazine/cc534993.aspx
I am comfortable with how to implement IDisposable..
However I'm not clear as to the exact consequences of not implementing this properly. Is it a case that unmanaged resources will never be cleared thus causing memory leaks or will the memory eventually get recycled anyway, just not in a timely manner?
It depends.
If the object in question fails to implement IDisposable, but does still implement a finalizer correctly, the resources will eventually get cleaned up.
However, if it doesn't properly implement a finalizer, AND it's wrapping an unmanaged resource, that resource will be leaked. This is more often a resource leak (ie: leaking a HANDLE stored as IntPtr) more than a traditional "memory leak" (though, of course, the object being pointed to does still use memory).
will the memory eventually get recycled anyway?
One point here... IDisposable has nothing to do with memory - it's about resources. These resources often use memory of their own, but implementing IDisposable does not cause your garbage collector to reclaim memory more quickly.
It depends entirely on what te implementation is, of course.
An incorrect dispose on a SqlTransaction could lead to excessive long blocking, for example - impacting multiple machines and users. An incorrect dispose on a SqlConnection could cause connection saturation and inability to connect. These are not just memory (etc) issues.
IIRC a missing graphic (pen/brush/whatever) dispose was responsible for a VS crash bug - GC didn't happen because there was no memory pressure, so instead it simply saturated the GDI handles and broke.
An incorrect file/stream dispose could cause exceptions due to a file bring unavailable for reading/writing - potentially losing data.
With unmanaged resources not properly handled, anything is possible.
Instead of thinking in terms of some vaguely-defined "resources", think of IDisposable as meaning "This object manipulates something outside itself (and possibly even outside the computer!) in a way that needs to be cleaned up, while the information necessary for such cleanup still exists." A finalizer which fires essentially says "Nobody else is using this stuff anymore, and I'm the only one who knows about it; I'd better clean it up because nobody else is going to."
It is dangerous to assume that finalizers will magically take care of things; while many objects can be safely abandoned, many others cannot. In some cases, it will be easy for an object to detect that it has been abandoned and clean up appropriately. In some other cases, it may be so difficult as to be impractical. Consider an object which is supposed to count how many times the word "quack" appears in a long-lived data stream. If the creator of the object has forgotten about it, and nobody is every going to query the count, the object may as well go away. Unfortunately, since the data stream holds a reference to the counting object (so as to let it know when data comes in), the counter object won't go away. If code creates a counter, then forgets about it, then creates another one, forgets about it, etc. this could become an unbounded memory leak.
A class that holds unmanaged resources should ensure that they are cleaned up during finalization. However, this only happens when the garbage collector gets around to it.
Not calling Dispose means waiting for the garbage collector to need memory before the resources are released.
The latter of the two. Just not in a timely manner.
The importance of this depends on the resources you are allocating. This is why you are more likely to see IDisposable interfaces on objects that open files and reference other, more crticital, resources.
Yes, unmanaged resources will never be released. (This assumes that the IDisposable types in question don't implement a "fallback" finaliser. If they do then the finaliser should sort things out... eventually.)
However, it's not just unmanaged memory allocations that you need to worry about; it's any unmanaged resources. For example, if you don't properly dispose of your database connections then you might find that the connection pool quickly gets exhausted.
Another potential problem with not implementing the disposable pattern properly - it is very easy to accidentally call .Dispose() more than once. If your Dispose method does not check for successive calls to it, and (eg.) attempts to free up an unmanaged resource more than once, you might end up with some unrecoverable error, or unexpected behaviour.