In my two-week long quest to solve a problem:
How to get notified before static variables are finalized
Unload event for the default Application Domain?
Profiling ASP.net applications over the long term?
the fundamental problem is the garbage collector. My object's Finalizer would have been the ideal time to react when the object is about to be freed (i.e. to have its resources reclaimed). The problem with the .NET garbage collection system is that by the time my finalizer is called, it's entirely likely that other objects i own have already been finalized.
The problem would be trivial to solve if i were interoping with native class objects. The garbage collector is unable to free those objects behind my back (and without my permission). So by the time my managed object's finalizer is called, i know that my internal state is still valid.
What i need is a way to tell the garbage collector to keep your hands off.
Is there a ways to prevent an object from being finalized?
For example
For example, the following code is buggy, because the finalizer mistakenly things that the private _values object still exists. In reality it is likely already finalized out from under me:
class Sqm
{
private List<Value> _values = new List<Value>();
//finalizer
public ~Sqm()
{
Shutdown();
}
protected void Shutdown()
{
foreach (var value in _values) //<-- crash; _values no longer exists
SaveValueToHardDrive(value); //<-- crash; value no longer exists
}
}
What i need is a way to tell the garbage collector not to finalize that list object, or any of the objects inside it:
class Sqm
{
private List<Value> _values = new List<Value>();
//constructor
public Sqm()
{
GC.LetMeManuallyFinalize(_values);
}
//finalizer
public ~Sqm()
{
Shutdown();
GC.ManuallyFinalize(_values);
}
protected void Shutdown()
{
foreach (var value in _values)
SaveValueToHardDrive(value);
}
}
That has two possible problems:
there is no GC.ManuallyFinalize method
it might suppress freeing the _values list itself, but the objects it references would likely still be finalized behind my back:
protected void Shutdown()
{
foreach (var value in _values)
SaveValueToHardDrive(value); //<---crash, contained object already finalized
}
So now i need to ensure that the objects, as they are added to the list, are also excluded from finalization:
public void AddSample(String name, Int64 value)
{
Entry entry = GetEntryByName(name);
if (entry == null)
{
entry = new Entry();
GC.LetMeManuallyFinalize(entry);
}
entry.Count += 1;
entry.Sum += value;
entry.Average = entry.Sum / entry.Count;
}
//finalizer
public ~Sqm()
{
foreach (var value in _values)
GC.ManuallyFinalize(value);
GC.ManuallyFinalize(_values);
}
That probably has the problem that while Entry has no other internal objects, i don't know about List<T>. And the garbage collector might perform an unwanted labotomy on _values even though _values itself hasn't been finalized.
Use GCAlloc.Alloc()
#MatthewWatson had an excellent idea. i think it would be useful to point out why it's wrong. Use GCAlloc.Alloc to hold a reference to the object. Then you use can access it during your finalizer:
public Sqm()
{
private List<Value> _values = new List<Value>();
private GCHandle _valuesHandle; //handle to keep _values alive
//constructor
Sqm()
{
//prevent _values from being finalized
_valuesHandle = GCAlloc.Alloc(_values);
}
//finalizer
~Sqm()
{
try
{
Shutdown(_values); //Safe, right? RIGHT? _values couldn't have been finalized
}
finally
{
_valuesHandle.Free();
}
}
private void Shutdown(List<Values> values)
{
foreach (var value in values)
{
//The list itself might not have been finalized
//But objects used internally to Microsoft's List<T> class have been finalized
//and objects in the list itself were already finalized
SaveValueToHardDrive(value); //<--BAD: values inside list were already finalized
}
}
}
Note: It also probably fails because of the pseudo-documented behavior. From The Truth About GCHandles:
When you create a new GCHandle, a new entry in the AppDomain's handle table is created. This entry is kept until the handle is freed (via GCHandle.Free()) or the AppDomain is unloaded.
Emphasis mine.
So that's right out. i need to tell the garbage collector
Do not finalize this object (and everything inside it)
The tricky part is the internal private members of classes i do not own; even an object still referenced by GCAlloc will still have objects it depends on finalized behind it's back.
Sample Usage
public static Foo
{
public static Sqm = new Sqm();
}
Foo.Sqm.AddSample("QueryCustomerInfo", stopwatch.TotalMicroseconds);
It's a very bad practice to do any long term operations during finalization, like input-output, anyway.
You should consider storing your _values lists of your objects in the static list - this will prevent values from destruction. When your object is being finalizing, you could save the reference to the inner list in another static list, which is checked periodically. When it contains references to lists, it will mean that your object was destructed and values it contained should be saved.
class Sqm
{
private static List<List<Value>> = _lists = new List<List<Value>>();
private static List<List<Value>> = _finalizationQueue = new List<List<Value>>();
private List<Value> _values = new List<Value>();
Sqm() { _lists.Add(_values); }
~Sqm() { _finalizationQueue.Add(_values); }
public static void CheckAndSave()
{
foreach(var list in _finalizationQueue)
SaveValues(list);
}
}
UPD.: If domain may be shutdown when you dont want it to, your only way is to store values in another domain.
Related
We have some data source classes which handle operations on disposable resources, like this:
public class SomeDataStore
{
private readonly object dictionaryLock = new object();
private readonly Dictionary<uint, SomeDisposableClass> dataStore = new Dictionary<uint, SomeDisposableClass>();
public bool Remove(uint key)
{
bool returnValue = false;
lock (dictionaryLock)
{
returnValue = dataStore.Remove(key);
}
//OR...
lock (dictionaryLock)
{
SomeDisposableClass element;
if (dataStore.TryGetValue(key, out element))
{
element.Dispose();
returnValue = dataStore.Remove(key);
}
}
return returnValue;
}
public void Clear()
{
lock (dictionaryLock)
{
dataStore.Clear();
}
//OR...
lock (dictionaryLock)
{
foreach (var value in dataStore.Values)
value.Dispose();
dataStore.Clear();
}
}
//Some other Datastore access members
}
public class SomeDisposableClass : IDisposable
{
public void Dispose()
{
//Dispose resources..
}
}
Not sure which version should be better and why? Does Dictionary's Clear or Remove internally handle disposable resource?
Why should an element within a dictionary be disposed automatically when removed from it? It might exist in another list or whatever anyway. Having said this it´s quite dangerous to dispose an object when removing it in some collection. None of the methods you mentioned (Remove, Clear, whatever) has any knowledge on disposable objects. All those methods do is to remove the reference to your instance from the internal cache. Removing a reference to an object however doesn´t mean it should be released (GC) or even disposed (IDisposable). This in fact has nothing to do with disposable objects. Even the GC won´t release your object if there´s another reference to it existing in another list for example.
So you should allways dispose your ressources where you have control over them - which is usually in the same context where you created them.
No, you should take care of Disposable objects your self. The dictionary does not take care of this.
Like Micky says, only when it has ownership on the objects.
Appearently, in java; "the object will not be collected until it gets unreachable again" (What if a finalizer makes an object reachable?). I assume the same holds in C#, but does it?
A quick example:
public static void MyWeakCache
{
private static readonly ICollection<WeakReference<MyFinalizableObject>> cache;
private static readonly IList<MyFinalizableObject> pendingRemoval;
// Other implementation
public static void Register(MyFinalizableObject myObj)
{
cache.Add(new WeakReference<MyFinalizableObject>(myObj));
}
public static void Deregister(MyFinalizableObject myObj)
{
pendingRemoval.Add(myObj);
}
}
public class MyFinalizableObject
{
public MyFinalizableObject()
{
MyWeakCache.Register(this);
}
~MyFinalizableObject()
{
// Object is reachable again after this call.
MyWeakCache.Deregister(this);
}
}
If the finalizer creates a new reference to the object then the object is not garbage collected. But there's no way of knowing when the finalizer will run or if it will run.
That creates a scenario where, for an unknowable period of time between when the last reference to the object is removed and when the finalizer is called, the object is in limbo, sitting in memory with no references to it. The natural flow of garbage collection is that no object should return from this state. It's like bringing Frankenstein to life. He's dead, let nature take its course.
It's interesting but has no practical application because there's no reason to do this. By definition finalizers don't exist to maintain references to objects. The only thing it can accomplish is creating unpredictable behaviors and bugs.
I tried with the below code, I got the output as 1000. I heard assigning object must share the reference instead of copying the entire object memory. Here the result is different.Can anyone help.
public aaaaa ad = new aaaaa();
static void Main(string[] args)
{
Program p = new Program();
p.fun1();
p.fun2();
}
public void fun1()
{
using(smallclass s = new smallclass())
{
s.j = 1000;
ad.fun1(s);
}
}
public void fun2()
{
ad.fun2();
}
public class aaaaa
{
public smallclass h = new smallclass();
public void fun1(smallclass d)
{
h = d;
}
public void fun2()
{
Console.WriteLine(h.j);
}
}
public class smallclass:IDisposable
{
public int j = 9;
public void Dispose()
{
GC.SuppressFinalize(this);
}
}
Update:
I expect an object reference exception as the referenced memory is disposed in p.fun1();
Here is an simple example how assinging works
using System;
namespace ConsoleApplication1
{
internal class Program
{
private static smallclass objA = new smallclass();
private static smallclass objB = new smallclass();
private static void Main(string[] args)
{
showValues();
objA.value = 1000;
showValues();
objB = objA;
showValues();
objA.value = 1055;
showValues();
}
private static void showValues()
{
Console.WriteLine("objA.value: " + objA.value);
Console.WriteLine("objB.value: " + objB.value);
Console.ReadLine();
}
}
internal class smallclass : IDisposable
{
public int value = 0;
public void Dispose()
{
//Here you can remove eventHandlers
//or do some other stuff before the GC will play with it
}
}
}
Like you can see
first we create 2 objects objA and objB
than we show the values like expected they are both 0
after that we increase the value of objA to 1000
the value of objA a is 1000 and the value of objB remains at 0
NOW we assingning objA and objB
so the value of objB got also 1000
if we now change the value of objA to 1055
the value of objB get also changed
because objB is no more an separate object it now holds the same
reference like objA does
EDIT
And now i will show you how you get your Error based on your example
change your aaaaa class to:
public class aaaaa
{
public WeakReference<smallclass> h;
public void fun1(smallclass d)
{
h = new WeakReference<smallclass>(d);
}
public void fun2()
{
smallclass k;
if(h.TryGetTarget(out k))
Console.WriteLine(k.j);
else
Console.WriteLine("ERROR ERRROR ERROR");
}
}
and modify your static void Main(string[] args) to:
static void Main(string[] args)
{
Program p = new Program();
p.fun1();
GC.Collect();
p.fun2();
Console.Read();
}
Ok lets get through the changes
we are using the WeakReference<T> (you could also use WeakReference)
if the GC now comes across our object he can't find a StrongReference so can Collect it
now to the GC.Collect() YOU need to call it because it forced the GC to do his work (now at this moment)
and remember like i told you before IDisposable will get called from the GC before he destroys the object (AFAIK) so there is the place to put all the stuff that need to be done before the object will get destroyed
No, assingning is not a "new" statement, it copies.... a reference, it does not create a new object. For a class.
For a struct, it does so.
I suggest learning C# by reading the documentation or a book - those basics are normally handled to great detail in those.
You will not go far wrong if you think of every reference type variable, field, parameter, array slot, or other such storage location, has holding either "null", or "object #24601" [or some other number]. There are really only a handful things that can be done with references:
You may create a null reference
You may ask the system to create a new object and return a reference to it
You may copy one reference to another
You may check whether two references are equal to each other, or whether one is equal to null.
You may ask the system to perform some action upon the object identified by a reference
If myCar is a variable of some reference type, a statement like myCar.Color = CarColors.Blue won't affect the variable myCar at all. Instead, it will observe that myCar holds [e.g.] "Object #8675309", and then ask the system to access the Color property or field of object #8675309. Conversely, if otherCar happens to hold "object #90210", a statement of the form otherCar=myCar won't do anything with object #8675309, nor object #90210, but will instead replace the "90210" stored in otherCar with "8675309".
Objects are guaranteed to exist as long as any form of reference to them exists, but if there are two objects which, although referenced by each other, are not referenced by anything else in the universe, both objects may simultaneously cease to exist. This rule is absolute, but there are a couple of twists: code may request a WeakReference to an object; an object is guaranteed to exist as long as a weak reference to it exists, but if the system discovers that no strong references to an object exist, it will invalidate every WeakReference to it. Further, the system keeps a list of all objects that have would like to be notified if they are abandoned. If the system finds that this list holds the only reference to an object, it will move the object to a strongly-referenced list of objects whose Finalize method should run at the first convenient opportunity. When the object's Finalize method is run, the reference will be removed from that latter list. If no reference to the object has been stored anywhere in the mean time, the object will cease to exist.
I have replaced GC.SuppressFinalize with GC.Collect() in dispose function, however this is not freeing the memory.. and am receiving 1000 as a result.
I guess, as it holds other reference(the variable h), GC will not free the memory, even if we invoked it explicit.
So we can very well pass and assign the objects irrespective of the allocated(new) object going out of scope.
Please correct me If i am wrong.
I have a type that contains a timer which fires an instance method every 2 seconds. My problem is that when the object instance drops out of scope the object is not collected because the timer thread maintains a reference.
My question is; Is it possible to determine, at runtime, how many references a object has and if the reference count is 1 is that reference a circular reference to the same object? Basically I was thinking of adding code to the timer call back to stop the timer and set the stack reference to null if the object only has 1 circular reference.
Thanks
public class TypeWithSelfRefresh<T>
{
private readonly IList<T> _cache = new List<T>();
private Timer _refreshTimer;
public TypeWithSelfRefresh()
{
_refreshTimer = new Timer(delegate { ClearCache();},null,2000,2000);
}
public void ClearCache()
{
lock (_cache)
{
_cache.Clear();
}
}
public void Add(T item)
{
lock (_cache)
{
_cache.Add(item);
}
}
public IEnumerable<T> GetItems()
{
lock (_cache)
{
foreach (T item in _cache)
{
yield return item;
}
}
}
}
Take a look at WeakReference.
The problem isn't with circular references. .NET framework handles that just fine. The problem is that you are dealing with an object that implements Dispose for a reason (Timer) without disposing of it properly.
The reason any timer has to implement Dispose is simple. For it work, it has to give away a reference of itself to the underlying mechanism running the timer in the background. And that is a reference you have no direct control over. To get rid of that reference you have to call Dispose.
The circular reference shouldn't prevent the object from being collected by the GC, because it is not reachable. What makes you think it isn't collected ? Did you call GC.Collect ?
Okay, I just can't get my head around multi-threading scenarios properly. Sorry for asking a similar question again, I'm just seeing many different "facts" around the internet.
public static class MyClass {
private static List<string> _myList = new List<string>;
private static bool _record;
public static void StartRecording()
{
_myList.Clear();
_record = true;
}
public static IEnumerable<string> StopRecording()
{
_record = false;
// Return a Read-Only copy of the list data
var result = new List<string>(_myList).AsReadOnly();
_myList.Clear();
return result;
}
public static void DoSomething()
{
if(_record) _myList.Add("Test");
// More, but unrelated actions
}
}
The idea is that if Recording is activated, calls to DoSomething() get recorded in an internal List, and returned when StopRecording() is called.
My specification is this:
StartRecording is not considered Thread-Safe. The user should call this while no other Thread is calling DoSomething(). But if it somehow could be, that would be great.
StopRecording is also not officially thread-safe. Again, it would be great if it could be, but that is not a requirement.
DoSomething has to be thread-safe
The usual way seems to be:
public static void DoSomething()
{
object _lock = new object();
lock(_lock){
if(_record) _myList.Add("Test");
}
// More, but unrelated actions
}
Alternatively, declaring a static variable:
private static object _lock;
public static void DoSomething()
{
lock(_lock){
if(_record) _myList.Add("Test");
}
// More, but unrelated actions
}
However, this answer says that this does not prevent other code from accessing it.
So I wonder
How would I properly lock a list?
Should I create the lock object in my function or as a static class variable?
Can I wrap the functionality of Start and StopRecording in a lock-block as well?
StopRecording() does two things: Set a boolean variable to false (to prevent DoSomething() from adding more stuff) and then copying the list to return a copy of the data to the caller). I assume that _record = false; is atomic and will be in effect immediately? So normally I wouldn't have to worry about Multi-Threading here at all, unless some other Thread calls StartRecording() again?
At the end of the day, I am looking for a way to express "Okay, this list is mine now, all other threads have to wait until I am done with it".
I will lock on the _myList itself here since it is private, but using a separate variable is more common. To improve on a few points:
public static class MyClass
{
private static List<string> _myList = new List<string>;
private static bool _record;
public static void StartRecording()
{
lock(_myList) // lock on the list
{
_myList.Clear();
_record = true;
}
}
public static IEnumerable<string> StopRecording()
{
lock(_myList)
{
_record = false;
// Return a Read-Only copy of the list data
var result = new List<string>(_myList).AsReadOnly();
_myList.Clear();
return result;
}
}
public static void DoSomething()
{
lock(_myList)
{
if(_record) _myList.Add("Test");
}
// More, but unrelated actions
}
}
Note that this code uses lock(_myList) to synchronize access to both _myList and _record. And you need to sync all actions on those two.
And to agree with the other answers here, lock(_myList) does nothing to _myList, it just uses _myList as a token (presumably as key in a HashSet). All methods must play fair by asking permission using the same token. A method on another thread can still use _myList without locking first, but with unpredictable results.
We can use any token so we often create one specially:
private static object _listLock = new object();
And then use lock(_listLock) instead of lock(_myList) everywhere.
This technique would have been advisable if myList had been public, and it would have been absolutely necessary if you had re-created myList instead of calling Clear().
Creating a new lock in DoSomething() would certainly be wrong - it would be pointless, as each call to DoSomething() would use a different lock. You should use the second form, but with an initializer:
private static object _lock = new object();
It's true that locking doesn't stop anything else from accessing your list, but unless you're exposing the list directly, that doesn't matter: nothing else will be accessing the list anyway.
Yes, you can wrap Start/StopRecording in locks in the same way.
Yes, setting a Boolean variable is atomic, but that doesn't make it thread-safe. If you only access the variable within the same lock, you're fine in terms of both atomicity and volatility though. Otherwise you might see "stale" values - e.g. you set the value to true in one thread, and another thread could use a cached value when reading it.
There are a few ways to lock the list. You can lock on _myList directly providing _myList is never changed to reference a new list.
lock (_myList)
{
// do something with the list...
}
You can create a locking object specifically for this purpose.
private static object _syncLock = new object();
lock (_syncLock)
{
// do something with the list...
}
If the static collection implements the System.Collections.ICollection interface (List(T) does), you can also synchronize using the SyncRoot property.
lock (((ICollection)_myList).SyncRoot)
{
// do something with the list...
}
The main point to understand is that you want one and only one object to use as your locking sentinal, which is why creating the locking sentinal inside the DoSomething() function won't work. As Jon said, each thread that calls DoSomething() will get its own object, so the lock on that object will succeed every time and grant immediate access to the list. By making the locking object static (via the list itself, a dedicated locking object, or the ICollection.SyncRoot property), it becomes shared across all threads and can effectively serialize access to your list.
The first way is wrong, as each caller will lock on a different object.
You could just lock on the list.
lock(_myList)
{
_myList.Add(...)
}
You may be misinterpreting the this answer, what is actually being stated is that they lock statement is not actually locking the object in question from being modified, rather it is preventing any other code using that object as a locking source from executing.
What this really means is that when you use the same instance as the locking object the code inside the lock block should not get executed.
In essence you are not really attempting to "lock" your list, you are attempting to have a common instance that can be used as a reference point for when you want to modify your list, when this is in use or "locked" you want to prevent other code from executing that would potentially modify the list.