Develop Reference

C# LazyCache concurrent dictionary garbage collection

Been having some problems with a web based .Net(C#) application. I'm using the LazyCache library to cache frequent JSON responses (some in & around 80+KB) for users belonging to the same company across user sessions.
One of the things we need to do is to keep track of the cache keys for a particular company so when any user in the company makes mutating changes to items being cached we need to clear the cache for those items for that particular company's users to force the cache to be repopulated immediately upon the receiving the next request.
We choose LazyCache library as we wanted to do this in memory without needing to use an external cache source such as Redis etc as we don't have heavy usage.
One of the problems we have using this approach is we need to keep track of all the cache keys belonging to a particular customer anytime we cache. So when any mutating change is made by company user's to the relevant resource we need to expire all the cache keys belonging to that company.
To achieve this we have a global cache which all web controllers have access to.
private readonly IAppCache _cache = new CachingService();
protected IAppCache GetCache()
{
return _cache;
}
A simplified example (forgive any typos!) of our controllers which use this cache would be something like below
[HttpGet]
[Route("{customerId}/accounts/users")]
public async Task<Users> GetUsers([Required]string customerId)
{
var usersBusinessLogic = await _provider.GetUsersBusinessLogic(customerId)
var newCacheKey= "GetUsers." + customerId;
CacheUtil.StoreCacheKey(customerId,newCacheKey)
return await GetCache().GetOrAddAsync(newCacheKey, () => usersBusinessLogic.GetUsers(), DateTimeOffset.Now.AddMinutes(10));
}
We use a util class with static methods and a static concurrent dictionary to store the cache keys - each company (GUID) can have many cache keys.
private static readonly ConcurrentDictionary<Guid, ConcurrentHashSet<string>> cacheKeys = new ConcurrentDictionary<Guid, ConcurrentHashSet<string>>();
public static void StoreCacheKey(Guid customerId, string newCacheKey)
{
cacheKeys.AddOrUpdate(customerId, new ConcurrentHashSet<string>() { newCacheKey }, (key, existingCacheKeys) =>
{
existingCacheKeys.Add(newCacheKey);
return existingCacheKeys;
});
}
Within that same util class when we need to remove all cache keys for a particular company we have a method similar to below (which is caused when mutating changes are made in other controllers)
public static void ClearCustomerCache(IAppCache cache, Guid customerId)
{
var customerCacheKeys = new ConcurrentHashSet<string>();
if (!cacheKeys.TryGetValue(customerId,out customerCacheKeys))
{
return new ConcurrentHashSet<string>();
}
foreach (var cacheKey in customerCacheKeys)
{
cache.Remove(cacheKey);
}
cacheKeys.TryRemove(customerId, out _);
}
We have recently been getting performance problems that our web requests response time slow significantly over time - we don't see significant change in terms of the number of requests per second.
Looking at the garbage collection metrics we seem to notice a large Gen 2 heap size and a large object size which seem to going upwards - we don't see memory been reclaimed.
We are still in the middle of debugging this but I'm wondering could using the approach described above lead to the problems we are seeing. We want thread safety but could there be an issue using the concurrent dictionary we have above that even after we remove items that memory is not being freed leading to excessive Gen 2 collection.
Also we are using workstation garbage collection mode, imagine switching to server mode GC will help us (our IIS server has 8 processors + 16 GBs ram) but not sure switching will fix all the problems.

You may want to take advantage of the ExpirationTokens property of the MemoryCacheEntryOptions class. You can also use it from the ICacheEntry argument passed in the delegate of the LazyCache.Providers.MemoryCacheProvider.GetOrCreateAsync method. For example:
Task<T> GetOrAddAsync<T>(string key, Func<Task<T>> factory,
int durationMilliseconds = Timeout.Infinite, string customerId = null)
{
return GetMemoryCacheProvider().GetOrCreateAsync<T>(key, (options) =>
{
if (durationMilliseconds != Timeout.Infinite)
{
options.SetSlidingExpiration(TimeSpan.FromMilliseconds(durationMilliseconds));
}
if (customerId != null)
{
options.ExpirationTokens.Add(GetCustomerExpirationToken(customerId));
}
return factory();
});
}
Now the GetCustomerExpirationToken should return an object implementing the IChangeToken interface. Things are becoming a bit complex, but bear with me for a minute. The .NET platform doesn't provide a built-in IChangeToken implementation suitable for this case, since it is mainly focused on file system watchers. Implementing one is not difficult though:
class ChangeToken : IChangeToken, IDisposable
{
private volatile bool _hasChanged;
private readonly ConcurrentQueue<(Action<object>, object)>
registeredCallbacks = new ConcurrentQueue<(Action<object>, object)>();
public void SignalChanged()
{
_hasChanged = true;
while (registeredCallbacks.TryDequeue(out var entry))
{
var (callback, state) = entry;
callback?.Invoke(state);
}
}
bool IChangeToken.HasChanged => _hasChanged;
bool IChangeToken.ActiveChangeCallbacks => true;
IDisposable IChangeToken.RegisterChangeCallback(Action<object> callback,
object state)
{
registeredCallbacks.Enqueue((callback, state));
return this; // return null doesn't work
}
void IDisposable.Dispose() { } // It is called by the framework after each callback
}
This is a general implementation of the IChangeToken interface, that is activated manually with the SignalChanged method. The signal will be propagated to the underlying MemoryCache object, which will subsequently invalidate all entries associated with this token.
Now what is left to do is to associate these tokens with a customer, and store them somewhere. I think that a ConcurrentDictionary should be quite adequate:
private static readonly ConcurrentDictionary<string, ChangeToken>
CustomerChangeTokens = new ConcurrentDictionary<string, ChangeToken>();
private static ChangeToken GetCustomerExpirationToken(string customerId)
{
return CustomerChangeTokens.GetOrAdd(customerId, _ => new ChangeToken());
}
Finally the method that is needed to signal that all entries of a specific customer should be invalidated:
public static void SignalCustomerChanged(string customerId)
{
if (CustomerChangeTokens.TryRemove(customerId, out var changeToken))
{
changeToken.SignalChanged();
}
}

Large objects (> 85k) belong in gen 2 Large Object Heap (LOH), and they are pinned in memory.
GC scans LOH and marks dead objects
Adjacent dead objects are combined into free memory
The LOH is not compacted
Further allocations only try to fill in the holes left by dead objects.
No compaction, but only reallocation may lead to memory fragmentation.
Long running server processes can be done in by this - it is not uncommon.
You are probably seeing fragmentation occur over time.
Server GC just happens to be multi-threaded - I wouldn't expect it to solve fragmentation.
You could try breaking up your large objects - this might not be feasible for your application.
You can try setting LargeObjectHeapCompaction after a cache clear - assuming it's infrequent.
GCSettings.LargeObjectHeapCompactionMode = GCLargeObjectHeapCompactionMode.CompactOnce;
GC.Collect();
Ultimately, I'd suggest profiling the heap to find out what works.

Garbage collection from Main method (infinite loop)

I am implementing an infinite task that runs every so often via a .net console application. However I am concerned that the below will result in a memory leak. Since Main is static (this is where my knowledge on garbage collection gets foggy) does it not mean that the object that I create within the try and catch won't be picked up by the garbage collector until Main finishes (which is never)?
Could someone please explain how the garbage collector would behave in this case?
public static void Main(string[] args)
{
Log.Logger = new LoggerConfiguration()
.MinimumLevel.Debug()
.WriteTo.RollingFile("Logs/log-{Date}.txt")
.CreateLogger();
while (true)
{
try
{
Thread.Sleep(1000);
new UpdatePlayer().Run();
}
catch (Exception ex)
{
Log.Error(ex.ToString());
}
}
}

You'll have no memory leak: Main doesn't have any reference to UpdatePlayer() instance:
...
try
{
Thread.Sleep(1000);
// Instance created, executed
new UpdatePlayer().Run();
}
// And can be safely collected here when the instance is out of scope (`try {...}`)
// the anonymous instance can't be accessed and that's enough
A sample with memory leak:
// please notice, that anchor is outside the `while(true)` scope
List<Object> anchor = new List<Object>();
...
while (true)
{
try
{
Thread.Sleep(1000);
// Instance created
var item = new UpdatePlayer();
// anchored
anchor.Add(item);
// and executed
item.Run();
}
// item can't be collected here: anchor has a reference to it
// And the item potentially can be accessed by, say, anchor[0]
Edit: and if you move the collection into the while(true) scope, the code will be without memory leak:
try
{
List<object> pseudoAnchor = new List<object>();
// Instance created
var item = new UpdatePlayer();
// tried to be anchored (illusion)
pseudoAnchor.Add(item);
// and executed
item.Run();
}
...
// the only reference to item is pseudoAnchor, but
// pseudoAnchor has not been referenced, and so GC
// can safely collect entire pseudoAnchor with all its items
// (just one `item` in fact)

Since Main is static (this is where my knowledge on garbage collection gets foggy) does it not mean that the object that I create within the try and catch won't be picked up by the garbage collector until Main finishes (which is never)?
No, that is false.
The Garbage Collector is able to free the memory of any object it can prove can no longer be accessed from managed code. In this case, it's clear that after the Run method ends (unless you store a reference to that object somewhere else) that object will no longer be accessible, and so the GC is allowed to free it (it may take some time to do so, but it's allowed to).

Why does disposing of an entity context not free up memory

In my tests I am reading a text file line by line and inserting an entity along with other related entities. The problem is when too many are inserted I receive an Out of memory exception.
In my attempt to prevent this I create a new DbContext for every 50 rows and dispose of the old one. It was my understanding that this would free up memory from the earlier entity operations, but the memory continues to climb and if the file is big enough an out of memory exception occurs. This is related to the entity code as if I remove the lines of code that adds the entity the memory stays at a consistent usage.
Below is a simplified version of my code.
public class TestClass
{
public void ImportData(byte[] fileBytes)
{
using (Stream stream = new MemoryStream(fileBytes))
{
TextFieldParser parser = new TextFieldParser(stream);
parser.TextFieldType = FieldType.Delimited;
parser.SetDelimiters(",");
while (!parser.EndOfData)
{
//Processes 50 lines creates a new DbContext each time its called
ImportBatch(parser);
}
}
}
public void ImportBatch(TextFieldParser parser)
{
using(myDbContext context = new myDbContext())
{
context.Configuration.AutoDetectChangesEnabled = false;
int batchCount = 0;
while (!parser.EndOfData && batchCount < 50)
{
string[] fields = parser.ReadFields();
//Here I call some code that will add an entity and add releated entities
//In its navigation properties
MyService.AddMyEntity(fields,myDbContext);
batchCount++;
}
myDbContext.ChangeTracker.DetectChanges();
myDbContext.SaveChanges();
}
}
}
As I am disposing and creating a new context every 50 inserts I would expect the memory usage to stay constant, but it seems to be constant for the first 2 thousands rows but after that the memory constantly climbs, unitl an OutOfMemory exception is hit.
Is there a reason why disposing of a dbContext in the following fashion would not result in the memory being released?
EDIT - added some simplified code of my add entity method
public void AddMyEntity(string[] fields, MyDbContext, myDbContext)
{
MyEntity myEntity = new MyEntity();
newRequest.InsertDate = DateTime.UtcNow;
newRequest.AmendDate = DateTime.UtcNow;
//If I remove this line the memory does not consistently climb
myDbContext.MyEntities.Add(myEntity);
foreach(string item in fields)
{
ReleatedEntity releatedEntity = new ReleatedEntity();
releatedEntity.Value = item;
newRequest.ReleatedEntities.Add(releatedEntity);
}
}
Another Edit
Turns out after more testing it is something to do with Glimpse profiler. I have included Glimpse in my project and the web config has a section similar to below.
<glimpse defaultRuntimePolicy="On" endpointBaseUri="~/Glimpse.axd">
<tabs>
<ignoredTypes>
<add type="Glimpse.Mvc.Tab.ModelBinding, Glimpse.Mvc5"/>
<add type="Glimpse.Mvc.Tab.Metadata, Glimpse.Mvc5"/>
</ignoredTypes>
</tabs>
<inspectors>
<ignoredTypes>
<add type="Glimpse.Mvc.Inspector.ModelBinderInspector, Glimpse.Mvc5"/>
</ignoredTypes>
</inspectors>
Turning defaultRuntimePolicy to Off fixed the memory leak. Still not sure why tho

Calling Dispose on an object does not necessarily free up memory. Objects are removed from memory by the garbage collector when they are no longer referenced by any live object. Calling Dispose might free up other resources (e.g. by closing an open SQL connection in the case of a DbContext), but it's only when the object is no longer referenced that it becomes a candidate for garbage collection.
There's no guarantee that the garbage collector will run at a specific point in time. Calling Dispose certainly doesn't cause it to run. Having said that, I'm surprised that it doesn't run before you run out of memory. You could force it to run with GC.Collect but that really shouldn't be necessary.
It's possible that you still have a reference to your context objects that's causing them not to be considered eligible for garbage collection. For example, you pass myDbContext to your AddEntity method in your service layer - does that store something that includes a back-reference (even indirectly) to the context?

Because whenever you call ImportBatch(parser) it creates a new DbContext. Not 1 DbContext for each 50. You may try a get property to count and give back the context to you. Something like this :
int _batchCount = 0;
public myDbContext _db;
public myDbContext Db
{
get
{
// If batchCount > 50 or _db is not created we need to create _db
if (_db == null || _batchCount > 50)
{
// If db is already created _batchcount > 50
if (_db != null)
{
_db.ChangeTracker.DetectChanges();
_db.SaveChanges();
_db.Dispose();
}
_db = new myDbContext();
_db.Configuration.AutoDetectChangesEnabled = false;
_batchCount = 0;
}
batchCount++;
return _db;
}
}
Additionally in MyService.AddMyEntity(fields); you are using the DbContext from the MyServiceclass, not the one you are creating in using line.

LuaInterface/C# - Closures created with .NET objects never get cleaned up

I am using the latest version of LuaInterface (http://code.google.com/p/luainterface/) in a C# application. I'm running into a problem where the Lua class is failing to clean up internal references in the ObjectTranslator 'objects' and 'objectsBackMap' dictionaries, resulting in always-growing memory usage.
The following code illustrates the problem:
public class Program
{
public static void Main()
{
Lua lua = new Lua();
string f = #"
return function(myClass, dotNetObject)
local f = function() dotNetObject:TestMethod() end
myClass:StoreFunction(f);
end";
var result = lua.DoString(f)[0] as LuaFunction;
MyClass testItem = new MyClass();
for (int i = 0; i < 50; i++)
{
DotNetObject o = new DotNetObject();
result.Call(testItem, o);
}
lua.DoString("collectgarbage()");
ObjectTranslator translator = (ObjectTranslator)typeof(Lua).GetField("translator", System.Reflection.BindingFlags.NonPublic | System.Reflection.BindingFlags.Instance).GetValue(lua);
Console.WriteLine("Object count: " + translator.objects.Count);
//Prints out 51. 50 of these items are instances of 'DotNetObject', despite them being unreachable.
//Forcing a .NET garbage collection does not help. They are still being held onto by the object translator.
}
public class MyClass
{
public void StoreFunction(LuaFunction function)
{
//I would normally save off the function here to be executed later
}
}
public class DotNetObject
{
public void TestMethod()
{
}
}
}
The problem arises when the anonymous function (local f = ...) creates a closure involving a .NET object from the outer scope. As long as the Lua interpreter stays alive, 50 instances of the DotNetObject class I've created will never be garbage collected, even when forcing a GC in Lua.
Manually disposing of the LuaFunction (function.Dispose()) object in MyClass.StoreFunction solves the problem, but this is not desirable because in my real application I don't know when the Function will execute--or if it ever will. Forcing me to dispose of the LuaFunction changes the entire architecture of the application such that I'm basically doing manual memory management by disposing the object that contains the LuaFunction, and the object that contains that, all the way up the chain.
So, is this a bug in LuaInterface, or am I using the library incorrectly? Any advice is greatly appreciated, thanks!

How to free memory on Dictionary in static class?

I have a problem with freeing memory in C#. I have a static class containing a static dictionary, which is filled with references to objects. Single object zajumie large amount of memory. From time to time I release the memory by deleting obsolete references to the object set to null and remove the item from the dictionary. Unfortunately, in this case, the memory is not slowing down, time after reaching the maximum size of the memory in the system is as if a sudden release of unused resources and the amount of memory used correctly decreases.
Below is the diagram of classes:
public class cObj
{
public DateTime CreatedOn;
public object ObjectData;
}
public static class cData
{
public static ConcurrentDictionary<Guid, cObj> ObjectDict = new ConcurrentDictionary<Guid, cObj>();
public static FreeData()
{
foreach(var o in ObjectDict)
{
if (o.Value.CreatedOn <= DateTime.Now.AddSeconds(-30))
{
cObj Data;
if (ObjectDict.TryGetValue(o.Key, out Data))
{
Data.Status = null;
Data.ObjectData = null;
ObjectDict.TryRemove(o.Key, out Data);
}
}
}
}
}
In this case, the memory is released. If, however, after this operation, I call
GC.Collect ();
Followed by the expected release of unused objects.
How to solve the problem, so you do not have to use the GC.Collect()?

You shouldn't have to call GC.Collect() in most cases. To GC.Collect or not?
I've had similar scenarios where I've just created a dictionary that's limited to n entries, I did this myself on top of ConcurrentDictionary but you could use BlockingCollection.
One possible advantage is that if 1 million entries get added at the same time, all except n will be available for garbage collection rather than 30 seconds later.