I am trying to implement some caching in my application and i want to use the default memory cache in C# (this requirement can be changed if this it wont work). My problem is that don't want to exceed the maximum amount of physical memory i have on the machine, but as i understand i can't add such a constraint to the default memory cache.
In general the policy is:
If the object has been in the cache for 10 min with no requests it is removed
If a new object is added to the cache and the maximum amount of avaliable physical memory is close to used, elements are removed based on LRU
My cache can contain many different objects and they range from 10mb to 2-3gb, so i can't really get the trim function to work.
Are there any suggestions on how to implement a LRU cache monitoring the ram usage? And hoppefully it can be done using the caches in .net?
Edit
I've added a simple example where the MemoryCache is limited to 100Mb and 20% of the physical memory, but that does not change anything. My memory is filled with no removal of cache entries. Note that the polling interval is changed to evert 5 second.
class Item
{
private List<Guid> data;
public Item(int capacity)
{
this.data = new List<Guid>(capacity);
for (var i = 0; i < capacity; i++)
data.Add(Guid.NewGuid());
}
}
class Program
{
static void Main(string[] args)
{
var cache = new MemoryCache(
"MyCache",
new NameValueCollection
{
{ "CacheMemoryLimitMegabytes", "100" },
{ "PhysicalMemoryLimitPercentage", "20" },
{ "PollingInterval", "00:00:05" }
});
for (var i = 0; i < 10000; i++)
{
var key = String.Format("key{0}", i);
Console.WriteLine("Add item: {0}", key);
cache.Set(key, new Item(1000000), new CacheItemPolicy() { UpdateCallback = UpdateHandler } );
}
Console.WriteLine("\nDone");
Console.ReadKey();
}
public static void UpdateHandler(CacheEntryUpdateArguments args)
{
Console.WriteLine("Remove: {0}, Reason: {1}", args.Key, args.RemovedReason.ToString());
}
}
Looks like the System.Runtime.Caching.MemoryCache class would fit this bill nicely. You set caching policy on a per-item basis, so if you add an item with a cache policy of SlidingExpiration with a TimeSpan of 10min, you should get the behavior you are looking for.
This is a .Net v4 class only, so it doesn't exist on earlier runtime versions. If you are in a web context, the ASP.Net cache behaves similarly, but will probably not let you manage system information.
You can set limits on cache size when you create it so that it does not exceed a certain memory footprint:
var myCache = new MemoryCache(
"MyCache",
new NameValueCollection { { "PhysicalMemoryLimit", "50" }} // set max mem pct
);
This should prevent any paging to disk, at least within your application. If there are outside memory pressures or the system is overly aggressive about paging memory to disk, your cache may still be paged out, but not due to overuse within your application. To my knowledge there is no way to reserve the memory pages in C#.
Related
Been having some problems with a web based .Net(C#) application. I'm using the LazyCache library to cache frequent JSON responses (some in & around 80+KB) for users belonging to the same company across user sessions.
One of the things we need to do is to keep track of the cache keys for a particular company so when any user in the company makes mutating changes to items being cached we need to clear the cache for those items for that particular company's users to force the cache to be repopulated immediately upon the receiving the next request.
We choose LazyCache library as we wanted to do this in memory without needing to use an external cache source such as Redis etc as we don't have heavy usage.
One of the problems we have using this approach is we need to keep track of all the cache keys belonging to a particular customer anytime we cache. So when any mutating change is made by company user's to the relevant resource we need to expire all the cache keys belonging to that company.
To achieve this we have a global cache which all web controllers have access to.
private readonly IAppCache _cache = new CachingService();
protected IAppCache GetCache()
{
return _cache;
}
A simplified example (forgive any typos!) of our controllers which use this cache would be something like below
[HttpGet]
[Route("{customerId}/accounts/users")]
public async Task<Users> GetUsers([Required]string customerId)
{
var usersBusinessLogic = await _provider.GetUsersBusinessLogic(customerId)
var newCacheKey= "GetUsers." + customerId;
CacheUtil.StoreCacheKey(customerId,newCacheKey)
return await GetCache().GetOrAddAsync(newCacheKey, () => usersBusinessLogic.GetUsers(), DateTimeOffset.Now.AddMinutes(10));
}
We use a util class with static methods and a static concurrent dictionary to store the cache keys - each company (GUID) can have many cache keys.
private static readonly ConcurrentDictionary<Guid, ConcurrentHashSet<string>> cacheKeys = new ConcurrentDictionary<Guid, ConcurrentHashSet<string>>();
public static void StoreCacheKey(Guid customerId, string newCacheKey)
{
cacheKeys.AddOrUpdate(customerId, new ConcurrentHashSet<string>() { newCacheKey }, (key, existingCacheKeys) =>
{
existingCacheKeys.Add(newCacheKey);
return existingCacheKeys;
});
}
Within that same util class when we need to remove all cache keys for a particular company we have a method similar to below (which is caused when mutating changes are made in other controllers)
public static void ClearCustomerCache(IAppCache cache, Guid customerId)
{
var customerCacheKeys = new ConcurrentHashSet<string>();
if (!cacheKeys.TryGetValue(customerId,out customerCacheKeys))
{
return new ConcurrentHashSet<string>();
}
foreach (var cacheKey in customerCacheKeys)
{
cache.Remove(cacheKey);
}
cacheKeys.TryRemove(customerId, out _);
}
We have recently been getting performance problems that our web requests response time slow significantly over time - we don't see significant change in terms of the number of requests per second.
Looking at the garbage collection metrics we seem to notice a large Gen 2 heap size and a large object size which seem to going upwards - we don't see memory been reclaimed.
We are still in the middle of debugging this but I'm wondering could using the approach described above lead to the problems we are seeing. We want thread safety but could there be an issue using the concurrent dictionary we have above that even after we remove items that memory is not being freed leading to excessive Gen 2 collection.
Also we are using workstation garbage collection mode, imagine switching to server mode GC will help us (our IIS server has 8 processors + 16 GBs ram) but not sure switching will fix all the problems.
You may want to take advantage of the ExpirationTokens property of the MemoryCacheEntryOptions class. You can also use it from the ICacheEntry argument passed in the delegate of the LazyCache.Providers.MemoryCacheProvider.GetOrCreateAsync method. For example:
Task<T> GetOrAddAsync<T>(string key, Func<Task<T>> factory,
int durationMilliseconds = Timeout.Infinite, string customerId = null)
{
return GetMemoryCacheProvider().GetOrCreateAsync<T>(key, (options) =>
{
if (durationMilliseconds != Timeout.Infinite)
{
options.SetSlidingExpiration(TimeSpan.FromMilliseconds(durationMilliseconds));
}
if (customerId != null)
{
options.ExpirationTokens.Add(GetCustomerExpirationToken(customerId));
}
return factory();
});
}
Now the GetCustomerExpirationToken should return an object implementing the IChangeToken interface. Things are becoming a bit complex, but bear with me for a minute. The .NET platform doesn't provide a built-in IChangeToken implementation suitable for this case, since it is mainly focused on file system watchers. Implementing one is not difficult though:
class ChangeToken : IChangeToken, IDisposable
{
private volatile bool _hasChanged;
private readonly ConcurrentQueue<(Action<object>, object)>
registeredCallbacks = new ConcurrentQueue<(Action<object>, object)>();
public void SignalChanged()
{
_hasChanged = true;
while (registeredCallbacks.TryDequeue(out var entry))
{
var (callback, state) = entry;
callback?.Invoke(state);
}
}
bool IChangeToken.HasChanged => _hasChanged;
bool IChangeToken.ActiveChangeCallbacks => true;
IDisposable IChangeToken.RegisterChangeCallback(Action<object> callback,
object state)
{
registeredCallbacks.Enqueue((callback, state));
return this; // return null doesn't work
}
void IDisposable.Dispose() { } // It is called by the framework after each callback
}
This is a general implementation of the IChangeToken interface, that is activated manually with the SignalChanged method. The signal will be propagated to the underlying MemoryCache object, which will subsequently invalidate all entries associated with this token.
Now what is left to do is to associate these tokens with a customer, and store them somewhere. I think that a ConcurrentDictionary should be quite adequate:
private static readonly ConcurrentDictionary<string, ChangeToken>
CustomerChangeTokens = new ConcurrentDictionary<string, ChangeToken>();
private static ChangeToken GetCustomerExpirationToken(string customerId)
{
return CustomerChangeTokens.GetOrAdd(customerId, _ => new ChangeToken());
}
Finally the method that is needed to signal that all entries of a specific customer should be invalidated:
public static void SignalCustomerChanged(string customerId)
{
if (CustomerChangeTokens.TryRemove(customerId, out var changeToken))
{
changeToken.SignalChanged();
}
}
Large objects (> 85k) belong in gen 2 Large Object Heap (LOH), and they are pinned in memory.
GC scans LOH and marks dead objects
Adjacent dead objects are combined into free memory
The LOH is not compacted
Further allocations only try to fill in the holes left by dead objects.
No compaction, but only reallocation may lead to memory fragmentation.
Long running server processes can be done in by this - it is not uncommon.
You are probably seeing fragmentation occur over time.
Server GC just happens to be multi-threaded - I wouldn't expect it to solve fragmentation.
You could try breaking up your large objects - this might not be feasible for your application.
You can try setting LargeObjectHeapCompaction after a cache clear - assuming it's infrequent.
GCSettings.LargeObjectHeapCompactionMode = GCLargeObjectHeapCompactionMode.CompactOnce;
GC.Collect();
Ultimately, I'd suggest profiling the heap to find out what works.
I have a problem with freeing memory in C#. I have a static class containing a static dictionary, which is filled with references to objects. Single object zajumie large amount of memory. From time to time I release the memory by deleting obsolete references to the object set to null and remove the item from the dictionary. Unfortunately, in this case, the memory is not slowing down, time after reaching the maximum size of the memory in the system is as if a sudden release of unused resources and the amount of memory used correctly decreases.
Below is the diagram of classes:
public class cObj
{
public DateTime CreatedOn;
public object ObjectData;
}
public static class cData
{
public static ConcurrentDictionary<Guid, cObj> ObjectDict = new ConcurrentDictionary<Guid, cObj>();
public static FreeData()
{
foreach(var o in ObjectDict)
{
if (o.Value.CreatedOn <= DateTime.Now.AddSeconds(-30))
{
cObj Data;
if (ObjectDict.TryGetValue(o.Key, out Data))
{
Data.Status = null;
Data.ObjectData = null;
ObjectDict.TryRemove(o.Key, out Data);
}
}
}
}
}
In this case, the memory is released. If, however, after this operation, I call
GC.Collect ();
Followed by the expected release of unused objects.
How to solve the problem, so you do not have to use the GC.Collect()?
You shouldn't have to call GC.Collect() in most cases. To GC.Collect or not?
I've had similar scenarios where I've just created a dictionary that's limited to n entries, I did this myself on top of ConcurrentDictionary but you could use BlockingCollection.
One possible advantage is that if 1 million entries get added at the same time, all except n will be available for garbage collection rather than 30 seconds later.
does MemoryCache has functionality to cache fixed number of items?
e.g. We are only interested in cache 2000 items from database. While keep adding items to the cache, if the specified number of items are exceeded, the oldest one can be removed.
If not, do we have to use another thread to do the house keeping regularly?
It doesn't have anything built in that will limit the number of objects. Instead, it checks how much memory is being used, and compares it to the CacheMemoryLimit. If the CacheMemoryLimit is exceeded, it will drop older items. You can also set items to automatically expire after a certain amount of time via the CacheItemPolicy.
These approaches both make more sense if you're really using it as a Memory Cache. In other words, if you're worried about the tradeoff between a memory limit and the cost of fetching data, these are great ways to determine when to evict items from the cache. So ask yourself:
Am I really trying to use this as a MemoryCache? Why do I even care if only 2000 items are loaded from the database?
If you are worried about the memory overhead, or if you are worried about the items getting out of date, there are other (better) ways to manage the cache than specifying a number of objects. If you've got some custom reason to keep a specific number of objects in a data structure, consider using a different class.
Another option would be to create a new MemoryCache provider which performs the object limit management for you. This would override some of the MemoryCache methods in such as add and remove, and automatically roll-off items once the arbitrary limit (e.g. 2000 objects) has been reached.
One such implementation may look like the following:
public class ObjectLimitMemoryCache : MemoryCache
{
private const int ObjectLimit = 2000;
private const string IndexKey = "ObjectLimitIndexKey";
public ObjectLimitMemoryCache(string name, NameValueCollection config)
: base (name, config)
{
}
new public static ObjectLimitMemoryCache Default { get { return new ObjectLimitMemoryCache(Guid.NewGuid().ToString(), new NameValueCollection());}}
public override bool Add(string key, Object value, DateTimeOffset absoluteExpiration, string region = null)
{
try
{
var indexedKeys = (List<string>)(base.Get(IndexKey) ?? new List<string>());
if (base.Add(key, value, absoluteExpiration))
{
string existingKey;
if (string.IsNullOrEmpty(existingKey = indexedKeys.FirstOrDefault(x=>x == key)))
{
indexedKeys.Add(key);
}
if (base.GetCount() > ObjectLimit)
{
base.Remove(indexedKeys.First());
indexedKeys.RemoveAt(0);
}
base.Add(IndexKey, indexedKeys, new DateTimeOffset(DateTime.Now.AddHours(2)));
return true;
}
return false;
}
catch
{
//Log something and other fancy stuff
throw;
}
}
}
This is untested code and meant solely to illustrate an example implementation of MemoryCache. Good luck!
I experience strange memory leak in computation expensive content-based image retrieval (CBIR) .NET application
The concept is that there is service class with thread loop which captures images from some source and then passes them to image tagging thread for annotation.
Image tags are queried from repository by the service class at specified time intervals and stored in its in-memory cache (Dictionary) to avoid frequent db hits.
The classes in the project are:
class Tag
{
public Guid Id { get; set; } // tag id
public string Name { get; set; } // tag name: e.g. 'sky','forest','road',...
public byte[] Jpeg { get; set; } // tag jpeg image patch sample
}
class IRepository
{
public IEnumerable<Tag> FindAll();
}
class Service
{
private IDictionary<Guid, Tag> Cache { get; set; } // to avoid frequent db reads
// image capture background worker (ICBW)
// image annotation background worker (IABW)
}
class Image
{
public byte[] Jpeg { get; set; }
public IEnumerable<Tag> Tags { get; set; }
}
ICBW worker captures jpeg image from some image source and passes it to IABW worker for annotation. IABW worker first tries to update Cache if time has come and then annotates the image by some algorithm creating Image object and attaching Tags to it then storing it to annotation repository.
Service cache update snippet in IABW worker is:
IEnumerable<Tag> tags = repository.FindAll();
Cache.Clear();
tags.ForEach(t => Cache.Add(t.Id, t));
IABW is called many times a second and is pretty processor extensive.
While running it for days I found memory increase in task manager. Using Perfmon to watch for Process/Private Bytes and .NET Memory/Bytes in all heaps I found them both increasing over the time.
Experimenting with the application I found that Cache update is the problem. If it is not updated there is no problem with the mem increase. But if the Cache update is as frequent as once in 1-5 minutes application gets ouf of mem pretty fast.
What might be the reason of that mem leak? Image objects are created quite often containing references to Tag objects in Cache. I presume when the Cache dictionary is created those references somehow are not garbage collected in the future.
Does it need to explicitly null managed byte[] objects to avoid memory leak e.g. by implementing Tag, Image as IDisposable?
Edit: 4 aug 2001, addition of the buggy code snippet causing quick mem leak.
static void Main(string[] args)
{
while (!Console.KeyAvailable)
{
IEnumerable<byte[]> data = CreateEnumeration(100);
PinEntries(data);
Thread.Sleep(900);
Console.Write(String.Format("gc mem: {0}\r", GC.GetTotalMemory(true)));
}
}
static IEnumerable<byte[]> CreateEnumeration(int size)
{
Random random = new Random();
IList<byte[]> data = new List<byte[]>();
for (int i = 0; i < size; i++)
{
byte[] vector = new byte[12345];
random.NextBytes(vector);
data.Add(vector);
}
return data;
}
static void PinEntries(IEnumerable<byte[]> data)
{
var handles = data.Select(d => GCHandle.Alloc(d, GCHandleType.Pinned));
var ptrs = handles.Select(h => h.AddrOfPinnedObject());
IntPtr[] dataPtrs = ptrs.ToArray();
Thread.Sleep(100); // unmanaged function call taking byte** data
handles.ToList().ForEach(h => h.Free());
}
No, you don't need to set anything to null or dispose of anything if it's just memory as you've shown.
I suggest you get hold of a good profiler to work out where the leak is. Do you have anything non-memory-related that you might be failing to dispose of, e.g. loading a GDI+ image to get the bytes?
MSDN:
public IntPtr MaxWorkingSet { get; set; }
Gets or sets the maximum allowable
working set size for the associated
process. Property Value: The maximum
working set size that is allowed in
memory for the process, in bytes.
So, as far as I understand, I can limit amount of memory that can be used by a process. I've tried this, but with no luck..
Some code:
public class A
{
public void Do()
{
List<string> guids = new List<string>();
do
{
guids.Add(Guid.NewGuid().ToString());
Thread.Sleep(5);
} while (true);
}
}
public static class App
{
public static void Main()
{
Process.GetCurrentProcess().MaxWorkingSet = new IntPtr(2097152);
try
{
new A().Do();
}
catch (Exception e)
{
}
}
}
I'm expecting OutOfMemory exception after the limit of 2mb is reached, but nothing happens.. If I open Task Manager I can see that the amount of memory my application uses is growing continiously without any limits.
What am I doing wrong?
Thanks in advance
No, this doesn't limit the amount of memory used by the process. It's simply a wrapper around SetProcessWorkingSetSize which a) is a recommendation, and b) limits the working set of the process, which is the amount of physical memory (RAM) this process can consume.
It will absolutely not cause an out of memory exception in that process, even if it allocates significantly more than what you set the MaxWorkingSet property to.
There is an alternative to what you're trying to do -- the Win32 Job Object API. There's a managed wrapper for that on Codeplex (http://jobobjectwrapper.codeplex.com/) to which I contributed. It allows you to create a process and limit the amount of memory this process can use.