I am using the code below to cache items. It's pretty basic.
The issue I have is that every time it caches an item, section of the code locks. So with roughly a million items arriving every hour or so, this is a problem.
I've tried creating a dictionary of static lock objects per cacheKey, so that locking is granular, but that in itself becomes an issue with managing expiration of them, etc...
Is there a better way to implement minimal locking?
private static readonly object cacheLock = new object();
public static T GetFromCache<T>(string cacheKey, Func<T> GetData) where T : class {
// Returns null if the string does not exist, prevents a race condition
// where the cache invalidates between the contains check and the retrieval.
T cachedData = MemoryCache.Default.Get(cacheKey) as T;
if (cachedData != null) {
return cachedData;
}
lock (cacheLock) {
// Check to see if anyone wrote to the cache while we where
// waiting our turn to write the new value.
cachedData = MemoryCache.Default.Get(cacheKey) as T;
if (cachedData != null) {
return cachedData;
}
// The value still did not exist so we now write it in to the cache.
cachedData = GetData();
MemoryCache.Default.Set(cacheKey, cachedData, new CacheItemPolicy(...));
return cachedData;
}
}
You may want to consider using ReaderWriterLockSlim, which you can obtain write lock only when needed.
Using cacheLock.EnterReadLock(); and cacheLock.EnterWriteLock(); should greatly improve the performance.
That link I gave even have an example of a cache, exactly what you need, I copy here:
public class SynchronizedCache
{
private ReaderWriterLockSlim cacheLock = new ReaderWriterLockSlim();
private Dictionary<int, string> innerCache = new Dictionary<int, string>();
public int Count
{ get { return innerCache.Count; } }
public string Read(int key)
{
cacheLock.EnterReadLock();
try
{
return innerCache[key];
}
finally
{
cacheLock.ExitReadLock();
}
}
public void Add(int key, string value)
{
cacheLock.EnterWriteLock();
try
{
innerCache.Add(key, value);
}
finally
{
cacheLock.ExitWriteLock();
}
}
public bool AddWithTimeout(int key, string value, int timeout)
{
if (cacheLock.TryEnterWriteLock(timeout))
{
try
{
innerCache.Add(key, value);
}
finally
{
cacheLock.ExitWriteLock();
}
return true;
}
else
{
return false;
}
}
public AddOrUpdateStatus AddOrUpdate(int key, string value)
{
cacheLock.EnterUpgradeableReadLock();
try
{
string result = null;
if (innerCache.TryGetValue(key, out result))
{
if (result == value)
{
return AddOrUpdateStatus.Unchanged;
}
else
{
cacheLock.EnterWriteLock();
try
{
innerCache[key] = value;
}
finally
{
cacheLock.ExitWriteLock();
}
return AddOrUpdateStatus.Updated;
}
}
else
{
cacheLock.EnterWriteLock();
try
{
innerCache.Add(key, value);
}
finally
{
cacheLock.ExitWriteLock();
}
return AddOrUpdateStatus.Added;
}
}
finally
{
cacheLock.ExitUpgradeableReadLock();
}
}
public void Delete(int key)
{
cacheLock.EnterWriteLock();
try
{
innerCache.Remove(key);
}
finally
{
cacheLock.ExitWriteLock();
}
}
public enum AddOrUpdateStatus
{
Added,
Updated,
Unchanged
};
~SynchronizedCache()
{
if (cacheLock != null) cacheLock.Dispose();
}
}
I don't know how MemoryCache.Default is implemented, or whether or not you have control over it.
But in general, prefer using ConcurrentDictionary over Dictionary with lock in a multi threaded environment.
GetFromCache would just become
ConcurrentDictionary<string, T> cache = new ConcurrentDictionary<string, T>();
...
cache.GetOrAdd("someKey", (key) =>
{
var data = PullDataFromDatabase(key);
return data;
});
There are two more things to take care about.
Expiry
Instead of saving T as the value of the dictionary, you can define a type
struct CacheItem<T>
{
public T Item { get; set; }
public DateTime Expiry { get; set; }
}
And store the cache as a CacheItem with a defined expiry.
cache.GetOrAdd("someKey", (key) =>
{
var data = PullDataFromDatabase(key);
return new CacheItem<T>() { Item = data, Expiry = DateTime.UtcNow.Add(TimeSpan.FromHours(1)) };
});
Now you can implement expiration in an asynchronous thread.
Timer expirationTimer = new Timer(ExpireCache, null, 60000, 60000);
...
void ExpireCache(object state)
{
var needToExpire = cache.Where(c => DateTime.UtcNow >= c.Value.Expiry).Select(c => c.Key);
foreach (var key in needToExpire)
{
cache.TryRemove(key, out CacheItem<T> _);
}
}
Once a minute, you search for all cache entries that need to be expired, and remove them.
"Locking"
Using ConcurrentDictionary guarantees that simultaneous read/writes won't corrupt the dictionary or throw an exception.
But, you can still end up with a situation where two simultaneous reads cause you to fetch the data from the database twice.
One neat trick to solve this is to wrap the value of the dictionary with Lazy
ConcurrentDictionary<string, Lazy<CacheItem<T>>> cache = new ConcurrentDictionary<string, Lazy<CacheItem<T>>>();
...
var data = cache.GetOrData("someKey", key => new Lazy<CacheItem<T>>(() =>
{
var data = PullDataFromDatabase(key);
return new CacheItem<T>() { Item = data, Expiry = DateTime.UtcNow.Add(TimeSpan.FromHours(1)) };
})).Value;
Explanation
with GetOrAdd you might end up invoking the "get from database if not in cache" delegate multiple times in the case of simultaneous requests.
However, GetOrAdd will end up using only one of the values that the delegate returned, and by returning a Lazy, you guaranty that only one Lazy will get invoked.
Related
EDIT: I've updated my examples to use the https://github.com/StephenCleary/AsyncEx library. Still waiting for usable hints.
There are resources, which are identified by strings (for example files, URLs, etc.). I'm looking for a locking mechanism over the resources. I've found 2 different solutions, but each has its problems:
The first is using the ConcurrentDictionary class with AsyncLock:
using Nito.AsyncEx;
using System.Collections.Concurrent;
internal static class Locking {
private static ConcurrentDictionary<string, AsyncLock> mutexes
= new ConcurrentDictionary<string, AsyncLock>();
internal static AsyncLock GetMutex(string resourceLocator) {
return mutexes.GetOrAdd(
resourceLocator,
key => new AsyncLock()
);
}
}
Async usage:
using (await Locking.GetMutex("resource_string").LockAsync()) {
...
}
Synchronous usage:
using (Locking.GetMutex("resource_string").Lock()) {
...
}
This works safely, but the problem is that the dictionary grows larger and larger, and I don't see a thread-safe way to remove items from the dictionary when no one is waiting on a lock. (I also want to avoid global locks.)
My second solution hashes the string to a number between 0 and N - 1, and locks on these:
using Nito.AsyncEx;
using System.Collections.Concurrent;
internal static class Locking {
private const UInt32 BUCKET_COUNT = 4096;
private static ConcurrentDictionary<UInt32, AsyncLock> mutexes
= new ConcurrentDictionary<UInt32, AsyncLock>();
private static UInt32 HashStringToInt(string text) {
return ((UInt32)text.GetHashCode()) % BUCKET_COUNT;
}
internal static AsyncLock GetMutex(string resourceLocator) {
return mutexes.GetOrAdd(
HashStringToInt(resourceLocator),
key => new AsyncLock()
);
}
}
As one can see, the second solution only decreases the probability of collisions, but doesn't avoid them. My biggest fear is that it can cause deadlocks: The main strategy to avoid deadlocks is to always lock items in a specific order. But with this approach, different items can map to the same buckets in different order, like: (A->X, B->Y), (C->Y, D->X). So with this solution one cannot lock on more than one resource safely.
Is there a better solution? (I also welcome critics of the above 2 solutions.)
You could probably improve upon the first solution by removing a lock from the dictionary when it stops being in use. The removed locks could then be added to a small pool, so that the next time you need a lock you just grab one from the pool instead of creating a new one.
Update: Here is an implementation of this idea. It is based on SemaphoreSlims instead of Stephen Cleary's AsyncLocks, because a custom disposable is required in order to remove unused semaphores from the dictionary.
public class MultiLock<TKey>
{
private object Locker { get; } = new object();
private Dictionary<TKey, LockItem> Dictionary { get; }
private Queue<LockItem> Pool { get; }
private int PoolSize { get; }
public MultiLock(int poolSize = 10)
{
Dictionary = new Dictionary<TKey, LockItem>();
Pool = new Queue<LockItem>(poolSize);
PoolSize = poolSize;
}
public WaitResult Wait(TKey key,
int millisecondsTimeout = Timeout.Infinite,
CancellationToken cancellationToken = default)
{
var lockItem = GetLockItem(key);
bool acquired;
try
{
acquired = lockItem.Semaphore.Wait(millisecondsTimeout,
cancellationToken);
}
catch
{
ReleaseLockItem(lockItem, key);
throw;
}
return new WaitResult(this, lockItem, key, acquired);
}
public async Task<WaitResult> WaitAsync(TKey key,
int millisecondsTimeout = Timeout.Infinite,
CancellationToken cancellationToken = default)
{
var lockItem = GetLockItem(key);
bool acquired;
try
{
acquired = await lockItem.Semaphore.WaitAsync(millisecondsTimeout,
cancellationToken).ConfigureAwait(false);
}
catch
{
ReleaseLockItem(lockItem, key);
throw;
}
return new WaitResult(this, lockItem, key, acquired);
}
private LockItem GetLockItem(TKey key)
{
LockItem lockItem;
lock (Locker)
{
if (!Dictionary.TryGetValue(key, out lockItem))
{
if (Pool.Count > 0)
{
lockItem = Pool.Dequeue();
}
else
{
lockItem = new LockItem();
}
Dictionary.Add(key, lockItem);
}
lockItem.UsedCount += 1;
}
return lockItem;
}
private void ReleaseLockItem(LockItem lockItem, TKey key)
{
lock (Locker)
{
lockItem.UsedCount -= 1;
if (lockItem.UsedCount == 0)
{
if (Dictionary.TryGetValue(key, out var stored))
{
if (stored == lockItem) // Sanity check
{
Dictionary.Remove(key);
if (Pool.Count < PoolSize)
{
Pool.Enqueue(lockItem);
}
}
}
}
}
}
internal class LockItem
{
public SemaphoreSlim Semaphore { get; } = new SemaphoreSlim(1);
public int UsedCount { get; set; }
}
public struct WaitResult : IDisposable
{
private MultiLock<TKey> MultiLock { get; }
private LockItem LockItem { get; }
private TKey Key { get; }
public bool LockAcquired { get; }
internal WaitResult(MultiLock<TKey> multiLock, LockItem lockItem, TKey key,
bool acquired)
{
MultiLock = multiLock;
LockItem = lockItem;
Key = key;
LockAcquired = acquired;
}
void IDisposable.Dispose()
{
MultiLock.ReleaseLockItem(LockItem, Key);
LockItem.Semaphore.Release();
}
}
}
Usage example:
var multiLock = new MultiLock<string>();
using (await multiLock.WaitAsync("SomeKey"))
{
//...
}
The default pool size for unused semaphores is 10. The optimal value should be the number of the concurrent workers that are using the MultiLock instance.
I did a performance test in my PC, and 10 workers were able to acquire the lock asynchronously 500,000 times in total per second (20 different string identifiers were used).
I'd like to create a static Cached class for an ASP.NET MVC site for quick access to cached items like dropdown lists. It needs to have locking implemented so that when a key comes back empty it can be pulled from the repository while any other request threads wait on it to come back. As such, it needs per-method thread locking (versus a shared lock). My first thought was to use nameof as the lock for each method instead of creating a separate object to lock for each method. A simplified version would look something like...
public static class Cached
{
public static List<Country> GetCountriesList()
{
List<Country> cacheItem = null;
if (HttpContext.Current.Cache["CountriesList"] != null)
cacheItem = (List<Country>)HttpContext.Current.Cache["CountriesList"];
else
{
lock (nameof(GetCountriesList))
{
// Check once more in case it got stored while waiting on the lock
if (HttpContext.Current.Cache["CountriesList"] == null)
{
using (var repo = new Repository())
{
cacheItem = repo.SelectCountries();
HttpContext.Current.Cache.Insert("CountriesList", cacheItem, null, DateTime.Now.AddHours(2), TimeSpan.Zero);
}
}
else
cacheItem = (List<Country>)HttpContext.Current.Cache["CountriesList"];
}
}
return cacheItem;
}
public static List<State> GetStatesList()
{
List<State> cacheItem = null;
if (HttpContext.Current.Cache["StatesList"] != null)
cacheItem = (List<State>)HttpContext.Current.Cache["StatesList"];
else
{
lock (nameof(GetStatesList))
{
// Check once more in case it got stored while waiting on the lock
if (HttpContext.Current.Cache["StatesList"] == null)
{
using (var repo = new Repository())
{
cacheItem = repo.SelectStates();
HttpContext.Current.Cache.Insert("StatesList", cacheItem, null, DateTime.Now.AddHours(2), TimeSpan.Zero);
}
}
else
cacheItem = (List<State>)HttpContext.Current.Cache["StatesList"];
}
}
return cacheItem;
}
}
Is there anything glaringly wrong with an approach like this?
UPDATE:
Per the advice that it is a bad idea to lock on strings, I've changed it to a pattern that I found in SO's Opserver code that uses a ConcurrentDictionary to store a lock object per cache key. Is there anything wrong with the following:
public static class Cached
{
private static readonly ConcurrentDictionary<string, object> _cacheLocks = new ConcurrentDictionary<string, object>();
private const string KEY_COUNTRIES_LIST = "CountriesList";
public static List<Country> GetCountriesList()
{
List<Country> cacheItem = null;
var nullLoadLock = _cacheLocks.AddOrUpdate(KEY_COUNTRIES_LIST, k => new object(), (k, old) => old);
if (HttpContext.Current.Cache[KEY_COUNTRIES_LIST] != null)
cacheItem = (List<Country>)HttpContext.Current.Cache[KEY_COUNTRIES_LIST];
else
{
lock (nullLoadLock)
{
// Check once more in case it got stored while waiting on the lock
if (HttpContext.Current.Cache[KEY_COUNTRIES_LIST] == null)
{
using (var repo = new Repository())
{
cacheItem = repo.SelectCountries();
HttpContext.Current.Cache.Insert(KEY_COUNTRIES_LIST, cacheItem, null, DateTime.Now.AddHours(2), TimeSpan.Zero);
}
}
else
cacheItem = (List<Country>)HttpContext.Current.Cache[KEY_COUNTRIES_LIST];
}
}
return cacheItem;
}
private const string KEY_STATES_LIST = "StatesList";
public static List<State> GetStatesList()
{
List<State> cacheItem = null;
var nullLoadLock = _cacheLocks.AddOrUpdate(KEY_COUNTRIES_LIST, k => new object(), (k, old) => old);
if (HttpContext.Current.Cache[KEY_STATES_LIST] != null)
cacheItem = (List<State>)HttpContext.Current.Cache[KEY_STATES_LIST];
else
{
lock (nullLoadLock)
{
// Check once more in case it got stored while waiting on the lock
if (HttpContext.Current.Cache[KEY_STATES_LIST] == null)
{
using (var repo = new Repository())
{
cacheItem = repo.SelectStates();
HttpContext.Current.Cache.Insert(KEY_STATES_LIST, cacheItem, null, DateTime.Now.AddHours(2), TimeSpan.Zero);
}
}
else
cacheItem = (List<State>)HttpContext.Current.Cache[KEY_STATES_LIST];
}
}
return cacheItem;
}
}
Based on what you posted so far, I think you're over-thinking this. :) I don't see a need to populate yet another dictionary with your locking objects. Since you are using them in explicitly named methods, just declare them as fields as needed.
First, the advice to not lock on string values is sound, but based on the problem that two string values can appear identical while still being different objects. You could avoid that in your scenario by storing the appropriate string value in a const field:
public static class Cached
{
private const string _kcountries = "CountriesList";
private const string _kstates = "StatesList";
public static List<Country> GetCountriesList()
{
List<Country> cacheItem = (List<Country>)HttpContext.Current.Cache[_kcountries];
if (cacheItem == null)
{
lock (_kcountries)
{
// Check once more in case it got stored while waiting on the lock
cacheItem = (List<Country>)HttpContext.Current.Cache[_kcountries];
if (cacheItem == null)
{
using (var repo = new Repository())
{
cacheItem = repo.SelectCountries();
HttpContext.Current.Cache.Insert(_kcountries, cacheItem, null, DateTime.Now.AddHours(2), TimeSpan.Zero);
}
}
}
}
return cacheItem;
}
public static List<State> GetStatesList()
{
// Same as above, except using _kstates instead of _kcountries
}
}
Note that you shouldn't be using string literals throughout the code anyway. It's much better practice to define const fields to represent those values. So you kill two birds with one stone doing the above. :)
The only remaining problem is that you are still using a possibly-public value to lock, since the string literals are interned, and if the exact same string was used somewhere else, it would likely be the same interned value as well. This is of debatable concern; it's my preference to avoid doing so, to ensure no other code outside my control could take the same lock my code is trying to use, but there are those who feel such concerns are overblown. YMMV. :)
If you do care (as I do) about using the possibly-public value, then you can associate a unique object value instead of using the string reference:
public static class Cached
{
private const string _kcountriesKey = "CountriesList";
private const string _kstatesKey = "StatesList";
private static readonly object _kcountriesLock = new object();
private static readonly object _kstatesLock = new object();
public static List<Country> GetCountriesList()
{
List<Country> cacheItem = (List<Country>)HttpContext.Current.Cache[_kcountriesKey];
if (cacheItem == null)
{
lock (_kcountriesLock)
{
// Check once more in case it got stored while waiting on the lock
cacheItem = (List<Country>)HttpContext.Current.Cache[_kcountriesKey];
if (cacheItem == null)
{
using (var repo = new Repository())
{
cacheItem = repo.SelectCountries();
HttpContext.Current.Cache.Insert(_kcountriesKey, cacheItem, null, DateTime.Now.AddHours(2), TimeSpan.Zero);
}
}
}
}
return cacheItem;
}
// etc.
}
I.e. use the ...Key field for your cache (since it does require string values for keys) but the ...Lock field for locking (so that you are sure no code outside your control would have access to the object value used for the lock).
I'll note that you do have an opportunity to reduce the repetition in the code, by writing a single Get...() implementation that can be shared by your various types of data:
public static class Cached
{
private const string _kcountriesKey = "CountriesList";
private const string _kstatesKey = "StatesList";
private static readonly object _kcountriesLock = new object();
private static readonly object _kstatesLock = new object();
public static List<Country> GetCountriesList()
{
// Assuming SelectCountries() is in fact declared to return List<Country>
// then you should actually be able to omit the type parameter in the method
// call and let type inference figure it out. Same thing for the call to
// _GetCachedData<State>() in the GetStatesList() method.
return _GetCachedData<Country>(_kcountriesKey, _kcountriesLock, repo => repo.SelectCountries());
}
public static List<State> GetStatesList()
{
return _GetCachedData<State>(_kstatesKey, _kstatesLock, repo => repo.SelectStates());
}
private static List<T> _GetCachedData<T>(string key, object lockObject, Func<Repository, List<T>> selector)
{
List<T> cacheItem = (List<T>)HttpContext.Current.Cache[key];
if (cacheItem == null)
{
lock (lockObject)
{
// Check once more in case it got stored while waiting on the lock
cacheItem = (List<T>)HttpContext.Current.Cache[key];
if (cacheItem == null)
{
using (var repo = new Repository())
{
cacheItem = selector(repo);
HttpContext.Current.Cache.Insert(key, cacheItem, null, DateTime.Now.AddHours(2), TimeSpan.Zero);
}
}
}
}
return cacheItem;
}
// etc.
}
Finally, I'll note that since the underlying cache (i.e. System.Web.Caching.Cache) is thread-safe, you could just skip all of this altogether, and instead choose to blindly populate the cache if your item (the List<T> in question) isn't found. The only downside is that you in some cases could retrieve the same list more than once. The upside is that the code is a lot simpler.
I have read lots of information about page caching and partial page caching in a MVC application. However, I would like to know how you would cache data.
In my scenario I will be using LINQ to Entities (entity framework). On the first call to GetNames (or whatever the method is) I want to grab the data from the database. I want to save the results in cache and on the second call to use the cached version if it exists.
Can anyone show an example of how this would work, where this should be implemented (model?) and if it would work.
I have seen this done in traditional ASP.NET apps , typically for very static data.
Here's a nice and simple cache helper class/service I use:
using System.Runtime.Caching;
public class InMemoryCache: ICacheService
{
public T GetOrSet<T>(string cacheKey, Func<T> getItemCallback) where T : class
{
T item = MemoryCache.Default.Get(cacheKey) as T;
if (item == null)
{
item = getItemCallback();
MemoryCache.Default.Add(cacheKey, item, DateTime.Now.AddMinutes(10));
}
return item;
}
}
interface ICacheService
{
T GetOrSet<T>(string cacheKey, Func<T> getItemCallback) where T : class;
}
Usage:
cacheProvider.GetOrSet("cache key", (delegate method if cache is empty));
Cache provider will check if there's anything by the name of "cache id" in the cache, and if there's not, it will call a delegate method to fetch data and store it in cache.
Example:
var products=cacheService.GetOrSet("catalog.products", ()=>productRepository.GetAll())
Reference the System.Web dll in your model and use System.Web.Caching.Cache
public string[] GetNames()
{
string[] names = Cache["names"] as string[];
if(names == null) //not in cache
{
names = DB.GetNames();
Cache["names"] = names;
}
return names;
}
A bit simplified but I guess that would work. This is not MVC specific and I have always used this method for caching data.
I'm referring to TT's post and suggest the following approach:
Reference the System.Web dll in your model and use System.Web.Caching.Cache
public string[] GetNames()
{
var noms = Cache["names"];
if(noms == null)
{
noms = DB.GetNames();
Cache["names"] = noms;
}
return ((string[])noms);
}
You should not return a value re-read from the cache, since you'll never know if at that specific moment it is still in the cache. Even if you inserted it in the statement before, it might already be gone or has never been added to the cache - you just don't know.
So you add the data read from the database and return it directly, not re-reading from the cache.
For .NET 4.5+ framework
add reference: System.Runtime.Caching
add using statement:
using System.Runtime.Caching;
public string[] GetNames()
{
var noms = System.Runtime.Caching.MemoryCache.Default["names"];
if(noms == null)
{
noms = DB.GetNames();
System.Runtime.Caching.MemoryCache.Default["names"] = noms;
}
return ((string[])noms);
}
In the .NET Framework 3.5 and earlier versions, ASP.NET provided an in-memory cache implementation in the System.Web.Caching namespace. In previous versions of the .NET Framework, caching was available only in the System.Web namespace and therefore required a dependency on ASP.NET classes. In the .NET Framework 4, the System.Runtime.Caching namespace contains APIs that are designed for both Web and non-Web applications.
More info:
https://msdn.microsoft.com/en-us/library/dd997357(v=vs.110).aspx
https://learn.microsoft.com/en-us/dotnet/framework/performance/caching-in-net-framework-applications
Steve Smith did two great blog posts which demonstrate how to use his CachedRepository pattern in ASP.NET MVC. It uses the repository pattern effectively and allows you to get caching without having to change your existing code.
http://ardalis.com/Introducing-the-CachedRepository-Pattern
http://ardalis.com/building-a-cachedrepository-via-strategy-pattern
In these two posts he shows you how to set up this pattern and also explains why it is useful. By using this pattern you get caching without your existing code seeing any of the caching logic. Essentially you use the cached repository as if it were any other repository.
I have used it in this way and it works for me.
https://msdn.microsoft.com/en-us/library/system.web.caching.cache.add(v=vs.110).aspx
parameters info for system.web.caching.cache.add.
public string GetInfo()
{
string name = string.Empty;
if(System.Web.HttpContext.Current.Cache["KeyName"] == null)
{
name = GetNameMethod();
System.Web.HttpContext.Current.Cache.Add("KeyName", name, null, DateTime.Noew.AddMinutes(5), Cache.NoSlidingExpiration, CacheitemPriority.AboveNormal, null);
}
else
{
name = System.Web.HttpContext.Current.Cache["KeyName"] as string;
}
return name;
}
AppFabric Caching is distributed and an in-memory caching technic that stores data in key-value pairs using physical memory across multiple servers. AppFabric provides performance and scalability improvements for .NET Framework applications. Concepts and Architecture
Extending #Hrvoje Hudo's answer...
Code:
using System;
using System.Runtime.Caching;
public class InMemoryCache : ICacheService
{
public TValue Get<TValue>(string cacheKey, int durationInMinutes, Func<TValue> getItemCallback) where TValue : class
{
TValue item = MemoryCache.Default.Get(cacheKey) as TValue;
if (item == null)
{
item = getItemCallback();
MemoryCache.Default.Add(cacheKey, item, DateTime.Now.AddMinutes(durationInMinutes));
}
return item;
}
public TValue Get<TValue, TId>(string cacheKeyFormat, TId id, int durationInMinutes, Func<TId, TValue> getItemCallback) where TValue : class
{
string cacheKey = string.Format(cacheKeyFormat, id);
TValue item = MemoryCache.Default.Get(cacheKey) as TValue;
if (item == null)
{
item = getItemCallback(id);
MemoryCache.Default.Add(cacheKey, item, DateTime.Now.AddMinutes(durationInMinutes));
}
return item;
}
}
interface ICacheService
{
TValue Get<TValue>(string cacheKey, Func<TValue> getItemCallback) where TValue : class;
TValue Get<TValue, TId>(string cacheKeyFormat, TId id, Func<TId, TValue> getItemCallback) where TValue : class;
}
Examples
Single item caching (when each item is cached based on its ID because caching the entire catalog for the item type would be too intensive).
Product product = cache.Get("product_{0}", productId, 10, productData.getProductById);
Caching all of something
IEnumerable<Categories> categories = cache.Get("categories", 20, categoryData.getCategories);
Why TId
The second helper is especially nice because most data keys are not composite. Additional methods could be added if you use composite keys often. In this way you avoid doing all sorts of string concatenation or string.Formats to get the key to pass to the cache helper. It also makes passing the data access method easier because you don't have to pass the ID into the wrapper method... the whole thing becomes very terse and consistant for the majority of use cases.
Here's an improvement to Hrvoje Hudo's answer. This implementation has a couple of key improvements:
Cache keys are created automatically based on the function to update data and the object passed in that specifies dependencies
Pass in time span for any cache duration
Uses a lock for thread safety
Note that this has a dependency on Newtonsoft.Json to serialize the dependsOn object, but that can be easily swapped out for any other serialization method.
ICache.cs
public interface ICache
{
T GetOrSet<T>(Func<T> getItemCallback, object dependsOn, TimeSpan duration) where T : class;
}
InMemoryCache.cs
using System;
using System.Reflection;
using System.Runtime.Caching;
using Newtonsoft.Json;
public class InMemoryCache : ICache
{
private static readonly object CacheLockObject = new object();
public T GetOrSet<T>(Func<T> getItemCallback, object dependsOn, TimeSpan duration) where T : class
{
string cacheKey = GetCacheKey(getItemCallback, dependsOn);
T item = MemoryCache.Default.Get(cacheKey) as T;
if (item == null)
{
lock (CacheLockObject)
{
item = getItemCallback();
MemoryCache.Default.Add(cacheKey, item, DateTime.Now.Add(duration));
}
}
return item;
}
private string GetCacheKey<T>(Func<T> itemCallback, object dependsOn) where T: class
{
var serializedDependants = JsonConvert.SerializeObject(dependsOn);
var methodType = itemCallback.GetType();
return methodType.FullName + serializedDependants;
}
}
Usage:
var order = _cache.GetOrSet(
() => _session.Set<Order>().SingleOrDefault(o => o.Id == orderId)
, new { id = orderId }
, new TimeSpan(0, 10, 0)
);
public sealed class CacheManager
{
private static volatile CacheManager instance;
private static object syncRoot = new Object();
private ObjectCache cache = null;
private CacheItemPolicy defaultCacheItemPolicy = null;
private CacheEntryRemovedCallback callback = null;
private bool allowCache = true;
private CacheManager()
{
cache = MemoryCache.Default;
callback = new CacheEntryRemovedCallback(this.CachedItemRemovedCallback);
defaultCacheItemPolicy = new CacheItemPolicy();
defaultCacheItemPolicy.AbsoluteExpiration = DateTime.Now.AddHours(1.0);
defaultCacheItemPolicy.RemovedCallback = callback;
allowCache = StringUtils.Str2Bool(ConfigurationManager.AppSettings["AllowCache"]); ;
}
public static CacheManager Instance
{
get
{
if (instance == null)
{
lock (syncRoot)
{
if (instance == null)
{
instance = new CacheManager();
}
}
}
return instance;
}
}
public IEnumerable GetCache(String Key)
{
if (Key == null || !allowCache)
{
return null;
}
try
{
String Key_ = Key;
if (cache.Contains(Key_))
{
return (IEnumerable)cache.Get(Key_);
}
else
{
return null;
}
}
catch (Exception)
{
return null;
}
}
public void ClearCache(string key)
{
AddCache(key, null);
}
public bool AddCache(String Key, IEnumerable data, CacheItemPolicy cacheItemPolicy = null)
{
if (!allowCache) return true;
try
{
if (Key == null)
{
return false;
}
if (cacheItemPolicy == null)
{
cacheItemPolicy = defaultCacheItemPolicy;
}
String Key_ = Key;
lock (Key_)
{
return cache.Add(Key_, data, cacheItemPolicy);
}
}
catch (Exception)
{
return false;
}
}
private void CachedItemRemovedCallback(CacheEntryRemovedArguments arguments)
{
String strLog = String.Concat("Reason: ", arguments.RemovedReason.ToString(), " | Key-Name: ", arguments.CacheItem.Key, " | Value-Object: ", arguments.CacheItem.Value.ToString());
LogManager.Instance.Info(strLog);
}
}
I use two classes. First one the cache core object:
public class Cacher<TValue>
where TValue : class
{
#region Properties
private Func<TValue> _init;
public string Key { get; private set; }
public TValue Value
{
get
{
var item = HttpRuntime.Cache.Get(Key) as TValue;
if (item == null)
{
item = _init();
HttpContext.Current.Cache.Insert(Key, item);
}
return item;
}
}
#endregion
#region Constructor
public Cacher(string key, Func<TValue> init)
{
Key = key;
_init = init;
}
#endregion
#region Methods
public void Refresh()
{
HttpRuntime.Cache.Remove(Key);
}
#endregion
}
Second one is list of cache objects:
public static class Caches
{
static Caches()
{
Languages = new Cacher<IEnumerable<Language>>("Languages", () =>
{
using (var context = new WordsContext())
{
return context.Languages.ToList();
}
});
}
public static Cacher<IEnumerable<Language>> Languages { get; private set; }
}
I will say implementing Singleton on this persisting data issue can be a solution for this matter in case you find previous solutions much complicated
public class GPDataDictionary
{
private Dictionary<string, object> configDictionary = new Dictionary<string, object>();
/// <summary>
/// Configuration values dictionary
/// </summary>
public Dictionary<string, object> ConfigDictionary
{
get { return configDictionary; }
}
private static GPDataDictionary instance;
public static GPDataDictionary Instance
{
get
{
if (instance == null)
{
instance = new GPDataDictionary();
}
return instance;
}
}
// private constructor
private GPDataDictionary() { }
} // singleton
HttpContext.Current.Cache.Insert("subjectlist", subjectlist);
You can also try and use the caching built into ASP MVC:
Add the following attribute to the controller method you'd like to cache:
[OutputCache(Duration=10)]
In this case the ActionResult of this will be cached for 10 seconds.
More on this here
I am searching for right thread-safe collection (concurrent collection) for the following scenario:
I may have requests from an external source which generates GUIDs (so it is unique and non-recurring). I need to store (say the last 100 requests) and check if duplicate GUIDs are delivered or not. I may not save all GUIDs more than 100 or so due to some limitations.
Now the problem is that when this mechanism is used in a service, it must be bound to 100 items and searching based on GUIDs is vital.
I decided to use ConcurrentDictionary yet I doubt it is a good decision since I may change the keys after using the whole 100 slots. I may find a good mechanism to replace the oldest requests when dictionary is full.
Any idea is much appreciated.
A code snippet is provided to show my incomplete implementation
public static ConcurrentDictionary<string, TimedProto> IncidentsCreated = new ConcurrentDictionary<string, TimedProto>(20, 100);
private static bool AddTo_AddedIncidents(proto ReceivedIncident)
{
try
{
int OldestCounter = 0;
DateTime OldestTime = DateTime.Now;
if (IncidentsCreated.Count < 100)
{
TimedProto tp = new TimedProto();
tp.IncidentProto = ReceivedIncident;
tp.time = DateTime.Now;
IncidentsCreated.AddOrUpdate(ReceivedIncident.IncidentGUID, tp,
(s,i) => i);
return true;
}
else //array is full, a replace oldest mechanism is required
{
}
return true;
}
catch (Exception ex)
{
LogEvent("AddTo_AddedIncidents\n"+ex.ToString(), EventLogEntryType.Error);
return false;
}
}
public struct proto
{
public string IncidentGUID;
//other variables
}
public struct TimedProto
{
public proto IncidentProto;
public DateTime time;
}
Thanks
Try this one: http://ayende.com/blog/162529/trivial-lru-cache-impl?key=02e8069c-62f8-4042-a7d2-d93806369824&utm_source=feedburner&utm_medium=feed&utm_campaign=Feed%3A+AyendeRahien+%28Ayende+%40+Rahien%29
Your implementation is flawed since you do use DateTime which has a granularity of 15ms. This means that you can accidentally delete even your most recent guids if you have a high inflow.
public class LruCache<TKey, TValue>
{
private readonly int _capacity;
private readonly Stopwatch _stopwatch = Stopwatch.StartNew();
class Reference<T> where T : struct
{
public T Value;
}
private class Node
{
public TValue Value;
public volatile Reference<long> Ticks;
}
private readonly ConcurrentDictionary<TKey, Node> _nodes = new ConcurrentDictionary<TKey, Node>();
public LruCache(int capacity)
{
Debug.Assert(capacity > 10);
_capacity = capacity;
}
public void Set(TKey key, TValue value)
{
var node = new Node
{
Value = value,
Ticks = new Reference<long> { Value = _stopwatch.ElapsedTicks }
};
_nodes.AddOrUpdate(key, node, (_, __) => node);
if (_nodes.Count > _capacity)
{
foreach (var source in _nodes.OrderBy(x => x.Value.Ticks).Take(_nodes.Count / 10))
{
Node _;
_nodes.TryRemove(source.Key, out _);
}
}
}
public bool TryGet(TKey key, out TValue value)
{
Node node;
if (_nodes.TryGetValue(key, out node))
{
node.Ticks = new Reference<long> {Value = _stopwatch.ElapsedTicks};
value = node.Value;
return true;
}
value = default(TValue);
return false;
}
}
I would use a Circular Buffer for this - there are plenty of implementations around, including this one, and making a thread-safe wrapper for one of them wouldn't be hard.
With only 100 or so slots, looking up by key would be reasonably efficient, and inserting would be extremely efficient (no reallocation as old items are discarded and replaced by new ones).
I have this code:
class Program
{
static void Main(string[] args)
{
TestClass instanceOfClass = new TestClass();
while (true)
{
Thread threadTest = new Thread(new ParameterizedThreadStart(AddNewToClass));
threadTest.Start(instanceOfClass);
}
}
static void AddNewToClass(object parameter)
{
var instance = (TestClass)parameter;
while (true)
{
if (instance.Contains(1))
{
continue;
}
else
{
instance.AddNew(1);
}
}
}
}
class TestClass
{
public Dictionary<int, string> dictionary;
public TestClass()
{
dictionary = new Dictionary<int, string>();
}
public void AddNew(int test)
{
lock (dictionary)
{
dictionary.Add(test, "Test string");
}
}
public bool Contains(int test)
{
lock (dictionary)
{
if (dictionary.ContainsKey(test))
{
return true;
}
else
{
return false;
}
}
}
}
What I want to do, is to have several different threads that add/remove objects from a Dictionary. I tried running this and I get this exception:
An item with the same key has already been added.
Which seems extremely weird. As far as I know the lock statement should block the dictionary in question and TestClass.Contains(1) should always return true, and it is throwing an exception as it returned true more than once (therefore the exception).
Anyone knows why this might happen? thanks
Your Contains() method is atomic. So is your Add() method. AddNewToClass(), however, is not. One thread may get a result from Contains()...but there's no guarantee regarding when it might or might not be suspended (or resumed).
That's your race condition.
Your lock only protects the blocks that it surrounds - it is this that needs protection
static void AddNewToClass(object parameter)
{
var instance = (TestClass)parameter;
while (true)
{
if (instance.Contains(1))
{
continue;
}
else
{
instance.AddNew(1);
}
}
}
Between the if (instance.Contains(1)) and the instance.AddNew(1); you can get preempted.
If you went with something like instance.AddItemIfMissing(1);
public void AddItemIfMissing(int test)
{
lock (dictionary)
{
if (!dictionary.ContainsKey(test))
{
dictionary.Add(test, "Test string");
}
}
}
This would do what you want.
You have a racing condition. After you lock on, you need to check again if the dictionary already contains an item by the same key, since another thread might have added it before you acquired the lock. But why reinvent the wheel? There are numerous helper classes, like ConcurrentBag, in the Parallel Extensions library. Or use a well thought through Singleton Pattern.
static void AddNewToClass(object parameter)
{
var instance = (TestClass)parameter;
while (true)
{
if (instance.Contains(1))
{
continue;
} // **thread switch maybe happens here will cause your problem**
else
{
instance.AddNew(1);
}
}
}
So following is better
lock(instance)
{
if (instance.Contains(1))
{
continue;
} // **thread switch maybe happens here will cause your problem**
else
{
instance.AddNew(1);
}
}