This code seems to do a good job of caching async method results. I would like to add some sort of expiration to it. I have tried Tuple but I was not successful in getting it to fully work / compile.
private static readonly ConcurrentDictionary<object, SemaphoreSlim> _keyLocks = new ConcurrentDictionary<object, SemaphoreSlim>();
private static readonly ConcurrentDictionary<object, Tuple<List<UnitDTO>, DateTime>> _cache = new ConcurrentDictionary<object, Tuple<List<UnitDTO>, DateTime>>();
public async Task<string> GetSomethingAsync(string key)
{
string value;
// get the semaphore specific to this key
var keyLock = _keyLocks.GetOrAdd(key, x => new SemaphoreSlim(1));
await keyLock.WaitAsync();
try
{
// try to get value from cache
if (!_cache.TryGetValue(key, out value))
{
// if value isn't cached, get it the long way asynchronously
value = await GetSomethingTheLongWayAsync();
// cache value
_cache.TryAdd(key, value);
}
}
finally
{
keyLock.Release();
}
return value;
}
Classical approaches and quotations
From msdn, by Stephen Cleary
Asynchronous code is often used to initialize a resource that’s then
cached and shared. There isn’t a built-in type for this, but Stephen
Toub developed an AsyncLazy that acts like a merge of Task and
Lazy. The original type is described on his blog, and an
updated version is available in my AsyncEx library.
public class AsyncLazy<T> : Lazy<Task<T>>
{
public AsyncLazy(Func<T> valueFactory) :
base(() => Task.Factory.StartNew(valueFactory)) { }
public AsyncLazy(Func<Task<T>> taskFactory) :
base(() => Task.Factory.StartNew(() => taskFactory()).Unwrap()) { }
}
Context
Let’s say in our program we have one of these AsyncLazy instances:
static string LoadString() { … }
static AsyncLazy<string> m_data = new AsyncLazy<string>(LoadString);
Usage
Thus, we can write an asynchronous method that does:
string data = await m_data.Value;
The Lazy<T> would be appropriate, but unfortunately it seems to lack the input parameter to index the result. The same issue was solved here where it is explained how to cache the results from a long-running, resource-intensive method, in case it is not async
Back to your proposed solution
Before I show the main changes related to the cache management and specific to your proposed implementation, let me suggest a couple of marginal optimization options, based on the following concerns.
often with locks, when you access them they’re uncontended, and in
such cases you really want acquiring and releasing the lock to be as
low-overhead as possible; in other words, accessing uncontended locks
should involve a fast path
Since they're just performance optimization tricks, I will leave them commented in the code so that you can measure their effects in your specific situation before.
You need to test TryGetValue again after awaiting because another parallel process could have added that value in the meantime
You don't need to keep the lock while you're awaiting
This balance of overhead vs cache misses was already pointed out in a previous answer to a similar question.
Obviously, there's overhead keeping SemaphoreSlim objects around to
prevent cache misses so it may not be worth it depending on the use
case. But if guaranteeing no cache misses is important than this
accomplishes that.
My main answer: the cache management
Regarding the cache expiration, I would suggest to add the creation DateTime to the value of the Dictionary (i.e. the time when the value is returned from GetSomethingTheLongWayAsync) and consequently discard the cached value after a fixed time span.
Find a draft below
private static readonly ConcurrentDictionary<object, SemaphoreSlim> _keyLocks = new ConcurrentDictionary<object, SemaphoreSlim>();
private static readonly ConcurrentDictionary<object, Tuple<string, DateTime>> _cache = new ConcurrentDictionary<object, Tuple<string, DateTime>>();
private static bool IsExpiredDelete(Tuple<string, DateTime> value, string key)
{
bool _is_exp = (DateTime.Now - value.Item2).TotalMinutes > Expiration;
if (_is_exp)
{
_cache.TryRemove(key, out value);
}
return _is_exp;
}
public async Task<string> GetSomethingAsync(string key)
{
Tuple<string, DateTime> cached;
// get the semaphore specific to this key
var keyLock = _keyLocks.GetOrAdd(key, x => new SemaphoreSlim(1));
await keyLock.WaitAsync();
try
{
// try to get value from cache
if (!_cache.TryGetValue(key, out cached) || IsExpiredDelete(cached,key))
{
//possible performance optimization: measure it before uncommenting
//keyLock.Release();
string value = await GetSomethingTheLongWayAsync(key);
DateTime creation = DateTime.Now;
// in case of performance optimization
// get the semaphore specific to this key
//keyLock = _keyLocks.GetOrAdd(key, x => new SemaphoreSlim(1));
//await keyLock.WaitAsync();
bool notFound;
if (notFound = !_cache.TryGetValue(key, out cached) || IsExpiredDelete(cached, key))
{
cached = new Tuple<string, DateTime>(value, creation);
_cache.TryAdd(key, cached);
}
else
{
if (!notFound && cached.Item2 < creation)
{
cached = new Tuple<string, DateTime>(value, creation);
_cache.TryAdd(key, cached);
}
}
}
}
finally
{
keyLock.Release();
}
return cached?.Item1;
}
Please, adapt the above code to your specific needs.
Making it more generic
Finally you may want to generalize it a little bit.
By the way, notice that the Dictionary are not static since one could cache two different methods with the same signature.
public class Cached<FromT, ToT>
{
private Func<FromT, Task<ToT>> GetSomethingTheLongWayAsync;
public Cached (Func<FromT, Task<ToT>> _GetSomethingTheLongWayAsync, int expiration_min ) {
GetSomethingTheLongWayAsync = _GetSomethingTheLongWayAsync;
Expiration = expiration_min;
}
int Expiration = 1;
private ConcurrentDictionary<FromT, SemaphoreSlim> _keyLocks = new ConcurrentDictionary<FromT, SemaphoreSlim>();
private ConcurrentDictionary<FromT, Tuple<ToT, DateTime>> _cache = new ConcurrentDictionary<FromT, Tuple<ToT, DateTime>>();
private bool IsExpiredDelete(Tuple<ToT, DateTime> value, FromT key)
{
bool _is_exp = (DateTime.Now - value.Item2).TotalMinutes > Expiration;
if (_is_exp)
{
_cache.TryRemove(key, out value);
}
return _is_exp;
}
public async Task<ToT> GetSomethingAsync(FromT key)
{
Tuple<ToT, DateTime> cached;
// get the semaphore specific to this key
var keyLock = _keyLocks.GetOrAdd(key, x => new SemaphoreSlim(1));
await keyLock.WaitAsync();
try
{
// try to get value from cache
if (!_cache.TryGetValue(key, out cached) || IsExpiredDelete(cached, key))
{
//possible performance optimization: measure it before uncommenting
//keyLock.Release();
ToT value = await GetSomethingTheLongWayAsync(key);
DateTime creation = DateTime.Now;
// in case of performance optimization
// get the semaphore specific to this key
//keyLock = _keyLocks.GetOrAdd(key, x => new SemaphoreSlim(1));
//await keyLock.WaitAsync();
bool notFound;
if (notFound = !_cache.TryGetValue(key, out cached) || IsExpiredDelete(cached, key))
{
cached = new Tuple<ToT, DateTime>(value, creation);
_cache.TryAdd(key, cached);
}
else
{
if (!notFound && cached.Item2 < creation)
{
cached = new Tuple<ToT, DateTime>(value, creation);
_cache.TryAdd(key, cached);
}
}
}
}
finally
{
keyLock.Release();
}
return cached.Item1;
}
}
For a generic FromT an IEqualityComparer is needed for the Dictionary
Usage/Demo
private static async Task<string> GetSomethingTheLongWayAsync(int key)
{
await Task.Delay(15000);
Console.WriteLine("Long way for: " + key);
return key.ToString();
}
static void Main(string[] args)
{
Test().Wait();
}
private static async Task Test()
{
int key;
string val;
key = 1;
var cache = new Cached<int, string>(GetSomethingTheLongWayAsync, 1);
Console.WriteLine("getting " + key);
val = await cache.GetSomethingAsync(key);
Console.WriteLine("getting " + key + " resulted in " + val);
Console.WriteLine("getting " + key);
val = await cache.GetSomethingAsync(key);
Console.WriteLine("getting " + key + " resulted in " + val);
await Task.Delay(65000);
Console.WriteLine("getting " + key);
val = await cache.GetSomethingAsync(key);
Console.WriteLine("getting " + key + " resulted in " + val);
Console.ReadKey();
}
Sophisticated alternatives
There are also more advanced possibilities like the overload of GetOrAdd that takes a delegate and Lazy objects to ensure that a generator function is called only once (instead of semaphores and locks).
public class AsyncCache<FromT, ToT>
{
private Func<FromT, Task<ToT>> GetSomethingTheLongWayAsync;
public AsyncCache(Func<FromT, Task<ToT>> _GetSomethingTheLongWayAsync, int expiration_min)
{
GetSomethingTheLongWayAsync = _GetSomethingTheLongWayAsync;
Expiration = expiration_min;
}
int Expiration;
private ConcurrentDictionary<FromT, Tuple<Lazy<Task<ToT>>, DateTime>> _cache =
new ConcurrentDictionary<FromT, Tuple<Lazy<Task<ToT>>, DateTime>>();
private bool IsExpiredDelete(Tuple<Lazy<Task<ToT>>, DateTime> value, FromT key)
{
bool _is_exp = (DateTime.Now - value.Item2).TotalMinutes > Expiration;
if (_is_exp)
{
_cache.TryRemove(key, out value);
}
return _is_exp;
}
public async Task<ToT> GetSomethingAsync(FromT key)
{
var res = _cache.AddOrUpdate(key,
t => new Tuple<Lazy<Task<ToT>>, DateTime>(new Lazy<Task<ToT>>(
() => GetSomethingTheLongWayAsync(key)
)
, DateTime.Now) ,
(k,t) =>
{
if (IsExpiredDelete(t, k))
{
return new Tuple<Lazy<Task<ToT>>, DateTime>(new Lazy<Task<ToT>>(
() => GetSomethingTheLongWayAsync(k)
), DateTime.Now);
}
return t;
}
);
return await res.Item1.Value;
}
}
Same usage, just replace AsyncCache instead of Cached.
Related
I am using the code below to cache items. It's pretty basic.
The issue I have is that every time it caches an item, section of the code locks. So with roughly a million items arriving every hour or so, this is a problem.
I've tried creating a dictionary of static lock objects per cacheKey, so that locking is granular, but that in itself becomes an issue with managing expiration of them, etc...
Is there a better way to implement minimal locking?
private static readonly object cacheLock = new object();
public static T GetFromCache<T>(string cacheKey, Func<T> GetData) where T : class {
// Returns null if the string does not exist, prevents a race condition
// where the cache invalidates between the contains check and the retrieval.
T cachedData = MemoryCache.Default.Get(cacheKey) as T;
if (cachedData != null) {
return cachedData;
}
lock (cacheLock) {
// Check to see if anyone wrote to the cache while we where
// waiting our turn to write the new value.
cachedData = MemoryCache.Default.Get(cacheKey) as T;
if (cachedData != null) {
return cachedData;
}
// The value still did not exist so we now write it in to the cache.
cachedData = GetData();
MemoryCache.Default.Set(cacheKey, cachedData, new CacheItemPolicy(...));
return cachedData;
}
}
You may want to consider using ReaderWriterLockSlim, which you can obtain write lock only when needed.
Using cacheLock.EnterReadLock(); and cacheLock.EnterWriteLock(); should greatly improve the performance.
That link I gave even have an example of a cache, exactly what you need, I copy here:
public class SynchronizedCache
{
private ReaderWriterLockSlim cacheLock = new ReaderWriterLockSlim();
private Dictionary<int, string> innerCache = new Dictionary<int, string>();
public int Count
{ get { return innerCache.Count; } }
public string Read(int key)
{
cacheLock.EnterReadLock();
try
{
return innerCache[key];
}
finally
{
cacheLock.ExitReadLock();
}
}
public void Add(int key, string value)
{
cacheLock.EnterWriteLock();
try
{
innerCache.Add(key, value);
}
finally
{
cacheLock.ExitWriteLock();
}
}
public bool AddWithTimeout(int key, string value, int timeout)
{
if (cacheLock.TryEnterWriteLock(timeout))
{
try
{
innerCache.Add(key, value);
}
finally
{
cacheLock.ExitWriteLock();
}
return true;
}
else
{
return false;
}
}
public AddOrUpdateStatus AddOrUpdate(int key, string value)
{
cacheLock.EnterUpgradeableReadLock();
try
{
string result = null;
if (innerCache.TryGetValue(key, out result))
{
if (result == value)
{
return AddOrUpdateStatus.Unchanged;
}
else
{
cacheLock.EnterWriteLock();
try
{
innerCache[key] = value;
}
finally
{
cacheLock.ExitWriteLock();
}
return AddOrUpdateStatus.Updated;
}
}
else
{
cacheLock.EnterWriteLock();
try
{
innerCache.Add(key, value);
}
finally
{
cacheLock.ExitWriteLock();
}
return AddOrUpdateStatus.Added;
}
}
finally
{
cacheLock.ExitUpgradeableReadLock();
}
}
public void Delete(int key)
{
cacheLock.EnterWriteLock();
try
{
innerCache.Remove(key);
}
finally
{
cacheLock.ExitWriteLock();
}
}
public enum AddOrUpdateStatus
{
Added,
Updated,
Unchanged
};
~SynchronizedCache()
{
if (cacheLock != null) cacheLock.Dispose();
}
}
I don't know how MemoryCache.Default is implemented, or whether or not you have control over it.
But in general, prefer using ConcurrentDictionary over Dictionary with lock in a multi threaded environment.
GetFromCache would just become
ConcurrentDictionary<string, T> cache = new ConcurrentDictionary<string, T>();
...
cache.GetOrAdd("someKey", (key) =>
{
var data = PullDataFromDatabase(key);
return data;
});
There are two more things to take care about.
Expiry
Instead of saving T as the value of the dictionary, you can define a type
struct CacheItem<T>
{
public T Item { get; set; }
public DateTime Expiry { get; set; }
}
And store the cache as a CacheItem with a defined expiry.
cache.GetOrAdd("someKey", (key) =>
{
var data = PullDataFromDatabase(key);
return new CacheItem<T>() { Item = data, Expiry = DateTime.UtcNow.Add(TimeSpan.FromHours(1)) };
});
Now you can implement expiration in an asynchronous thread.
Timer expirationTimer = new Timer(ExpireCache, null, 60000, 60000);
...
void ExpireCache(object state)
{
var needToExpire = cache.Where(c => DateTime.UtcNow >= c.Value.Expiry).Select(c => c.Key);
foreach (var key in needToExpire)
{
cache.TryRemove(key, out CacheItem<T> _);
}
}
Once a minute, you search for all cache entries that need to be expired, and remove them.
"Locking"
Using ConcurrentDictionary guarantees that simultaneous read/writes won't corrupt the dictionary or throw an exception.
But, you can still end up with a situation where two simultaneous reads cause you to fetch the data from the database twice.
One neat trick to solve this is to wrap the value of the dictionary with Lazy
ConcurrentDictionary<string, Lazy<CacheItem<T>>> cache = new ConcurrentDictionary<string, Lazy<CacheItem<T>>>();
...
var data = cache.GetOrData("someKey", key => new Lazy<CacheItem<T>>(() =>
{
var data = PullDataFromDatabase(key);
return new CacheItem<T>() { Item = data, Expiry = DateTime.UtcNow.Add(TimeSpan.FromHours(1)) };
})).Value;
Explanation
with GetOrAdd you might end up invoking the "get from database if not in cache" delegate multiple times in the case of simultaneous requests.
However, GetOrAdd will end up using only one of the values that the delegate returned, and by returning a Lazy, you guaranty that only one Lazy will get invoked.
I have MVC application in which action methods must be executed on certain order. Recently, I am having some strange issues and I suppose that it is due to the fact that I do not do any thread synchronization. I have barely worked with multithreading and I don't know much about it. I tried to implement some kind of locking where I have to lock according to Id. So I implemented class like below to get required lock objects.
public class ReportLockProvider
: IReportLockProvider
{
readonly ConcurrentDictionary<long, object> lockDictionary
= new ConcurrentDictionary<long, object>();
public object ProvideLockObject(long reportId)
{
return lockDictionary.GetOrAdd(reportId, new object());
}
}
I tried to use this as below:
ReportLockProvider lockProvider = new ReportLockProvider();
public async ActionResult MyAction(long reportId)
{
lock(lockProvider.ProvideLockObject(reportId))
{
// Some operations
await Something();
// Some operation
}
}
I hoped that it would work, but it event didn't compiled because I have used await inside lock body. I have searched a bit and came across to SemaphoreSlim in this answer. Now, the problem is that I have to get lock object according to Id. How can I do this? Is it OK to create multiple SemaphoreSlim objects? Is it OK if I modify code like below :
public class ReportLockProvider
: IReportLockProvider
{
readonly ConcurrentDictionary<long, SemaphoreSlim> lockDictionary
= new ConcurrentDictionary<long, SemaphoreSlim>();
public SemaphoreSlim ProvideLockObject(long reportId)
{
return lockDictionary.GetOrAdd(reportId, new SemaphoreSlim(1, 1));
}
}
public async ActionResult MyAction(long reportId)
{
var lockObject = ReportLockProvider.ProvideLockObject(reportId);
await lockObject.WaitAsync();
try
{
// Some operations
await Something();
// Some operation
}
finally
{
lockObject.Release();
}
}
The other question is that, can I use SemaphoreSlim in non-async methods? Is there any better option?
I think you are missing a static keyword in front of your lockDictionary, but it depends on how you instanciate the provider.
Here is a sample with a little change code I cooked up in LinqPad:
async Task Main()
{
ReportLockProvider reportLockProvider = new ReportLockProvider();
List<Task> tasks = new List<Task>(10);
for (long i = 1; i <= 5; i++) {
var local = i;
tasks.Add(Task.Run(() => Enter(local) ));
tasks.Add(Task.Run(() => Enter(local) ));
}
async Task Enter(long id)
{
Console.WriteLine(id + " waiting to enter");
await reportLockProvider.WaitAsync(id);
Console.WriteLine(id + " entered!");
Thread.Sleep(1000 * (int)id);
Console.WriteLine(id + " releasing");
reportLockProvider.Release(id);
}
await Task.WhenAll(tasks.ToArray());
}
public class ReportLockProvider
{
static readonly ConcurrentDictionary<long, SemaphoreSlim> lockDictionary = new ConcurrentDictionary<long, SemaphoreSlim>();
public async Task WaitAsync(long reportId)
{
await lockDictionary.GetOrAdd(reportId, new SemaphoreSlim(1, 1)).WaitAsync();
}
public void Release(long reportId)
{
SemaphoreSlim semaphore;
if (lockDictionary.TryGetValue(reportId, out semaphore))
{
semaphore.Release();
}
}
}
Let's say I have a method it gets data from server
Task<Result> GetDataFromServerAsync(...)
If there is an ongoing call in progress, I don't want to start a new request to server but wait for the original to finish.
Let's say I have
var result = await objet.GetDataFromServerAsync(...);
and in a different place, called almost at the same time I have a second call
var result2 = await objet.GetDataFromServerAsync(...);
I don't want the second to start a new request to server if the first didn't finish. I want both calls to get the same result as soon as first call finish. This is a proof of concept, I have options but I wanted to see how easy it's to do this.
Here is a quick example using Lazy<Task<T>>:
var lazyGetDataFromServer = new Lazy<Task<Result>>
(() => objet.GetDataFromServerAsync(...));
var result = await lazyGetDataFromServer.Value;
var result2 = await lazyGetDataFromServer.Value;
It doesn't matter if these 2 awaits are done from separate threads as Lazy<T> is thread-safe, so result2 if ran second will still wait and use the same output from result.
Using the code from here you can wrap this up in a class called AsyncLazy<T>, and add a custom GetAwaiter so that you can just await it without the need to do .Value, very tidy =)
You can use anything for syncrhonization in your method.
For example, I used SemaphoreSlim:
public class PartyMaker
{
private bool _isProcessing;
private readonly SemaphoreSlim _slowStuffSemaphore = new SemaphoreSlim(1, 1);
private DateTime _something;
public async Task<DateTime> ShakeItAsync()
{
try
{
var needNewRequest = !_isProcessing;
await _slowStuffSemaphore.WaitAsync().ConfigureAwait(false);
if (!needNewRequest) return _something;
_isProcessing = true;
_something = await ShakeItSlowlyAsync().ConfigureAwait(false);
return _something;
}
finally
{
_isProcessing = false;
_slowStuffSemaphore.Release();
}
}
private async Task<DateTime> ShakeItSlowlyAsync()
{
await Task.Delay(TimeSpan.FromSeconds(1)).ConfigureAwait(false);
return DateTime.UtcNow;
}
}
Usage:
var maker = new PartyMaker();
var tasks = new[] {maker.ShakeItAsync(), maker.ShakeItAsync()};
Task.WaitAll(tasks);
foreach (var task in tasks)
{
Console.WriteLine(task.Result);
}
Console.WriteLine(maker.ShakeItAsync().Result);
Here is result:
17.01.2017 22:28:39
17.01.2017 22:28:39
17.01.2017 22:28:41
UPD
With this modification you can call async methods with args:
public class PartyMaker
{
private readonly SemaphoreSlim _slowStuffSemaphore = new SemaphoreSlim(1, 1);
private readonly ConcurrentDictionary<int, int> _requestCounts = new ConcurrentDictionary<int, int>();
private readonly ConcurrentDictionary<int, DateTime> _cache = new ConcurrentDictionary<int, DateTime>();
public async Task<DateTime> ShakeItAsync(Argument argument)
{
var key = argument.GetHashCode();
DateTime result;
try
{
if (!_requestCounts.ContainsKey(key))
{
_requestCounts[key] = 1;
}
else
{
++_requestCounts[key];
}
var needNewRequest = _requestCounts[key] == 1;
await _slowStuffSemaphore.WaitAsync().ConfigureAwait(false);
if (!needNewRequest)
{
_cache.TryGetValue(key, out result);
return result;
}
_cache.TryAdd(key, default(DateTime));
result = await ShakeItSlowlyAsync().ConfigureAwait(false);
_cache[key] = result;
return result;
}
finally
{
_requestCounts[key]--;
if (_requestCounts[key] == 0)
{
int temp;
_requestCounts.TryRemove(key, out temp);
_cache.TryRemove(key, out result);
}
_slowStuffSemaphore.Release();
}
}
private async Task<DateTime> ShakeItSlowlyAsync()
{
await Task.Delay(TimeSpan.FromSeconds(1)).ConfigureAwait(false);
return DateTime.UtcNow;
}
}
public class Argument
{
public Argument(int value)
{
Value = value;
}
public int Value { get; }
public override int GetHashCode()
{
return Value.GetHashCode();
}
}
I'd like to create a static Cached class for an ASP.NET MVC site for quick access to cached items like dropdown lists. It needs to have locking implemented so that when a key comes back empty it can be pulled from the repository while any other request threads wait on it to come back. As such, it needs per-method thread locking (versus a shared lock). My first thought was to use nameof as the lock for each method instead of creating a separate object to lock for each method. A simplified version would look something like...
public static class Cached
{
public static List<Country> GetCountriesList()
{
List<Country> cacheItem = null;
if (HttpContext.Current.Cache["CountriesList"] != null)
cacheItem = (List<Country>)HttpContext.Current.Cache["CountriesList"];
else
{
lock (nameof(GetCountriesList))
{
// Check once more in case it got stored while waiting on the lock
if (HttpContext.Current.Cache["CountriesList"] == null)
{
using (var repo = new Repository())
{
cacheItem = repo.SelectCountries();
HttpContext.Current.Cache.Insert("CountriesList", cacheItem, null, DateTime.Now.AddHours(2), TimeSpan.Zero);
}
}
else
cacheItem = (List<Country>)HttpContext.Current.Cache["CountriesList"];
}
}
return cacheItem;
}
public static List<State> GetStatesList()
{
List<State> cacheItem = null;
if (HttpContext.Current.Cache["StatesList"] != null)
cacheItem = (List<State>)HttpContext.Current.Cache["StatesList"];
else
{
lock (nameof(GetStatesList))
{
// Check once more in case it got stored while waiting on the lock
if (HttpContext.Current.Cache["StatesList"] == null)
{
using (var repo = new Repository())
{
cacheItem = repo.SelectStates();
HttpContext.Current.Cache.Insert("StatesList", cacheItem, null, DateTime.Now.AddHours(2), TimeSpan.Zero);
}
}
else
cacheItem = (List<State>)HttpContext.Current.Cache["StatesList"];
}
}
return cacheItem;
}
}
Is there anything glaringly wrong with an approach like this?
UPDATE:
Per the advice that it is a bad idea to lock on strings, I've changed it to a pattern that I found in SO's Opserver code that uses a ConcurrentDictionary to store a lock object per cache key. Is there anything wrong with the following:
public static class Cached
{
private static readonly ConcurrentDictionary<string, object> _cacheLocks = new ConcurrentDictionary<string, object>();
private const string KEY_COUNTRIES_LIST = "CountriesList";
public static List<Country> GetCountriesList()
{
List<Country> cacheItem = null;
var nullLoadLock = _cacheLocks.AddOrUpdate(KEY_COUNTRIES_LIST, k => new object(), (k, old) => old);
if (HttpContext.Current.Cache[KEY_COUNTRIES_LIST] != null)
cacheItem = (List<Country>)HttpContext.Current.Cache[KEY_COUNTRIES_LIST];
else
{
lock (nullLoadLock)
{
// Check once more in case it got stored while waiting on the lock
if (HttpContext.Current.Cache[KEY_COUNTRIES_LIST] == null)
{
using (var repo = new Repository())
{
cacheItem = repo.SelectCountries();
HttpContext.Current.Cache.Insert(KEY_COUNTRIES_LIST, cacheItem, null, DateTime.Now.AddHours(2), TimeSpan.Zero);
}
}
else
cacheItem = (List<Country>)HttpContext.Current.Cache[KEY_COUNTRIES_LIST];
}
}
return cacheItem;
}
private const string KEY_STATES_LIST = "StatesList";
public static List<State> GetStatesList()
{
List<State> cacheItem = null;
var nullLoadLock = _cacheLocks.AddOrUpdate(KEY_COUNTRIES_LIST, k => new object(), (k, old) => old);
if (HttpContext.Current.Cache[KEY_STATES_LIST] != null)
cacheItem = (List<State>)HttpContext.Current.Cache[KEY_STATES_LIST];
else
{
lock (nullLoadLock)
{
// Check once more in case it got stored while waiting on the lock
if (HttpContext.Current.Cache[KEY_STATES_LIST] == null)
{
using (var repo = new Repository())
{
cacheItem = repo.SelectStates();
HttpContext.Current.Cache.Insert(KEY_STATES_LIST, cacheItem, null, DateTime.Now.AddHours(2), TimeSpan.Zero);
}
}
else
cacheItem = (List<State>)HttpContext.Current.Cache[KEY_STATES_LIST];
}
}
return cacheItem;
}
}
Based on what you posted so far, I think you're over-thinking this. :) I don't see a need to populate yet another dictionary with your locking objects. Since you are using them in explicitly named methods, just declare them as fields as needed.
First, the advice to not lock on string values is sound, but based on the problem that two string values can appear identical while still being different objects. You could avoid that in your scenario by storing the appropriate string value in a const field:
public static class Cached
{
private const string _kcountries = "CountriesList";
private const string _kstates = "StatesList";
public static List<Country> GetCountriesList()
{
List<Country> cacheItem = (List<Country>)HttpContext.Current.Cache[_kcountries];
if (cacheItem == null)
{
lock (_kcountries)
{
// Check once more in case it got stored while waiting on the lock
cacheItem = (List<Country>)HttpContext.Current.Cache[_kcountries];
if (cacheItem == null)
{
using (var repo = new Repository())
{
cacheItem = repo.SelectCountries();
HttpContext.Current.Cache.Insert(_kcountries, cacheItem, null, DateTime.Now.AddHours(2), TimeSpan.Zero);
}
}
}
}
return cacheItem;
}
public static List<State> GetStatesList()
{
// Same as above, except using _kstates instead of _kcountries
}
}
Note that you shouldn't be using string literals throughout the code anyway. It's much better practice to define const fields to represent those values. So you kill two birds with one stone doing the above. :)
The only remaining problem is that you are still using a possibly-public value to lock, since the string literals are interned, and if the exact same string was used somewhere else, it would likely be the same interned value as well. This is of debatable concern; it's my preference to avoid doing so, to ensure no other code outside my control could take the same lock my code is trying to use, but there are those who feel such concerns are overblown. YMMV. :)
If you do care (as I do) about using the possibly-public value, then you can associate a unique object value instead of using the string reference:
public static class Cached
{
private const string _kcountriesKey = "CountriesList";
private const string _kstatesKey = "StatesList";
private static readonly object _kcountriesLock = new object();
private static readonly object _kstatesLock = new object();
public static List<Country> GetCountriesList()
{
List<Country> cacheItem = (List<Country>)HttpContext.Current.Cache[_kcountriesKey];
if (cacheItem == null)
{
lock (_kcountriesLock)
{
// Check once more in case it got stored while waiting on the lock
cacheItem = (List<Country>)HttpContext.Current.Cache[_kcountriesKey];
if (cacheItem == null)
{
using (var repo = new Repository())
{
cacheItem = repo.SelectCountries();
HttpContext.Current.Cache.Insert(_kcountriesKey, cacheItem, null, DateTime.Now.AddHours(2), TimeSpan.Zero);
}
}
}
}
return cacheItem;
}
// etc.
}
I.e. use the ...Key field for your cache (since it does require string values for keys) but the ...Lock field for locking (so that you are sure no code outside your control would have access to the object value used for the lock).
I'll note that you do have an opportunity to reduce the repetition in the code, by writing a single Get...() implementation that can be shared by your various types of data:
public static class Cached
{
private const string _kcountriesKey = "CountriesList";
private const string _kstatesKey = "StatesList";
private static readonly object _kcountriesLock = new object();
private static readonly object _kstatesLock = new object();
public static List<Country> GetCountriesList()
{
// Assuming SelectCountries() is in fact declared to return List<Country>
// then you should actually be able to omit the type parameter in the method
// call and let type inference figure it out. Same thing for the call to
// _GetCachedData<State>() in the GetStatesList() method.
return _GetCachedData<Country>(_kcountriesKey, _kcountriesLock, repo => repo.SelectCountries());
}
public static List<State> GetStatesList()
{
return _GetCachedData<State>(_kstatesKey, _kstatesLock, repo => repo.SelectStates());
}
private static List<T> _GetCachedData<T>(string key, object lockObject, Func<Repository, List<T>> selector)
{
List<T> cacheItem = (List<T>)HttpContext.Current.Cache[key];
if (cacheItem == null)
{
lock (lockObject)
{
// Check once more in case it got stored while waiting on the lock
cacheItem = (List<T>)HttpContext.Current.Cache[key];
if (cacheItem == null)
{
using (var repo = new Repository())
{
cacheItem = selector(repo);
HttpContext.Current.Cache.Insert(key, cacheItem, null, DateTime.Now.AddHours(2), TimeSpan.Zero);
}
}
}
}
return cacheItem;
}
// etc.
}
Finally, I'll note that since the underlying cache (i.e. System.Web.Caching.Cache) is thread-safe, you could just skip all of this altogether, and instead choose to blindly populate the cache if your item (the List<T> in question) isn't found. The only downside is that you in some cases could retrieve the same list more than once. The upside is that the code is a lot simpler.
I am searching for right thread-safe collection (concurrent collection) for the following scenario:
I may have requests from an external source which generates GUIDs (so it is unique and non-recurring). I need to store (say the last 100 requests) and check if duplicate GUIDs are delivered or not. I may not save all GUIDs more than 100 or so due to some limitations.
Now the problem is that when this mechanism is used in a service, it must be bound to 100 items and searching based on GUIDs is vital.
I decided to use ConcurrentDictionary yet I doubt it is a good decision since I may change the keys after using the whole 100 slots. I may find a good mechanism to replace the oldest requests when dictionary is full.
Any idea is much appreciated.
A code snippet is provided to show my incomplete implementation
public static ConcurrentDictionary<string, TimedProto> IncidentsCreated = new ConcurrentDictionary<string, TimedProto>(20, 100);
private static bool AddTo_AddedIncidents(proto ReceivedIncident)
{
try
{
int OldestCounter = 0;
DateTime OldestTime = DateTime.Now;
if (IncidentsCreated.Count < 100)
{
TimedProto tp = new TimedProto();
tp.IncidentProto = ReceivedIncident;
tp.time = DateTime.Now;
IncidentsCreated.AddOrUpdate(ReceivedIncident.IncidentGUID, tp,
(s,i) => i);
return true;
}
else //array is full, a replace oldest mechanism is required
{
}
return true;
}
catch (Exception ex)
{
LogEvent("AddTo_AddedIncidents\n"+ex.ToString(), EventLogEntryType.Error);
return false;
}
}
public struct proto
{
public string IncidentGUID;
//other variables
}
public struct TimedProto
{
public proto IncidentProto;
public DateTime time;
}
Thanks
Try this one: http://ayende.com/blog/162529/trivial-lru-cache-impl?key=02e8069c-62f8-4042-a7d2-d93806369824&utm_source=feedburner&utm_medium=feed&utm_campaign=Feed%3A+AyendeRahien+%28Ayende+%40+Rahien%29
Your implementation is flawed since you do use DateTime which has a granularity of 15ms. This means that you can accidentally delete even your most recent guids if you have a high inflow.
public class LruCache<TKey, TValue>
{
private readonly int _capacity;
private readonly Stopwatch _stopwatch = Stopwatch.StartNew();
class Reference<T> where T : struct
{
public T Value;
}
private class Node
{
public TValue Value;
public volatile Reference<long> Ticks;
}
private readonly ConcurrentDictionary<TKey, Node> _nodes = new ConcurrentDictionary<TKey, Node>();
public LruCache(int capacity)
{
Debug.Assert(capacity > 10);
_capacity = capacity;
}
public void Set(TKey key, TValue value)
{
var node = new Node
{
Value = value,
Ticks = new Reference<long> { Value = _stopwatch.ElapsedTicks }
};
_nodes.AddOrUpdate(key, node, (_, __) => node);
if (_nodes.Count > _capacity)
{
foreach (var source in _nodes.OrderBy(x => x.Value.Ticks).Take(_nodes.Count / 10))
{
Node _;
_nodes.TryRemove(source.Key, out _);
}
}
}
public bool TryGet(TKey key, out TValue value)
{
Node node;
if (_nodes.TryGetValue(key, out node))
{
node.Ticks = new Reference<long> {Value = _stopwatch.ElapsedTicks};
value = node.Value;
return true;
}
value = default(TValue);
return false;
}
}
I would use a Circular Buffer for this - there are plenty of implementations around, including this one, and making a thread-safe wrapper for one of them wouldn't be hard.
With only 100 or so slots, looking up by key would be reasonably efficient, and inserting would be extremely efficient (no reallocation as old items are discarded and replaced by new ones).