Develop Reference

Is it possible to avoid caching when calling the extension method GetOrCreateAsync on an IMemoryCache?

I'm using an IMemoryCache to cache a token retrieved from an Identity server.
In the past I've used the GetOrCreateAsync extension method available in the Microsoft.Extensions.Caching.Abstractions library.
It is very helpful because I can define the function and the expiration date at the same time.
However with a token, I won't know the expires in x amount of seconds value until the request is finished.
I want to account for a use case of the value not existing by not caching the token at all.
I have tried the following
var token = await this.memoryCache.GetOrCreateAsync<string>("SomeKey", async cacheEntry =>
{
var jwt = await GetTokenFromServer();
var tokenHasValidExpireValue = int.TryParse(jwt.ExpiresIn, out int tokenExpirationSeconds);
if (tokenHasValidExpireValue)
{
cacheEntry.AbsoluteExpirationRelativeToNow = TimeSpan.FromSeconds(tokenExpirationSeconds);
}
else // Do not cache value. Just return it.
{
cacheEntry.AbsoluteExpirationRelativeToNow = TimeSpan.FromSeconds(0); //Exception thrown. Value needs to be positive.
}
return jwt.token;
}
As you can see, an exception is thrown when I try to set an expiration of no time TimeSpan.FromSeconds(0).
Is there a way around this besides calling the Get and Set methods separately?
I would like to use the GetOrCreateAsync method if possible.

You can't actually accomplish this with the current extension because it will always create an entry BEFORE calling the factory method. That said, you can encapsulate the behavior in your own extension in a manner which feels very similar to GetOrCreateAsync.
public static class CustomMemoryCacheExtensions
{
public static async Task<TItem> GetOrCreateIfValidTimestampAsync<TItem>(
this IMemoryCache cache, object key, Func<Task<(int, TItem)>> factory)
{
if (!cache.TryGetValue(key, out object result))
{
(int tokenExpirationSeconds, TItem factoryResult) =
await factory().ConfigureAwait(false);
if (tokenExpirationSeconds <= 0)
{
// if the factory method did not return a positive timestamp,
// return the data without caching.
return factoryResult;
}
// since we have a valid timestamp:
// 1. create a cache entry
// 2. Set the result
// 3. Set the timestamp
using ICacheEntry entry = cache.CreateEntry(key);
entry.Value = result;
entry.AbsoluteExpirationRelativeToNow = TimeSpan.FromSeconds(tokenExpirationSeconds);
}
return (TItem)result;
}
}
You can then call your extension method in a very similar manner:
var memoryCache = new MemoryCache(new MemoryCacheOptions());
var token = await memoryCache.GetOrCreateIfValidTimestampAsync<string>("SomeKey", async () =>
{
var jwt = await GetTokenFromServer();
var tokenHasValidExpireValue = int.TryParse(jwt.ExpiresIn, out int tokenExpirationSeconds);
return (tokenExpirationSeconds, jwt.token);
}

I have the need for this and simply set the expiration time to now, I assume you can set it to past time as well just to be sure:
// Don't wanna cache if this is the result
if (key == null || key.Expiration < DateTime.UtcNow)
{
entry.AbsoluteExpiration = DateTimeOffset.Now;
return null;
}

Maybe something in these lines:
await _memoryCache.GetOrCreateAsync("key",
async entry =>
{
var value = // get your value here;
entry.AbsoluteExpiration = value != null
? DateTimeOffset.UtcNow.AddMinutes(5)
: DateTimeOffset.UtcNow; // don't cache nulls
return value;
});

Simple counter increment in Asp.net Core

In my controller, I want to count the number of bad attempts to login.
else
{
// Increment counter by 1
// check if counter == 3
// ban user
logonAttempt++;
if (logonAttempt >= MAX_LOGON_ATTEMPT)
{
ModelState.AddModelError("", "This account has been locked. Please contact the help desk for further support.");
} else
{
ModelState.AddModelError("", "You have entered an invalid username or password.");
}
}
Every time the user clicks "submit" and the code enters into this else statement, the logonAttempt resets to 0.
Is there anyway to prevent it from resetting? for the users session?

I think you are misunderstanding some stuff.
You can count in the session, but the new architecture patterns tend to work as session less.
The lifecycle should be, request, instantiate something really small, respond, and then release the instance to start working lightweight.
One thing you can do is just update the cookie stored with the validation token or method you are using for login.
My advice is just to check this framework, really flexible, where all these concerns are already solved.
https://learn.microsoft.com/en-us/aspnet/core/security/authentication/identity?view=aspnetcore-3.1&tabs=visual-studio

Maybe you can do it in a way like this:
// my class
private static XDic<DateTime, XDic<string, int>> daily_reg_attemps = new XDic<DateTime,XDic<string, int>>();
const int limit = 20;
public bool is_banned(string key,HttpRequestMessage request)
{
//handle attempt monitoring
var today = DateTime.Today.Date;
if (!daily_reg_attemps.ContainsKey(today))
daily_reg_attemps.Add(today, new XDic<string, int>());
var reg_attemps = daily_reg_attemps[today];
//handle attempts
var ip = GetClientIp(request);
if (!reg_attemps.ContainsKey(ip)) reg_attemps.Add(ip, 0);
if (!reg_attemps.ContainsKey(key)) reg_attemps.Add(key, 0);
//prevent localhost
if (!request.RequestUri.Host.Contains("localhost"))
{
reg_attemps[ip]++;
reg_attemps[key]++;
}
return (reg_attemps[ip] > limit || reg_attemps[key] > limit);
}
//my custom dictionary
public class XDic<TKey, TValue> : Dictionary<TKey, TValue>
{
public virtual void Add(TKey key, TValue value, bool updateIfExist = true)
{
if (key == null)
{
throw new ArgumentException("Key parameter is Null.");
}
if (base.ContainsKey(key))
{
if (updateIfExist)
{
base[key] = value;
}
else {
throw new ArgumentException("Error")
}
}
else {
base.Add(key, value);
}
}

recursive call in stack trace but nothing in code C#

I have recently come across an issue in C#.Net app. The unmodified stack trace looks like below :
2018-09-12 21:08:31,596 [] [112] ERROR PLoggerFactory::RunLogic - Exception : System.Exception: Serialisation errorSystem.ArgumentException: An item with the same key has already been added.
at System.ThrowHelper.ThrowArgumentException(ExceptionResource resource)
at System.Collections.Generic.Dictionary`2.Insert(TKey key, TValue value, Boolean add)
at Block`2.GetConfigFromDB()
at Block`2.GetConfigFromDB()
at Block`2.Begin(IParcelGettable`1 P, Action`1 f)
at Run.<>c__DisplayClass45_0.<RunInNewThread>b__1()
In the above, GetConfigFromDB is called in a stack.
But I have verified the code, there is nothing recursive GetConfigFromDB in that. Is this possible?
Please let me know if the code of GetConfigFromDB is required, I will modify it and share.
-----EDIT ------- Code added
private Dictionary<string, object> GetConfigFromDB()
{
blockConfigJson = controller.BlockConfig.GetConfig(this.BlockInstanceId);
if (String.IsNullOrEmpty(blockConfigJson))
{
return new Dictionary<string, object>();
}
Dictionary<string, object> configDictionary = new Dictionary<string, object>();
try
{
configDictionary = JsonConvert.DeserializeObject<Dictionary<string, object>>(blockConfigJson);
foreach (var v in configDictionary)
{
var key = "__" + this.BlockInstanceId + "." + v.Key;
if (SharedConfig.ContainsKey(key))
{
SharedConfig[key] = v.Value;
}
else
{
SharedConfig.Add(key, v.Value);
}
if (v.Key.Trim() == "_extraInfo_")
{
dynamic extraInfo = JsonConvert.DeserializeObject(configDictionary["_extraInfo_"].ToString());
JsonConvert.DeserializeObject<List<Variable>>(extraInfo["Variables"].ToString());
Dictionary<string, string> _variablesTemp = new Dictionary<string, string>();
try
{
_variablesTemp = JsonConvert.DeserializeObject<Dictionary<string, string>>(extraInfo["Variables"].ToString());
}
catch (Exception ex)
{
mLogger.Debug("Variable parsing error " + ex);
}
List<Variable> _variables = new List<Variable>();
foreach (KeyValuePair<string, string> kyp in _variablesTemp)
{
_variables.Add(new Variable()
{
variableName = kyp.Key,
variableValue = kyp.Value
});
}
foreach (Variable _variable in _variables)
{
if (!SharedVariables.ContainsKey(_variable.variableName))
{
SharedVariables.Add(_variable.variableName, _variable.variableValue);
new Caching().Send(_variable.variableName, _EvaluateConfigValue(_variable.variableValue, this.blockConfig, this.SharedConfig, this.SharedVariables, this.propagatedConfig, this.propagatedVariables, this.ControlId));
}
}
}
}
}
catch (Exception ex)
{
configDictionary = new Dictionary<string, object>();
throw;
}
return configDictionary;
}

That stack trace shows your method calling Dictionary<,>.Insert(TKey, TValue, bool), but of course you never do. You can't, because that is a private method called by Dictionary<,>.Add, which you do call.
When optimisations are enabled, stack traces are not always 100% accurate. Especially when trivial methods are called, which almost certainly get inlined. Dictionary<,>.Add is such a trivial method: it literally does nothing else but call Dictionary<,>.Insert. It looks like enough information was recovered to determine that there was something between Dictionary<,>.Insert and GetConfigFromDB, but not what that something might be. Because nothing better is available, the name GetConfigFromDB is used a second time.

Locking with Constructors causing Null Reference?

So I encountered a strange issue today - I had a simple creation of an instance inside the critical section of a lock, and it would throw a null reference exception when I manually dragged the next line to execute. To illustrate:
public class SearchEngineOptimizationParser
{
protected static ConcurrentDictionary<string, SearchEngineOptimizationInfo> _referralInformation = null;
protected static DateTime _lastRecordingDate;
protected static object _lockRecordingObject = new object();
protected static Dictionary<string, string> _searchProviderLookups = null;
static SearchEngineOptimizationParser()
{
_referralInformation = new ConcurrentDictionary<string, SearchEngineOptimizationInfo>();
_lastRecordingDate = DateTime.Now;
_searchProviderLookups = new Dictionary<string, string>();
_searchProviderLookups.Add("google.com", "q");
_searchProviderLookups.Add("yahoo.com", "p");
_searchProviderLookups.Add("bing.com", "q");
}
public SearchEngineOptimizationParser()
{
}
public virtual void ParseReferrer(Uri requestUrl, NameValueCollection serverVariables, ISession session)
{
string corePath = requestUrl.PathAndQuery.SmartSplit('?')[0].ToLower();
string referrer = serverVariables["HTTP_REFERER"];
if (!string.IsNullOrWhiteSpace(referrer))
{
NameValueCollection queryString = HttpUtility.ParseQueryString(referrer);
string dictionaryKey = session.AffiliateID + "|" + corePath;
foreach (var searchProvider in _searchProviderLookups)
{
if (referrer.Contains(searchProvider.Key))
{
if (queryString[searchProvider.Value] != null)
{
string keywords = queryString[searchProvider.Value];
SearchEngineOptimizationInfo info = new SearchEngineOptimizationInfo
{
Count = 1,
CorePath = corePath,
AffiliateId = session.AffiliateID,
Keywords = keywords
};
_referralInformation.AddOrUpdate(dictionaryKey, info, (key, oldValue) =>
{
oldValue.Count++;
return oldValue;
});
break;
}
}
}
}
if (DateTime.Now > _lastRecordingDate.AddHours(1))
{
lock (_lockRecordingObject)
{
if (DateTime.Now > _lastRecordingDate.AddHours(1))
{
SearchEngineKeywordRepository repository = new SearchEngineKeywordRepository();
List<KeyValuePair<string, SearchEngineOptimizationInfo>> currentInfo = _referralInformation.ToList();
Action logData = () =>
{
foreach (var item in currentInfo)
repository.LogKeyword(item.Value);
};
Thread logThread = new Thread(new ThreadStart(logData));
logThread.Start();
_lastRecordingDate = DateTime.Now;
_referralInformation.Clear();
}
}
}
}
EDIT: Updated Real Object
public class SearchEngineKeywordRepository
{
public virtual void LogKeyword(SearchEngineOptimizationInfo keywordInfo)
{
LogSearchEngineKeywords procedure = new LogSearchEngineKeywords();
procedure.Execute(keywordInfo.CorePath, keywordInfo.AffiliateId, keywordInfo.Keywords, keywordInfo.Count);
}
}
The general pattern being that I want to do this 'something' only every hour (in the context of a website application that gets a lot of traffic). I would breakpoint my first if statement, and then step the next line to execute inside the second if statement. When doing so, the act of initializing the SomeObject instance would cause a null reference exception. It had a completely 100% default constructor - I didn't even specify one.
However, when I let the code go through naturally, it would execute without problem. For some reason, it seems that when I skipped over the lock call into the critical section to just test run that code, it caused all kinds of errors.
I'm curious to know why that is; I understand the lock keyword is just syntactic sugar for a Monitor.Enter(o) try / finally block, but that seems to be that when invoking the constructor, something else was happening.
Anyone have any ideas?
EDIT: I've added the actual code to this. I'm able to reproduce this at will, but I still don't understand why this is happening. I've tried copying this code to another solution and the problem does not seem to occur.

I've tried to reproduce your situation, but as I expected I could not. I've tried both the 2.0 and 4.0 runtime, in 32 and 64 bit mode (debugging sometimes behaves differently under x64).
Is the code shown a simplification? Have you checked all your assumptions? I understand you're skipping 3 lines of code, both the if statements and the lock? In that case, even setting the lock object to null does not cause the exception you describe.
(Having _lockRecordingObject set to null causes an ArgumentNullException when leaving the lock scope)

What is the best way to lock cache in asp.net?

I know in certain circumstances, such as long running processes, it is important to lock ASP.NET cache in order to avoid subsequent requests by another user for that resource from executing the long process again instead of hitting the cache.
What is the best way in c# to implement cache locking in ASP.NET?

Here's the basic pattern:
Check the cache for the value, return if its available
If the value is not in the cache, then implement a lock
Inside the lock, check the cache again, you might have been blocked
Perform the value look up and cache it
Release the lock
In code, it looks like this:
private static object ThisLock = new object();
public string GetFoo()
{
// try to pull from cache here
lock (ThisLock)
{
// cache was empty before we got the lock, check again inside the lock
// cache is still empty, so retreive the value here
// store the value in the cache here
}
// return the cached value here
}

For completeness a full example would look something like this.
private static object ThisLock = new object();
...
object dataObject = Cache["globalData"];
if( dataObject == null )
{
lock( ThisLock )
{
dataObject = Cache["globalData"];
if( dataObject == null )
{
//Get Data from db
dataObject = GlobalObj.GetData();
Cache["globalData"] = dataObject;
}
}
}
return dataObject;

There is no need to lock the whole cache instance, rather we only need to lock the specific key that you are inserting for.
I.e. No need to block access to the female toilet while you use the male toilet :)
The implementation below allows for locking of specific cache-keys using a concurrent dictionary. This way you can run GetOrAdd() for two different keys at the same time - but not for the same key at the same time.
using System;
using System.Collections.Concurrent;
using System.Web.Caching;
public static class CacheExtensions
{
private static ConcurrentDictionary<string, object> keyLocks = new ConcurrentDictionary<string, object>();
/// <summary>
/// Get or Add the item to the cache using the given key. Lazily executes the value factory only if/when needed
/// </summary>
public static T GetOrAdd<T>(this Cache cache, string key, int durationInSeconds, Func<T> factory)
where T : class
{
// Try and get value from the cache
var value = cache.Get(key);
if (value == null)
{
// If not yet cached, lock the key value and add to cache
lock (keyLocks.GetOrAdd(key, new object()))
{
// Try and get from cache again in case it has been added in the meantime
value = cache.Get(key);
if (value == null && (value = factory()) != null)
{
// TODO: Some of these parameters could be added to method signature later if required
cache.Insert(
key: key,
value: value,
dependencies: null,
absoluteExpiration: DateTime.Now.AddSeconds(durationInSeconds),
slidingExpiration: Cache.NoSlidingExpiration,
priority: CacheItemPriority.Default,
onRemoveCallback: null);
}
// Remove temporary key lock
keyLocks.TryRemove(key, out object locker);
}
}
return value as T;
}
}

Just to echo what Pavel said, I believe this is the most thread safe way of writing it
private T GetOrAddToCache<T>(string cacheKey, GenericObjectParamsDelegate<T> creator, params object[] creatorArgs) where T : class, new()
{
T returnValue = HttpContext.Current.Cache[cacheKey] as T;
if (returnValue == null)
{
lock (this)
{
returnValue = HttpContext.Current.Cache[cacheKey] as T;
if (returnValue == null)
{
returnValue = creator(creatorArgs);
if (returnValue == null)
{
throw new Exception("Attempt to cache a null reference");
}
HttpContext.Current.Cache.Add(
cacheKey,
returnValue,
null,
System.Web.Caching.Cache.NoAbsoluteExpiration,
System.Web.Caching.Cache.NoSlidingExpiration,
CacheItemPriority.Normal,
null);
}
}
}
return returnValue;
}

Craig Shoemaker has made an excellent show on asp.net caching:
http://polymorphicpodcast.com/shows/webperformance/

I have come up with the following extension method:
private static readonly object _lock = new object();
public static TResult GetOrAdd<TResult>(this Cache cache, string key, Func<TResult> action, int duration = 300) {
TResult result;
var data = cache[key]; // Can't cast using as operator as TResult may be an int or bool
if (data == null) {
lock (_lock) {
data = cache[key];
if (data == null) {
result = action();
if (result == null)
return result;
if (duration > 0)
cache.Insert(key, result, null, DateTime.UtcNow.AddSeconds(duration), TimeSpan.Zero);
} else
result = (TResult)data;
}
} else
result = (TResult)data;
return result;
}
I have used both #John Owen and #user378380 answers. My solution allows you to store int and bool values within the cache aswell.
Please correct me if there's any errors or whether it can be written a little better.

I saw one pattern recently called Correct State Bag Access Pattern, which seemed to touch on this.
I modified it a bit to be thread-safe.
http://weblogs.asp.net/craigshoemaker/archive/2008/08/28/asp-net-caching-and-performance.aspx
private static object _listLock = new object();
public List List() {
string cacheKey = "customers";
List myList = Cache[cacheKey] as List;
if(myList == null) {
lock (_listLock) {
myList = Cache[cacheKey] as List;
if (myList == null) {
myList = DAL.ListCustomers();
Cache.Insert(cacheKey, mList, null, SiteConfig.CacheDuration, TimeSpan.Zero);
}
}
}
return myList;
}

This article from CodeGuru explains various cache locking scenarios as well as some best practices for ASP.NET cache locking:
Synchronizing Cache Access in ASP.NET

I've wrote a library that solves that particular issue: Rocks.Caching
Also I've blogged about this problem in details and explained why it's important here.

I modified #user378380's code for more flexibility. Instead of returning TResult now returns object for accepting different types in order. Also adding some parameters for flexibility. All the idea belongs to
#user378380.
private static readonly object _lock = new object();
//If getOnly is true, only get existing cache value, not updating it. If cache value is null then set it first as running action method. So could return old value or action result value.
//If getOnly is false, update the old value with action result. If cache value is null then set it first as running action method. So always return action result value.
//With oldValueReturned boolean we can cast returning object(if it is not null) appropriate type on main code.
public static object GetOrAdd<TResult>(this Cache cache, string key, Func<TResult> action,
DateTime absoluteExpireTime, TimeSpan slidingExpireTime, bool getOnly, out bool oldValueReturned)
{
object result;
var data = cache[key];
if (data == null)
{
lock (_lock)
{
data = cache[key];
if (data == null)
{
oldValueReturned = false;
result = action();
if (result == null)
{
return result;
}
cache.Insert(key, result, null, absoluteExpireTime, slidingExpireTime);
}
else
{
if (getOnly)
{
oldValueReturned = true;
result = data;
}
else
{
oldValueReturned = false;
result = action();
if (result == null)
{
return result;
}
cache.Insert(key, result, null, absoluteExpireTime, slidingExpireTime);
}
}
}
}
else
{
if(getOnly)
{
oldValueReturned = true;
result = data;
}
else
{
oldValueReturned = false;
result = action();
if (result == null)
{
return result;
}
cache.Insert(key, result, null, absoluteExpireTime, slidingExpireTime);
}
}
return result;
}

The accepted answer (recommending reading outside of the lock) is very bad advice and is being implemented since 2008. It could work if the cache uses a concurrent dictionary, but that itself has a lock for reads.
Reading outside of the lock means that other threads could be modifying the cache in the middle of read. This means that the read could be inconsistent.
For example, depending on the implementation of the cache (probably a dictionary whose internals are unknown), the item could be checked and found in the cache, at a certain index in the underlying array of the cache, then another thread could modify the cache so that the items from the underlying array are no longer in the same order, and then the actual read from the cache could be from a different index / address.
Another scenario is that the read could be from an index that is now outside of the underlying array (because items were removed), so you can get exceptions.