I have a very simple Code, but what it does it completely weird. It is a simple Cache abstraction and goes like this:
public class CacheAbstraction
{
private MemoryCache _cache;
public CacheAbstraction()
{
_cache = new MemoryCache(new MemoryCacheOptions { });
}
public async Task<T> GetItemAsync<T>(TimeSpan duration, Func<Task<T>> factory,
[CallerMemberName] string identifier = null ) where T : class
{
return await _cache.GetOrCreateAsync<T>(identifier, async x =>
{
x.SetAbsoluteExpiration(DateTime.UtcNow.Add(duration));
T result = null;
result = await factory();
return result;
});
}
}
Now the fun part: I'm passing expiration durations of 1h - 1d
If I run it in a test suite, everything is fine.
If I run it as a .net core app, the expiration is always set to "now" and the item expires on the next cache check. WTF!?
I know it's been two years, but I ran across this same problem (cache items seeming to expire instantly) recently and found a possible cause. Two essentially undocumented features in MemoryCache: linked cache entries and options propagation.
This allows a child cache entry object to passively propagate it's options up to a parent cache entry when the child goes out of scope. This is done via IDisposable, which ICacheEntry implements and is used internally by MemoryCache in extension methods like Set() and GetOrCreate/Async(). What this means is that if you have "nested" cache operations, the inner ones will propagate their cache entry options to the outer ones, including cancellation tokens, expiration callbacks, and expiration times.
In my case, we were using GetOrCreateAsync() and a factory method that made use of a library which did its own caching using the same injected IMemoryCache. For example:
public async Task<Foo> GetFooAsync() {
return await _cache.GetOrCreateAsync("cacheKey", async c => {
c.AbsoluteExpirationRelativeToNow = TimeSpan.FromHours(1);
return await _library.DoSomething();
});
}
The library uses IMemoryCache internally (the same instance, injected via DI) to cache results for a few seconds, essentially doing this:
_cache.Set(queryKey, queryResult, TimeSpan.FromSeconds(5));
Because GetOrCreateAsync() is implemented by creating a CacheEntry inside a using block, the effect is that the 5 second expiration used by the library propagates up to the parent cache entry in GetFooAsync(), resulting in the Foo object always only being cached for 5 seconds instead of 1 hour, effectively expiring it immediately.
DotNet Fiddle showing this behavior: https://dotnetfiddle.net/fo14BT
You can avoid this propagation behavior in a few ways:
(1) Use TryGetValue() and Set() instead of GetOrCreateAsync()
if (_cache.TryGetValue("cacheKey", out Foo result))
return result;
result = await _library.DoSomething();
return _cache.Set("cacheKey", result, TimeSpan.FromHours(1));
(2) Assign the cache entry options after invoking the other code that may also use the cache
return await _cache.GetOrCreateAsync("cacheKey", async c => {
var result = await _library.DoSomething();
// set expiration *after*
c.AbsoluteExpiration = DateTime.Now.AddHours(1);
return result;
});
(and since GetOrCreate/Async() does not prevent reentrancy, the two are effectively the same from a concurrency standpoint).
Warning: Even then it's easy to get wrong. If you try to use AbsoluteExpirationRelativeToNow in option (2) it won't work because setting that property doesn't remove the AbsoluteExpiration value if it exists resulting in both properties having a value in the CacheEntry, and AbsoluteExpiration is honored before the relative.
For the future, Microsoft has added a feature to control this behavior via a new property MemoryCacheOptions.TrackLinkedCacheEntries, but it won't be available until .NET 7. Without this future feature, I haven't been able to think of a way for libraries to prevent propagation, aside from using a different MemoryCache instance.
Related
I have a .NET 6 application configured as a Windows Service where I am trying to make use of IMemoryCache to store temporary access tokens used for API interactions. I am seeing some really odd behavior in the cache that is resulting in not being able to retrieve values that I have just added in a previous execution.
I have a simple abstraction for IMemoryCache called ITokenCacheService, which is implemented as such:
public class TokenCacheService : ITokenCacheService
{
private readonly IMemoryCache _cache;
public TokenCacheService(IMemoryCache cache)
{
_cache = cache;
}
public bool TryGetToken(TokenScope scope, out string token)
{
return _cache.TryGetValue(scope, out token);
}
public void AddTokenToCache(TokenScope scope, string token, int expiresIn)
{
_cache.Set(scope, token, TimeSpan.FromTicks(expiresIn));
}
}
The caller of this class is first attempting to TryGetToken() to use the cached value, and if that fails, it reaches out to the API to get a new access token and then caches it with AddTokenToCache(). This works well on the first pass, where the cache is expected to be empty and the token is therefore added. However, on the second pass when the background service runs again, the IMemoryCache will fail to retrieve that same value that was just put in the cache.
Debugging this process, I can actually see my previously cached value in the cache before calling TryGetValue. Even so, it returns false and fails to return the cached value. Where it gets really strange - not only is it returning false for a value that is actually cached, but after TryGetValue executes, that value is then missing from the cache, even though it was there before the call. I modified TryGetToken in the following way to test this:
public bool TryGetToken(TokenScope scope, out string token)
{
// Casting to have access to .Count
var cacheImpl = (MemoryCache)_cache;
var countBefore = cacheImpl.Count; // Count here is 1
var result = _cache.TryGetValue(scope, out token); // Result here is false
var countAfter = cacheImpl.Count; // Count here is 0 ???
return result;
}
I can get the token out if I try to retrieve it right after I set it. I tested this by modifying AddTokenToCache just to check the value:
public void AddTokenToCache(TokenScope scope, string token, int expiresIn)
{
_cache.Set(scope, token, TimeSpan.FromTicks(expiresIn));
// Success here will be 'true' and 'test' will contain the token value.
var success = _cache.TryGetValue(scope, out var test);
}
The number of ticks being set for this token is 31536000, which amounts to 18 days and 6 hours. I don't think I'm hitting an expiration window. I am using the default registration method for the memory cache without any options:
services.AddMemoryCache();
Am I expecting this service to work in a way that it's not designed, or do I need to configure it some way that I'm not doing? Really not sure what I'm missing here. The value is clearly there in the cache when I come back around on the second pass, but TryGetValue() is failing and seems to be removing it from the cache.
The number of ticks being set for this token is 31536000, which amounts to 18 days and 6 hours
31536000 ticks is about 3.1 seconds, which is probably the cause of what you're seeing. MemoryCache will only go looking for entries to evict when you interact with it. The fact that your entry is disappearing as soon as you try to query it is consistent with that entry expiring, and the MemoryCache only noticing at that point.
18 days and 6 hours is a whopping 15768000000000 ticks.
I'm running integration tests with xUnit in ASP.NET, and one of which ensures that querying the data multiple times results in only 1 query to the server. If I run this test alone, then it works. If I run all the tests, then the server gets queried 0 times instead of 1 by this test. Which indicates that the result was already in the cache because of other tests.
How can I ensure the IAppCache is empty at the beginning of the test? I'm using LazyCache implementation.
My guess is that the class instance is recreated for each tests, but static data is shared; and the cache is static. I don't see any "flush" method on the cache.
As mentioned in my OP comment LazyCache afaik doesn't have a clear operation or anything native to nuke the cache. However I think there are a few options at your disposal.
Implement a method before/after each test to remove the cache entries, using Remove;
Supply a different LazyCache cache provider for the tests that doesn't persist the cache between tests
Dig into LazyCache, get the underlying cache provider and see if that has any methods to clear the cache entries
1 or 3 would be my picks. From a testing perspective, 1 means you need to know the internals of what you're testing. If it were me, I'm a bit lazy and would probably write the few lines to just nuke the cache.
By default LazyCache uses MemoryCache as the cache provider. MemoryCache doesn't have an explicit clear operation either but Compact looks like it can essentially clear the cache when the compact percentage is set to 1.0. To access it, you'll need to get the underlying MemoryCache object from LazyCache:
IAppCache cache = new CachingService();
var cacheProvider = cache.CacheProvider;
var memoryCache = (MemoryCache)cacheProvider.GetType().GetField("cache", BindingFlags.Instance | BindingFlags.NonPublic).GetValue(cacheProvider);
memoryCache.Compact(1.0);
Complete LINQPad working sample:
void Main()
{
IAppCache cache = new CachingService();
Console.WriteLine(cache.GetOrAdd("Foo", () => Foo.NewFoo, DateTimeOffset.Now.AddHours(1.0)));
var cacheProvider = cache.CacheProvider;
var memoryCache = (MemoryCache)cacheProvider.GetType().GetField("cache", BindingFlags.Instance | BindingFlags.NonPublic).GetValue(cacheProvider);
memoryCache.Compact(1.0);
Console.WriteLine(cache.Get<Foo>("Foo"));
}
public class Foo
{
public static Foo NewFoo
{
get
{
Console.WriteLine("Factory invoked");
return new Foo();
}
}
public override string ToString()
{
return "I am a foo";
}
}
This results in the following each run:
If I remove the compact invocation we get the following:
So that shows that Compact(1.0) will nuke the cache entry even with an expiry date of +1 hour.
I'm in a situation where two calls at the same time write to the session (of an asp.net core application running on the old framework), and one of the session variables gets overwritten.
Given the following controller code, assume that the long session gets called first, 200 ms later the short session gets called, and 800 ms later (when the long session is done) the result of both sessions gets called.
[HttpPost("[action]")]
public async Task<IActionResult> TestLongSession() {
HttpContext.Session.SetString("testb", "true");
// If we do this delay BEFORE the session ("testb") is set, then all is fine.
await Task.Delay(1000);
return Ok();
}
[HttpPost("[action]")]
public async Task<IActionResult> TestShortSession() {
HttpContext.Session.SetString("testa", "true");
return Ok();
}
[HttpGet("[action]")]
public async Task<IActionResult> TestResultOfBothSessions() {
string a = HttpContext.Session.GetString("testa");
string b = HttpContext.Session.GetString("testb");
return Ok($"A: {a}, B: {b}");
}
The result of the final call (TestBothSessions) is "A: , B: true".
The question is then: Is there something I missed to make the session work (aka, return "A: true, B: true")?
Obviously, I could remove the delay and all is fine, but in the real application there's a call that potentially can take some time, and I prefer not to write the session variable at a later time (I guess I could with a bit of custom error handling, but then the problem still remains that I no longer trust the asp.net session to work with synchronous calls).
Edit: The typescript code that calls these endpoints from the browser:
this.service.testLongSession().subscribe(() => {
this.service.testBothSessions().subscribe((result: string) => {
console.log(result);
});
});
setTimeout(() => {
this.service.testShortSession().subscribe();
}, 200);
I believe the behavior you observe is what the ASP.NET authors intended. I look at the interfaces that session stores need to implement, namely ISession and ISessionStore, and I see no synchronization mechanisms to prevent the overwriting of data during simultaneous requests.
The benefit of such a simple interface is that it's much easier to implement, and can be easily implemented by a variety of caches and databases.
ASP.NET 4 had a much more complex session store base class SessionStateStoreProviderBase that included locking logic, but it was really challenging to implement.
It seems like the execution context is not kept until Dispose is called on elements resolved in the controller scope. This is probably due to the fact that asp.net core has to jump between native and managed code and resets the execution context at each jump. Seems like the correct context is not restored any more before the scope is disposed.
The following demonstrates the issue - simply put this in the default asp.net core sample project and register TestRepo as a transient dependency.
When calling GET api/values/ we set the value for the current task to 5 in a static AsyncLocal at the start of the call. That value flows as expected through awaits without any problem. But when the controller and its dependencies are disposed after the call the AsyncLocal context is already reset.
[Route("api/[controller]")]
public class ValuesController : Controller
{
private readonly TestRepo _testRepo;
public ValuesController(TestRepo testRepo) => _testRepo = testRepo;
[HttpGet()]
public async Task<IActionResult> Get()
{
_testRepo.SetValue(5);
await Task.Delay(100);
var val = _testRepo.GetValue(); // val here has correctly 5.
return Ok();
}
}
public class TestRepo : IDisposable
{
private static readonly AsyncLocal<int?> _asyncLocal = new AsyncLocal<int?>();
public int? GetValue() => _asyncLocal.Value;
public void SetValue(int x) => _asyncLocal.Value = x;
public void Foo() => SetValue(5);
public void Dispose()
{
if (GetValue() == null)
{
throw new InvalidOperationException(); //GetValue() should be 5 here :(
}
}
}
Is this intentional? And if yes is there any workaround around this problem?
The behavior you are seeing is an unfortunate quirk in the way that ASP.NET Core works. It's unclear to me why Microsoft choose this behavior, but it seems copied from the way Web API worked, which has the exact behavior. Disposing is obviously done at the end of the request, but for some reason the asynchronous context is already cleared before that point, making it impossible to run the complete request in a single asynchronous context.
You've basically got two options:
Instead of using ambient state to share state, flow state through the object graph instead of using ambient state. In other words, make TestRepo Scoped, and store value in a private field.
Move the operation that uses that value to an earlier stage in the request. For instance, you can define some middleware that wraps a request and invokes that operation at the end. At that stage, the asynchronous context will still exist.
Some DI containers actually apply this second technique. Simple Injector, for instance, uses scoping that is based on ambient state, using AsyncLocal<T> under the covers. When integrated in ASP.NET Core, it will wrap the request in a piece of middleware that applies this scope. This means that any Scoped component, resolved from Simple Injector, will be disposed before the ASP.NET Core pipeline disposes its services, and this happens while the asynchronous context is still available.
This question already has an answer here:
Automatically refresh ASP.NET Output Cache on expiry
(1 answer)
Closed 5 years ago.
I have a website with a lot of data in it.
I use C# .NET MVC4 for development.
I have a big slow page loading problem when the cache is empty.
currently I'm using a cache that contains all the data that I need and when the cache is on the pages loads right away, but when the cache expires it takes about 10s the page to be fully loaded.
I'm looking for an option to auto refresh the cache when it expires,
I've been searching over goolge, but couldn't find anything in that matter
How is it should be done?
Or are there other options to solve this problem?
Thanks
You could cache it on the first call with a TTL, let it invalidate, and then the next call will get it and cache it back again. The problem with this is that you are slowing down your thread while it has to go fetch the data as it is unavailable, and multiple threads will wait for it (assuming you lock the read to prevent flooding).
One way to get around the first load issue is to prime your cache on application start up. This assures that when your application is ready to be used, the data is already loaded up and will be fast. Create a quick interface like ICachePrimer { void Prime() }, scan your assemblies for it, resolve them, then run them.
The way I like to get around the empty cache on invalidation issue is to refresh the data before it is removed. To easily do this in .Net, you can utilize the MemoryCache's CacheItemPolicy callbacks.
UpdateCallback occurs before the item is removed, and allows you to refresh the item.
RemovedCallback occurs after the item has been removed.
In the example below, my CachedRepository will refresh the cached item when it is invalidated. Other threads will continue to receive the "old" value until the refresh completes.
public class MyCachedRepository : IMyRepository
{
private readonly IMyRepository _baseRepository;
private readonly ObjectCache _cache;
public MyCachedRepository(IMyRepository baseRepository, ObjectCache cache)
{
_baseRepository = baseRepository;
_cache = cache;
}
public string GetById(string id)
{
var value = _cache.Get(id) as string;
if (value == null)
{
value = _baseRepository.GetById(id);
if (value != null)
_cache.Set(id, value, GetPolicy());
}
return value;
}
private CacheItemPolicy GetPolicy()
{
return new CacheItemPolicy
{
UpdateCallback = CacheItemRemoved,
SlidingExpiration = TimeSpan.FromMinutes(0.1), //set your refresh interval
};
}
private void CacheItemRemoved(CacheEntryUpdateArguments args)
{
if (args.RemovedReason == CacheEntryRemovedReason.Expired || args.RemovedReason == CacheEntryRemovedReason.Removed)
{
var id = args.Key;
var updatedEntity = _baseRepository.GetById(id);
args.UpdatedCacheItem = new CacheItem(id, updatedEntity);
args.UpdatedCacheItemPolicy = GetPolicy();
}
}
}
Source: http://pdalinis.blogspot.in/2013/06/auto-refresh-caching-for-net-using.html
There is no mechanism to auto refresh a cache when the keys expire. All caching systems employ passive expiration. The keys are invalidated the first time they are requested after the expiration, not automatically at that exact expiration time.
What you're talking about is essentially a cache that never expires, which is easy enough to achieve. Simply either pass no expiration (if the caching mechanism allows it) or a far-future expiration. Then, your only problem is refreshing the cache on some schedule, so that it does not become stale. For that, one option is to create a console application that sets the values in the cache (importantly, without caring if there's something there already) and then use Task Scheduler or similar to schedule it to run at set intervals. Another option is to use something like Revalee to schedule callbacks into your web application at defined intervals. This basically the same as creating a console app, only the code could be integrated into your same website project.
You can also use Hangfire to perform the scheduling directly within your web application, and could use that to run a console application, hit a URL, whatever. The power of Hangfire is that it allow you pretty much schedule any process you want, but that also means you have to actually provide the code for what should happen, i.e. actually connect with HttpClient and fetch the URL, rather than just telling Revallee to hit a particular URL.