Custom ChangeMonitor for .Net MemoryCache causes invalid operation exception - c#

I have written my own custom change monitor class for the .NET MemoryCache. It seems to initialize fine, but when I attempt to add it to the Cache, it throws an InvalidOperation exception - The method has already been invoked, and can only be invoked once.
My change monitor class:
internal class MyChangeMonitor : ChangeMonitor
{
private Timer _timer;
private readonly string _uniqueId;
private readonly TypeAsOf _typeAsOf;
private readonly string _tableName;
public GprsChangeMonitor(TypeAsOf typeAsOf, string tableName)
{
bool initComplete = false;
try
{
_typeAsOf = typeAsOf;
_tableName = tableName;
_uniqueId = Guid.NewGuid().ToString();
TimeSpan ts = new TimeSpan(0, 0, 5, 0, 0);
_timer = new Timer {Interval = ts.TotalMilliseconds};
_timer.Elapsed += CheckForChanges;
_timer.Enabled = true;
_timer.Start();
initComplete = true;
}
finally
{
base.InitializationComplete();
if(!initComplete)
Dispose(true);
}
}
void CheckForChanges(object sender, System.Timers.ElapsedEventArgs e)
{
//check for changes, if different
base.OnChanged(_typeAsOf);
}
}
The code I use to create the cache policy and add the key/value pair to the cache:
CacheItemPolicy policy = new CacheItemPolicy
{
UpdateCallback = OnCacheEntryUpdateCallback
};
policy.AbsoluteExpiration = SystemTime.Today.AddHours(24);
//monitor the for changes
string tableName = QuickRefreshItems[type];
MyChangeMonitor cm = new MyChangeMonitor(typeAsOf, tableName);
policy.ChangeMonitors.Add(cm);
cm.NotifyOnChanged(OnRefreshQuickLoadCacheItems);
MyCache.Set(cacheKey, value, policy);
The Set call throws the invalid operation exception which is weird because, according to the MSDN documentation, it only throws the ArgumentNull, Argument, ArgumentOutOfRange, and NotSupported exceptions.
I am sure that I must be making a simple mistake. But it's hard to find good documentation or examples on writing your own custom change monitor. Any help would be appreciated.

I know the comments have the answer, but I wanted it to be more obvious...
When a ChangeMonitor is used, it will fire immediately if the cache entry does not exist.
MSDN documentation states it this way:
A monitored entry is considered to have changed for any of the
following reasons:
A) The key does not exist at the time of the call to the
CreateCacheEntryChangeMonitor method. In that case, the resulting
CacheEntryChangeMonitor instance is immediately set to a changed
state. This means that when code subsequently binds a
change-notification callback, the callback is triggered immediately.
B) The associated cache entry was removed from the cache. This can
occur if the entry is explicitly removed, if it expires, or if it is
evicted to recover memory

I've had the exact same error:
Source: System.Runtime.Caching
Exception type: System.InvalidOperationException
Message: The method has already been invoked, and can only be invoked once.
Stacktrace: at System.Runtime.Caching.ChangeMonitor.NotifyOnChanged(OnChangedCallback onChangedCallback)
at System.Runtime.Caching.MemoryCacheEntry.CallNotifyOnChanged()
at System.Runtime.Caching.MemoryCacheStore.AddToCache(MemoryCacheEntry entry)
at System.Runtime.Caching.MemoryCacheStore.Set(MemoryCacheKey key, MemoryCacheEntry entry)
at System.Runtime.Caching.MemoryCache.Set(String key, Object value, CacheItemPolicy policy, String regionName)
I've searched for it for hours.. until the light of logic struck me:
I was using a static policy object that was reused.. (some unconscious process in me reuses all objects if they are equal,maybe I am afraid of constructing objects that consume some bytes in memory )
By creating a new policy object for every item in the cache, the error was gone. Pretty logical if you think about it.

Posting a late answer as I've just faced the same issue and conducted my own investigation.
When you register your change monitor with a cached item policy — policy.ChangeMonitors.Add(cm) — the CacheItemPolicy implementation registers its own change callback on it via ChangeMonitor.NotifyOnChanged. You're not supposed to be calling cm.NotifyOnChanged to register yet another callback, or it will throw The method has already been invoked, and can only be invoked once at that point.
Instead, use CacheItemPolicy.UpdateCallback or CacheItemPolicy.RemovedCallback to update/remove the cache item, e.g. as described in this blog post.

Related

Attach listener to MemoryCache which can remove expired items

I am using MemoryCache in one of my project to cache some keys and value. I have attached a listener on my MemoryCache that whenever it is expiring any items then that listener should get called so that I can remove those keys from CacheKeys HashSet as well. Basically I want to be consistent with what I have in CacheKeys and MemoryCache.
I read about MemoryCache and looks like I can use RegisterPostEvictionCallback as shown below:
private static readonly HashSet<string> CacheKeys = new HashSet<string>();
private bool CacheEntries<T>(MemoryCache memoryCache, string cacheKey, T value, Configuration config, Action<MemoryCacheEntryOptions> otherOptions = null)
{
int minutes = randomGenerator.Next(config.LowTime, config.HighTime);
MemoryCacheEntryOptions options = new MemoryCacheEntryOptions()
{
Size = config.Size,
Priority = config.Priority,
AbsoluteExpirationRelativeToNow = TimeSpan.FromMinutes(minutes)
};
// attaching listener
options.RegisterPostEvictionCallback(callback: EvictionCallback, state: this);
if (otherOptions != null) otherOptions(options);
CacheKeys.Add(cacheKey);
memoryCache.Set<T>(cacheKey, value, options);
return true;
}
private void EvictionCallback(object key, object value, EvictionReason reason, object state)
{
CacheKeys.Remove(key);
var message = $"Entry was evicted. Reason: {reason}.";
Console.WriteLine(message);
}
It looks like there is some issue where items doesn't expire automatically as per this this thread.
So to avoid the issue mentioned in that thread. Do I need to remove AbsoluteExpirationRelativeToNow and use CancellationTokenSource here?
If yes, how should I go ahead and make the change since I don't want to change the current functionality that I have already in my original code so if I remove AbsoluteExpirationRelativeToNow and use CancellationTokenSource here will it behave differently than what my original code is doing?

How to use Lazy to handle concurrent request?

I'm new in C# and trying to understand how to work with Lazy.
I need to handle concurrent request by waiting the result of an already running operation. Requests for data may come in simultaneously with same/different credentials.
For each unique set of credentials there can be at most one GetDataInternal call in progress, with the result from that one call returned to all queued waiters when it is ready
private readonly ConcurrentDictionary<Credential, Lazy<Data>> Cache
= new ConcurrentDictionary<Credential, Lazy<Data>>();
public Data GetData(Credential credential)
{
// This instance will be thrown away if a cached
// value with our "credential" key already exists.
Lazy<Data> newLazy = new Lazy<Data>(
() => GetDataInternal(credential),
LazyThreadSafetyMode.ExecutionAndPublication
);
Lazy<Data> lazy = Cache.GetOrAdd(credential, newLazy);
bool added = ReferenceEquals(newLazy, lazy); // If true, we won the race.
Data data;
try
{
// Wait for the GetDataInternal call to complete.
data = lazy.Value;
}
finally
{
// Only the thread which created the cache value
// is allowed to remove it, to prevent races.
if (added) {
Cache.TryRemove(credential, out lazy);
}
}
return data;
}
Is that right way to use Lazy or my code is not safe?
Update:
Is it good idea to start using MemoryCache instead of ConcurrentDictionary? If yes, how to create a key value, because it's a string inside MemoryCache.Default.AddOrGetExisting()
This is correct. This is a standard pattern (except for the removal) and it's a really good cache because it prevents cache stampeding.
I'm not sure you want to remove from the cache when the computation is done because the computation will be redone over and over that way. If you don't need the removal you can simplify the code by basically deleting the second half.
Note, that Lazy has a problem in the case of an exception: The exception is stored and the factory will never be re-executed. The problem persists forever (until a human restarts the app). In my mind this makes Lazy completely unsuitable for production use in most cases.
This means that a transient error such as a network issue can render the app unavailable permanently.
This answer is directed to the updated part of the original question. See #usr answer regarding thread-safety with Lazy<T> and the potential pitfalls.
I would like to know how to avoid using ConcurrentDictionary<TKey, TValue> and start
using MemoryCache? How to implement
MemoryCache.Default.AddOrGetExisting()?
If you're looking for a cache which has a mechanism for auto expiry, then MemoryCache is a good choice if you don't want to implement the mechanics yourself.
In order to utilize MemoryCache which forces a string representation for a key, you'll need to create a unique string representation of a credential, perhaps a given user id or a unique username?
If you can, you can create an override of ToString which represents your unique identifier or simply use the said property, and utilize MemoryCache like this:
public class Credential
{
public Credential(int userId)
{
UserId = userId;
}
public int UserId { get; private set; }
}
And now your method will look like this:
private const EvictionIntervalMinutes = 10;
public Data GetData(Credential credential)
{
Lazy<Data> newLazy = new Lazy<Data>(
() => GetDataInternal(credential), LazyThreadSafetyMode.ExecutionAndPublication);
CacheItemPolicy evictionPolicy = new CacheItemPolicy
{
AbsoluteExpiration = DateTimeOffset.UtcNow.AddMinutes(EvictionIntervalMinutes)
};
var result = MemoryCache.Default.AddOrGetExisting(
new CacheItem(credential.UserId.ToString(), newLazy), evictionPolicy);
return result != null ? ((Lazy<Data>)result.Value).Value : newLazy.Value;
}
MemoryCache provides you with a thread-safe implementation, this means that two threads accessing AddOrGetExisting will only cause a single cache item to be added or retrieved. Further, Lazy<T> with ExecutionAndPublication guarantess only a single unique invocation of the factory method.

MemoryCache does not work after several minutes

Here is my code:
public class ConfigCache
{
private static volatile ObjectCache _cache = MemoryCache.Default;
private const string KeyModule = "MODULE_XDOC_KEY";
private static string _settingFile;
public ConfigCache(string file)
{
_settingFile = file;
}
public XDocument Get()
{
var doc = _cache[KeyModule] as XDocument;
if (doc == null)
{
doc = XDocument.Load(_settingFile);
var policy = new CacheItemPolicy();
var filePaths = new List<string> {_settingFile};
policy.ChangeMonitors.Add(new HostFileChangeMonitor(filePaths));
var callback = new CacheEntryRemovedCallback(this.MyCachedItemRemovedCallback);
policy.RemovedCallback = callback;
_cache.Set(KeyModule, doc, policy);
}
return _cache[KeyModule] as XDocument;
}
private void MyCachedItemRemovedCallback(CacheEntryRemovedArguments arguments)
{
// Log these values from arguments list
}
}
When run into _cache.Set() first time, it works fine:
_cache.Set() works well, it add the xdoc into cache.
But after several minutes(1 or 2 minutes), cache will not work anymore:
_cache.Set() does not insert anything into cache
_cache.Set() does not report any error.
the callback MyCachedItemRemovedCallback never triggered.
Someone met same issue:
MemoryCache always returns "null" after first expiration
But it seems not resolved yet. Anyone have any idea on this?
Your problem might be that the cache is being disposed. This will result in the MemoryCache silently returning null rather than throwing any exception. First search your code to be sure you are not disposing it. If you are sure you aren't disposing it, try breaking on the AppDomain.UnhandledException event. MemoryCache subscribes to this event (see this answer and disposes itself. If you are handling the UnhandledException event in your app as a global error handler, this could be the problem.
If this is the issue, a simple workaround is to handle the event and recreate a new instance of the cache. (See my answer here)

Why does the synchronized wrapper for ArrayList not work?

I have generated the proxy classes for a web service in Visual Studio with 'Add Web Reference'. The generated RTWebService class has a method SetValueAsync. I extended this class and added a SetValueRequest which keeps track of the requests and cancels all pending requests when an error occurs. With every request I store the userState object in an ArrayList I created as follows:
requests = ArrayList.Synchronized(new ArrayList());
I created a method:
public void CancelPendingRequests() {
lock (requests.SyncRoot) {
if (requests.Count > 0) {
foreach (object request in requests) {
this.CancelAsync(request);
}
requests.Clear();
}
}
}
I call this method when a request returns on the SetValueCompleted event:
private void onRequestComplete(
object sender,
Service.SetValueCompletedEventArgs args
) {
lock (syncResponse) {
if (args.Cancelled) {
return;
}
if (args.UserState != null) {
requests.Remove(args.UserState);
}
if (args.Error != null) {
CancelPendingRequests();
}
}
}
To start a new request I call:
public void SetValueRequest(string tag, string value) {
var request = new object();
this.SetValueAsync(tag, value, request);
requests.Add(request);
}
Everytime I make a request and at the same time a response returns with an error, I get a TargetInvocationException in the CancelPendingRequests. The inner exception is an InvalidOperationException on an ArrayList in the CancelPendingRequests method saying:
Collection was modified; enumeration operation may not execute.
So it seems SetValueRequest has modified the requests object while I was enumerating it. I thought this was impossible because I used the synchronized wrapper for ArrayList and use the SyncRoot to synchronize the enumeration. I'm a bit stuck on this so if anyone has an idea?
never use SyncRoot it's inherently broken. (if you share the list you just invite a deadlock)
Don't use ArrayList, it should be marked "Deprecated".
ArrayList.Synchronized return's something that works more slowly but is not thread safe, i.e. it's not thread safe during a set of operations.
you can either use something from System.Collection.Concurrent, or use ReaderWriterLockSlim
ORIGINAL ANSWER
I worked around the problem by removing the enumeration. I now use:
public void CancelPendingRequests() {
lock (requests.SyncRoot) {
if (requests.Count > 0) {
for (int i = 0; i < requests.Count; i++) {
this.CancelAsync(requests[i]);
}
requests.Clear();
}
}
}
This seems to do the trick. I'm still a bit worried that this lock (requests.SyncRoot) didn't work on the enumeration so why would it work here? Anyway, I am now unable to reproduce the exception like i could before so I consider this problem as solved. I can't waste any more time on this.
EDIT
Forget my silly answer above. I was working on a project and needed to make progress. I tracked down the problem now:
So it appeared this bug was not multithreading related at all. All code was executed on the same thread, I didn't need those locks. The problem lies in the fact that I was canceling the requests in my enumeration. The CancelAsync method raises the SetValueCompleted event which in turn calls requests.Remove, thus modifying the requests inside the enumeration. I Learnt some pitfall with events today.
I solved the problem by enumerating over a local copy of the requests object which I created with the ToArray method.
public void CancelPendingRequests()
if (requests.Count > 0) {
for (object request in requests.ToArray()) {
this.CancelAsync(request);
}
}
}
Trying adding a local variable to your CancelPendingRequests method for each request object like this:
public void CancelPendingRequests() {
lock (requests.SyncRoot)
{
if (requests.Count > 0)
{
foreach (object request in requests)
{
object currentRequest = request; //Add this
this.CancelAsync(currentRequest);
}
requests.Clear();
}
}
}

Is this thread.abort() normal and safe?

I created a custom autocomplete control, when the user press a key it queries the database server (using Remoting) on another thread. When the user types very fast, the program must cancel the previously executing request/thread.
I previously implemented it as AsyncCallback first, but i find it cumbersome, too many house rules to follow (e.g. AsyncResult, AsyncState, EndInvoke) plus you have to detect the thread of the BeginInvoke'd object, so you can terminate the previously executing thread. Besides if I continued the AsyncCallback, there's no method on those AsyncCallbacks that can properly terminate previously executing thread.
EndInvoke cannot terminate the thread, it would still complete the operation of the to be terminated thread. I would still end up using Abort() on thread.
So i decided to just implement it with pure Thread approach, sans the AsyncCallback. Is this thread.abort() normal and safe to you?
public delegate DataSet LookupValuesDelegate(LookupTextEventArgs e);
internal delegate void PassDataSet(DataSet ds);
public class AutoCompleteBox : UserControl
{
Thread _yarn = null;
[System.ComponentModel.Category("Data")]
public LookupValuesDelegate LookupValuesDelegate { set; get; }
void DataSetCallback(DataSet ds)
{
if (this.InvokeRequired)
this.Invoke(new PassDataSet(DataSetCallback), ds);
else
{
// implements the appending of text on textbox here
}
}
private void txt_TextChanged(object sender, EventArgs e)
{
if (_yarn != null) _yarn.Abort();
_yarn = new Thread(
new Mate
{
LookupValuesDelegate = this.LookupValuesDelegate,
LookupTextEventArgs =
new LookupTextEventArgs
{
RowOffset = offset,
Filter = txt.Text
},
PassDataSet = this.DataSetCallback
}.DoWork);
_yarn.Start();
}
}
internal class Mate
{
internal LookupTextEventArgs LookupTextEventArgs = null;
internal LookupValuesDelegate LookupValuesDelegate = null;
internal PassDataSet PassDataSet = null;
object o = new object();
internal void DoWork()
{
lock (o)
{
// the actual code that queries the database
var ds = LookupValuesDelegate(LookupTextEventArgs);
PassDataSet(ds);
}
}
}
NOTES
The reason for cancelling the previous thread when the user type keys in succession, is not only to prevent the appending of text from happening, but also to cancel the previous network roundtrip, so the program won't be consuming too much memory resulting from successive network operation.
I'm worried if I avoid thread.Abort() altogether, the program could consume too much memory.
here's the code without the thread.Abort(), using a counter:
internal delegate void PassDataSet(DataSet ds, int keyIndex);
public class AutoCompleteBox : UserControl
{
[System.ComponentModel.Category("Data")]
public LookupValuesDelegate LookupValuesDelegate { set; get; }
static int _currentKeyIndex = 0;
void DataSetCallback(DataSet ds, int keyIndex)
{
if (this.InvokeRequired)
this.Invoke(new PassDataSet(DataSetCallback), ds, keyIndex);
else
{
// ignore the returned DataSet
if (keyIndex < _currentKeyIndex) return;
// implements the appending of text on textbox here...
}
}
private void txt_TextChanged(object sender, EventArgs e)
{
Interlocked.Increment(ref _currentKeyIndex);
var yarn = new Thread(
new Mate
{
KeyIndex = _currentKeyIndex,
LookupValuesDelegate = this.LookupValuesDelegate,
LookupTextEventArgs =
new LookupTextEventArgs
{
RowOffset = offset,
Filter = txt.Text
},
PassDataSet = this.DataSetCallback
}.DoWork);
yarn.Start();
}
}
internal class Mate
{
internal int KeyIndex;
internal LookupTextEventArgs LookupTextEventArgs = null;
internal LookupValuesDelegate LookupValuesDelegate = null;
internal PassDataSet PassDataSet = null;
object o = new object();
internal void DoWork()
{
lock (o)
{
// the actual code that queries the database
var ds = LookupValuesDelegate(LookupTextEventArgs);
PassDataSet(ds, KeyIndex);
}
}
}
No, it is not safe. Thread.Abort() is sketchy enough at the best of times, but in this case your control has no (heh) control over what's being done in the delegate callback. You don't know what state the rest of the app will be left in, and may well find yourself in a world of hurt when the time comes to call the delegate again.
Set up a timer. Wait a bit after the text change before calling the delegate. Then wait for it to return before calling it again. If it's that slow, or the user is typing that fast, then they probably don't expect autocomplete anyway.
Regarding your updated (Abort()-free) code:
You're now launching a new thread for (potentially) every keypress. This is not only going to kill performance, it's unnecessary - if the user isn't pausing, they probably aren't looking for the control to complete what they're typing.
I touched on this earlier, but P Daddy said it better:
You'd be better off just implementing
a one-shot timer, with maybe a
half-second timeout, and resetting it
on each keystroke.
Think about it: a fast typist might create a score of threads before the first autocomplete callback has had a chance to finish, even with a fast connection to a fast database. But if you delay making the request until a short period of time after the last keystroke has elapsed, then you have a better chance of hitting that sweet spot where the user has typed all they want to (or all they know!) and is just starting to wait for autocomplete to kick in. Play with the delay - a half-second might be appropriate for impatient touch-typists, but if your users are a bit more relaxed... or your database is a bit more slow... then you may get better results with a 2-3 second delay, or even longer. The most important part of this technique though, is that you reset the timer on every keystroke.
And unless you expect database requests to actually hang, don't bother trying to allow multiple concurrent requests. If a request is currently in-progress, wait for it to complete before making another one.
There are many warnings all over the net about using Thread.Abort. I would recommend avoiding it unless it's really needed, which in this case, I don't think it is. You'd be better off just implementing a one-shot timer, with maybe a half-second timeout, and resetting it on each keystroke. This way your expensive operation would only occur after a half-second or more (or whatever length you choose) of user inactivity.
You might want to have a look at An Introduction to Programming with C# Threads - Andrew D. Birrell. He outlines some of the best practices surrounding threading in C#.
On page 4 he says:
When you look at the
“System.Threading” namespace, you will
(or should) feel daunted by the range
of choices facing you: “Monitor” or
“Mutex”; “Wait” or “AutoResetEvent”;
“Interrupt” or “Abort”? Fortunately,
there’s a simple answer: use the
“lock” statement, the “Monitor” class,
and the “Interrupt” method. Those are
the features that I’ll use for most of
the rest of the paper. For now, you
should ignore the rest of
“System.Threading”, though I’ll
outline it for you section 9.
No, I would avoid ever calling Thread.Abort on your own code. You want your own background thread to complete normally and unwind its stack naturally. The only time I might consider calling Thread.Abort is in a scenario where my code is hosting foreign code on another thread (such as a plugin scenario) and I really want to abort the foreign code.
Instead, in this case, you might consider simply versioning each background request. In the callback, ignore responses that are "out-of-date" since server responses may return in the wrong order. I wouldn't worry too much about aborting a request that's already been sent to the database. If you find your database isn't responsive or is being overwhelmed by too many requests, then consider also using a timer as others have suggested.
Use Thread.Abort only as a last-resort measure when you are exiting application and KNOW that all IMPORTANT resources are released safely.
Otherwise, don't do it. It's even worse then
try
{
// do stuff
}
catch { } // gulp the exception, don't do anything about it
safety net...

Categories

Resources