I am stuck in a scenario.
My code is like below :
Update : its not about how to use data cache, i am already using it and its working , its about expanding it so the method don't make call between the time of expiry and getting new data from external source
object = (string)this.GetDataFromCache(cache, cacheKey);
if(String.IsNullOrEmpty(object))
{
// get the data. It takes 100ms
SetDataIntoCache(cache, cacheKey, object, DateTime.Now.AddMilliseconds(500));
}
So user hit the cache and get data from it if the item expire it calls and get the data from service and save it in case , the problem is , when ever there is a pending request ( request ongoing ) the service send another request because the object is expired . in final there should be max 2-3 calls/ seconds and there are 10-20 calls per seconds to external service .
Is there any optimal way to doing it so no conflict between requests time other then creating own custom class with arrays and time stamps etc?
btw the saving code for cache is-
private void SetDataIntoCache(ObjectCache cacheStore, string cacheKey, object target, DateTime slidingExpirationDuration)
{
CacheItemPolicy cacheItemPolicy = new CacheItemPolicy();
cacheItemPolicy.AbsoluteExpiration = slidingExpirationDuration;
cacheStore.Add(cacheKey, target, cacheItemPolicy);
}
Use Double-checked locking pattern:
var cachedItem = (string)this.GetDataFromCache(cache, cacheKey);
if (String.IsNullOrEmpty(object)) { // if no cache yet, or is expired
lock (_lock) { // we lock only in this case
// you have to make one more check, another thread might have put item in cache already
cachedItem = (string)this.GetDataFromCache(cache, cacheKey);
if (String.IsNullOrEmpty(object)) {
//get the data. take 100ms
SetDataIntoCache(cache, cacheKey, cachedItem, DateTime.Now.AddMilliseconds(500));
}
}
}
This way, while there is an item in your cache (so, not expired yet), all requests will be completed without locking. But if there is no cache entry yet, or it expired - only one thread will get data and put it into the cache.
Make sure you understand that pattern, because there are some caveats while implementing it in .NET.
As noted in comments, it is not necessary to use one "global" lock object to protect every single cache access. Suppose you have two methods in your code, and each of those methods caches object using it's own cache key (but still using the same cache). Then you have to use two separate lock objects, because if you will use one "global" lock object, calls to one method will unnecessary wait for calls to the other method, while they never work with the same cache keys.
I have adapted the solution from Micro Caching in .NET for use with the System.Runtime.Caching.ObjectCache for MvcSiteMapProvider. The full implementation has an ICacheProvider interface that allows swapping between System.Runtime.Caching and System.Web.Caching, but this is a cut down version that should meet your needs.
The most compelling feature of this pattern is that it uses a lightweight version of a lazy lock to ensure that the data is loaded from the data source only 1 time after the cache expires regardless of how many concurrent threads there are attempting to load the data.
using System;
using System.Runtime.Caching;
using System.Threading;
public interface IMicroCache<T>
{
bool Contains(string key);
T GetOrAdd(string key, Func<T> loadFunction, Func<CacheItemPolicy> getCacheItemPolicyFunction);
void Remove(string key);
}
public class MicroCache<T> : IMicroCache<T>
{
public MicroCache(ObjectCache objectCache)
{
if (objectCache == null)
throw new ArgumentNullException("objectCache");
this.cache = objectCache;
}
private readonly ObjectCache cache;
private ReaderWriterLockSlim synclock = new ReaderWriterLockSlim(LockRecursionPolicy.NoRecursion);
public bool Contains(string key)
{
synclock.EnterReadLock();
try
{
return this.cache.Contains(key);
}
finally
{
synclock.ExitReadLock();
}
}
public T GetOrAdd(string key, Func<T> loadFunction, Func<CacheItemPolicy> getCacheItemPolicyFunction)
{
LazyLock<T> lazy;
bool success;
synclock.EnterReadLock();
try
{
success = this.TryGetValue(key, out lazy);
}
finally
{
synclock.ExitReadLock();
}
if (!success)
{
synclock.EnterWriteLock();
try
{
if (!this.TryGetValue(key, out lazy))
{
lazy = new LazyLock<T>();
var policy = getCacheItemPolicyFunction();
this.cache.Add(key, lazy, policy);
}
}
finally
{
synclock.ExitWriteLock();
}
}
return lazy.Get(loadFunction);
}
public void Remove(string key)
{
synclock.EnterWriteLock();
try
{
this.cache.Remove(key);
}
finally
{
synclock.ExitWriteLock();
}
}
private bool TryGetValue(string key, out LazyLock<T> value)
{
value = (LazyLock<T>)this.cache.Get(key);
if (value != null)
{
return true;
}
return false;
}
private sealed class LazyLock<T>
{
private volatile bool got;
private T value;
public T Get(Func<T> activator)
{
if (!got)
{
if (activator == null)
{
return default(T);
}
lock (this)
{
if (!got)
{
value = activator();
got = true;
}
}
}
return value;
}
}
}
Usage
// Load the cache as a static singleton so all of the threads
// use the same instance.
private static IMicroCache<string> stringCache =
new MicroCache<string>(System.Runtime.Caching.MemoryCache.Default);
public string GetData(string key)
{
return stringCache.GetOrAdd(
key,
() => LoadData(key),
() => LoadCacheItemPolicy(key));
}
private string LoadData(string key)
{
// Load data from persistent source here
return "some loaded string";
}
private CacheItemPolicy LoadCacheItemPolicy(string key)
{
var policy = new CacheItemPolicy();
// This ensures the cache will survive application
// pool restarts in ASP.NET/MVC
policy.Priority = CacheItemPriority.NotRemovable;
policy.AbsoluteExpiration = DateTimeOffset.Now.AddMinutes(1);
// Load Dependencies
// policy.ChangeMonitors.Add(new HostFileChangeMonitor(new string[] { fileName }));
return policy;
}
NOTE: As was previously mentioned, you are probably not gaining anything by caching a value that takes 100ms to retrieve for only 500ms. You should most likely choose a longer time period to hold items in the cache. Are the items really that volatile in the data source that they could change that quickly? If so, maybe you should look at using a ChangeMonitor to invalidate any stale data so you don't spend so much of the CPU time loading the cache. Then you can change the cache time to minutes instead of milliseconds.
You will have to use locking to make sure request is not send when cache is expired and another thread is getting it from remote/slow service, it will look something like this (there are better implementations out there that are easier to use, but they require separate classes):
private static readonly object _Lock = new object();
...
object = (string)this.GetDataFromCache(cache, cacheKey);
if(object == null)
{
lock(_Lock)
{
object = (string)this.GetDataFromCache(cache, cacheKey);
if(String.IsNullOrEmpty(object))
{
get the data // take 100ms
SetDataIntoCache(cache, cacheKey, object, DateTime.Now.AddMilliseconds(500));
}
}
}
return object;
Also, you want to make sure your service doesn't return null as it will assume that no cache exists and will try to get the data on every request. That is why more advanced implementations typically use something like CacheObject, which supports null values storage.
By the way, 500 milliseconds is too small time to cache, you will end up lots of CPU cycle just to add/remove cache which will eventually remove cache too soon before any other request can get benefit of cache. You should profile your code to see if it actually benefits.
Remember, cache has lot of code in terms of locking, hashing and many other moving around data, which costs good amount of CPU cycles and remember, all though CPU cycles are small, but in multi threaded, multi connection server, CPU has lot of other things to do.
Original Answer https://stackoverflow.com/a/16446943/85597
private string GetDataFromCache(
ObjectCache cache,
string key,
Func<string> valueFactory)
{
var newValue = new Lazy<string>(valueFactory);
//The line below returns existing item or adds
// the new value if it doesn't exist
var value = cache.AddOrGetExisting(key, newValue, DateTimeOffset.Now.AddMilliseconds(500)) as Lazy<string>;
// Lazy<T> handles the locking itself
return (value ?? newValue).Value;
}
// usage...
object = this.GetDataFromCache(cache, cacheKey, () => {
// get the data...
// this method will be called only once..
// Lazy will automatically do necessary locking
return data;
});
Related
I am working with windows services to get some data from an API
service#1 gets data from "http://api.provider.com/Entity1"
service#2 gets data from "http://api.provider.com/Entity2"
and I have both these services in one .csproj and I use a singleton httpClient to retrieve data from API:
public sealed class Client : HttpClient{
private static readonly object padlock = new object();
private static Client instance = null;
public static Client Instance
{
get
{
if (instance == null)
{
lock (padlock)
{
if (instance == null)
{
instance = new Client();
}
}
}
return instance;
}
}
private Client()
{
DefaultRequestHeaders.Accept.Clear();
DefaultRequestHeaders.Accept.Add(MediaTypeWithQualityHeaderValue.Parse("application/json"));
DefaultRequestHeaders.Add("...", "...");
}
public async Task<string> Get(string url)
{
var result = await GetStringAsync(url);
return result;
}}
but these processes are parallel so the singleton class is a shared static class between the two.And Then I have this class as the consumer fo the first:
class APIHAndler{
public List<obj1> f1()
{
var jsonResult = Client.Instance.Get(url1).Result;
//make list of obj1 out of json
}
public List<obj2> f2()
{
var jsonResult = Client.Instance.Get(url2).Result;
//make list of obj2 out of json
}
}
What I did is to create one instance of the class APIHAndler in each of my services and call f1 and f2 based on the ongoing business and I get this error:
Response status code does not indicate success: 429 (Too Many Requests).
I think it is probably due to the fact of having two different connections open at the same time. but I don't know how to avoid this. If you can help me fix this or have a better solution I will be very happy to hear about.
I don't know how to avoid this
The service you're calling should have documentation that explains what causes a 429. Sometimes it's only a single request at a time; more often it's a certain number of requests within a time window. Either way, you'll need to throttle your requests, and you can build throttling with a SemaphoreSlim.
I have implemented a custom state "blocked" that moves into the enqueued state after certain external requirements have been fulfilled.
Sometimes these external requirements are never fulfilled which causes the job to be stuck in the blocked state. What I'd like to have is for jobs in this state to automatically expire after some configurable time.
Is there any support for such a requirement? There is the ExpirationDate field, but from looking at the code it seems to be only used for final states.
The state is as simple as can be:
internal sealed class BlockedState : IState
{
internal const string STATE_NAME = "Blocked";
public Dictionary<string, string> SerializeData()
{
return new Dictionary<string, string>();
}
public string Name => STATE_NAME;
public string Reason => "Waiting for external resource";
public bool IsFinal => false;
public bool IgnoreJobLoadException => false;
}
and is used simply as _hangfireBackgroundJobClient.Create(() => Console.WriteLine("hello world"), new BlockedState());
At a later stage it is then moved forward via _hangfireBackgroundJobClient.ChangeState(jobId, new EnqueuedState(), BlockedState.STATE_NAME)
I would go for a custom implementation IBackgroundProcess taking example from DelayedJobScheduler
which picks up delayed jobs on a regular basis to enqueue it.
In this custom implementation I would use a JobStorageConnection.GetAllItemsFromSet("blocked") to get all the blocked job ids (where the DelayedJobScheduler uses JobStorageConnection.GetFirstByLowestScoreFromSet)
Then I would get each blocked job data with JobStorageConnection.GetJobData(jobId). For each of them, depending on its CreatedAt field, I would do nothing if the job is not expired, or change its state to another state (Failed ?) if it is expired.
The custom job process can be declared like this :
app.UseHangfireServer(storage, options,
new IBackgroundProcess[] {
new MyCustomJobProcess(
myTimeSpanForExpiration,
(IBackgroundJobStateChanger) new BackgroundJobStateChanger(filterProvider)) });
A difficulty here is to obtain an IBackgroundJobStateChanger as the server does not seem to expose its own.
If you use a custom FilterProvider as option for your server pass its value as filterProvider, else use (IJobFilterProvider) JobFilterProviders.Providers
Can you take advantage of EventWaitHandle?
Have a look at Generic Timout.
For example:
//action : your job
//timeout : your desired ExpirationDate
void DoSomething(Action action, int timeout)
{
EventWaitHandle waitHandle = new EventWaitHandle(false, EventResetMode.ManualReset);
AsyncCallback callback = ar => waitHandle.Set();
action.BeginInvoke(callback, null);
if (!waitHandle.WaitOne(timeout))
{
// Expired here
}
}
I want to log EF query (or transaction) times back to the database so that I can monitor performance in a live system. Note I know that more than one query can happen within my UOW. That is fine, I just want to be able to tell what areas of the system are slowing down etc.
What is the best way for me to do this? My initial thought was to do this in UnitOfWork.Dispose() so that every query would automatically be logged once completed. But this smells a bit to me because I'd have to call Save() to persist the information, but what if the caller didn't want to save?
Is there another, better way I can automatically log the query time to the database?
protected virtual void Dispose(bool disposing)
{
if (this.logQueryToDatabase)
{
var timeTaken = DateTime.UtcNow - this.start;
LogPerformance(this.callingFnName, timeTaken.TotalMilliseconds);
}
this.ctx.Dispose();
}
If you know which actions to measure you can create a simple class to handle this. All you need is to wrap around the action.
The profiler class.
public class Profiler:IDisposable
{
private readonly string _msg;
private Stopwatch _sw;
public Profiler(string msg)
{
_msg = msg;
_sw = Stopwatch.StartNew();
}
public void Dispose()
{
_sw.Stop();
LogPerformance(_msg, _sw.ElapsedMilliseconds);
}
}
The usage:
using (new Profiler("Saving products"))
{
db.SaveChanges();
}
I'm working on a Cloud-Hosted ZipFile creation service.
This is a Cross-Origin WebApi2 service used to provide ZipFiles from a file system that cannot host any server side code.
The basic operation goes like this:
User makes a POST request with a string[] of Urls that correlate to file locations
WebApi reads the array into memory, and creates a ticket number
WebApi returns the ticket number to the user
AJAX callback then redirects the user to a web address with the ticket number appended, which returns the zip file in the HttpResponseMessage
In order to handle the ticket system, my design approach was to set up a Global Dictionary that paired a randomly generated 10 digit number to a List<String> value, and the dictionary was paired to a Queue storing 10,000 entries at a time. (Reference here)
This is partially due to the fact that WebApi does not support Cache
When I make my AJAX call locally, it works 100% of the time. When I make the call remotely, it works about 20% of the time.
When it fails, this is the error I get:
The given key was not present in the dictionary.
Meaning, the ticket number was not found in the Global Dictionary Object.
I've implemented quite a few Lazy Singletons in the last few months, and I've never run into this.
Where did I go wrong?
//Initital POST request, sent to the service with the string[]
public string Post([FromBody]string value)
{
try
{
var urlList = new JavaScriptSerializer().Deserialize<List<string>>(value);
var helper = new Helper();
var random = helper.GenerateNumber(10);
CacheDictionary<String, List<String>>.Instance.Add(random, urlList);
return random;
}
catch (Exception ex)
{
return ex.Message;
}
}
//Response, cut off where the error occurs
public async Task<HttpResponseMessage> Get(string id)
{
try
{
var urlList = CacheDictionary<String, List<String>>.Instance[id];
}
catch (Exception ex)
{
var response = new HttpResponseMessage(HttpStatusCode.InternalServerError)
{
Content = new StringContent(ex.Message)
};
return response;
}
}
//CacheDictionary in its Lazy Singleton form:
public class CacheDictionary<TKey, TValue>
{
private Dictionary<TKey, TValue> dictionary;
private Queue<TKey> keys;
private int capacity;
private static readonly Lazy<CacheDictionary<String, List<String>>> lazy =
new Lazy<CacheDictionary<String, List<String>>>(() => new CacheDictionary<String, List<String>>(10000));
public static CacheDictionary<String, List<String>> Instance { get { return lazy.Value; } }
private CacheDictionary(int capacity)
{
this.keys = new Queue<TKey>(capacity);
this.capacity = capacity;
this.dictionary = new Dictionary<TKey, TValue>(capacity);
}
public void Add(TKey key, TValue value)
{
if (dictionary.Count == capacity)
{
var oldestKey = keys.Dequeue();
dictionary.Remove(oldestKey);
}
dictionary.Add(key, value);
keys.Enqueue(key);
}
public TValue this[TKey key]
{
get { return dictionary[key]; }
}
}
More Error Detail
at System.Collections.Generic.Dictionary`2.get_Item(TKey key)
at ZipperUpper.Models.CacheDictionary`2.get_Item(TKey key)
I think you will find it's to do with where you're locating your Global Dictionary. For example, if this was a web farm, and your dictionary was in Session, one instance of your app could access a different Session from another instance, unless the Session state handling was set up correctly. In your case it's in the cloud, so you will need to make provision in the same kind of way for related requests and responses being handled by different machines. Therefore one could send out the key, and another could receive the AJAX redirect but not have that key in its own "global" data.
I have an ASP.NET (4.0) web site that has few operations running on the server independently from users requests.
I extensively use cache during web requests and keep objects inside of HttpContext.Current.Cache.
The problem is that for all threads that are not caused by user requests HttpContext.Current is null and I can't access Cache.
In order to access HttpContext.Current.Cache I plan to use the following:
class CacheWrapper
{
public void Insert(string key, Object obj)
{
Cache cache = CacheInstance;
if (cache == null)
{
return;
}
cache.Insert(key, obj);
}
public Object Get(string key)
{
Cache cache = CacheInstance;
if (cache == null)
{
return;
}
return cache.Get(key);
}
private Cache CacheInstance
{
get
{
if (_cache == null)
{
if (HttpContext.Current == null)
{
return null;
}
lock (_lock)
{
if (_cache == null)
{
_cache = HttpContext.Current.Cache;
}
}
}
return _cache;
}
}
}
Therefore until the 1st request to the web site is made, no any caching will be applied, but once at least one request made, the reference to HttpContext.Current.Cache will be saved and all background server operations will be able to access cache.
Question:
Can I rely that once obtained HttpContext.Current.Cache will be always valid?
Thank you very much. Any ideas or comments with regards to the idea are more than welcome!
Instead of using HttpContext.Current.Cache, I would recommend using HttpRuntime.Cache - both properties point to the same cache except the latter is not dependent on a current context like the former is.
If you're writing a general cache wrapper to be used in a number of different types of applications/services, you may want to take a look at ObjectCache and MemoryCache to see if they would be useful for your need.