Develop Reference

Triggering DynamicData cache update using Reactive Subject

As a caveat I'm a novice with Rx (2 weeks) and have been experimenting with using Rx, RxUI and Roland Pheasant's DynamicData.
I have a service that initially loads data from local persistence and then, upon some user (or system) instruction will contact the server (TriggerServer in the example) to get additional or replacement data. The solution I've come up with uses a Subject and I've come across many a site discussing the pros/cons of using them. Although I understand the basics of hot/cold it's all based on reading rather than real world.
So, using the below as a simplified version, is this 'right' way of going about this problem or is there something I haven't properly understood somewhere?
NB: I'm not sure how important it is, but the actual code is taken from a Xamarin.Forms app, that uses RxUI, the user input being a ReactiveCommand.
Example:
using DynamicData;
using System;
using System.Linq;
using System.Reactive;
using System.Reactive.Disposables;
using System.Reactive.Linq;
using System.Reactive.Subjects;
using System.Threading.Tasks;
public class MyService : IDisposable
{
private CompositeDisposable _cleanup;
private Subject<Unit> _serverSubject = new Subject<Unit>();
public MyService()
{
var data = Initialise().Publish();
AllData = data.AsObservableCache();
_cleanup = new CompositeDisposable(AllData, data.Connect());
}
public IObservableCache<MyData, Guid> AllData { get; }
public void TriggerServer()
{
// This is what I'm not sure about...
_serverSubject.OnNext(Unit.Default);
}
private IObservable<IChangeSet<MyData, Guid>> Initialise()
{
return ObservableChangeSet.Create<MyData, Guid>(async cache =>
{
// inital load - is this okay?
cache.AddOrUpdate(await LoadLocalData());
// is this a valid way of doing this?
var sync = _serverSubject.Select(_ => GetDataFromServer())
.Subscribe(async task =>
{
var data = await task.ConfigureAwait(false);
cache.AddOrUpdate(data);
});
return new CompositeDisposable(sync);
}, d=> d.Id);
}
private IObservable<MyData> LoadLocalData()
{
return Observable.Timer(TimeSpan.FromSeconds(3)).Select(_ => new MyData("localdata"));
}
private async Task<MyData> GetDataFromServer()
{
await Task.Delay(2000).ConfigureAwait(true);
return new MyData("serverdata");
}
public void Dispose()
{
_cleanup?.Dispose();
}
}
public class MyData
{
public MyData(string value)
{
Value = value;
}
public Guid Id { get; } = Guid.NewGuid();
public string Value { get; set; }
}
And a simple Console app to run:
public static class TestProgram
{
public static void Main()
{
var service = new MyService();
service.AllData.Connect()
.Bind(out var myData)
.Subscribe(_=> Console.WriteLine("data in"), ()=> Console.WriteLine("COMPLETE"));
while (Continue())
{
Console.WriteLine("");
Console.WriteLine("");
Console.WriteLine($"Triggering Server Call, current data is: {string.Join(", ", myData.Select(x=> x.Value))}");
service.TriggerServer();
}
}
private static bool Continue()
{
Console.WriteLine("Press any key to call server, x to exit");
var key = Console.ReadKey();
return key.Key != ConsoleKey.X;
}
}

Looks very good for first try with Rx
I would suggest few changes:
1) Remove the Initialize() call from the constructor and make it a public method - helps a lot with unit tests and now you can await it if you need to
public static void Main()
{
var service = new MyService();
service.Initialize();
2) Add Throttle to you trigger - this fixes parallel calls to the server returning the same results
3) Don't do anything that can throw in Subscribe, use Do instead:
var sync = _serverSubject
.Throttle(Timespan.FromSeconds(0.5), RxApp.TaskPoolScheduler) // you can pass a scheduler via arguments, or use TestScheduler in unit tests to make time pass faster
.Do(async _ =>
{
var data = await GetDataFromServer().ConfigureAwait(false); // I just think this is more readable, your way was also correct
cache.AddOrUpdate(data);
})
// .Retry(); // or anything alese to handle failures
.Subscribe();

I'm putting what I've come to as my solution just in case there's others that find this while they're wandering the internets.
I ended up removing the Subjects all together and chaining together several SourceCache, so when one changed it pushed into the other and so on. I've removed some code for brevity:
public class MyService : IDisposable
{
private SourceCache<MyData, Guid> _localCache = new SourceCache<MyData, Guid>(x=> x.Id);
private SourceCache<MyData, Guid> _serverCache = new SourceCache<MyData, Guid>(x=> x.Id);
public MyService()
{
var localdata = _localCache.Connect();
var serverdata = _serverCache.Connect();
var alldata = localdata.Merge(serverdata);
AllData = alldata.AsObservableCache();
}
public IObservableCache<MyData, Guid> AllData { get; }
public IObservable<Unit> TriggerLocal()
{
return LoadLocalAsync().ToObservable();
}
public IObservable<Unit> TriggerServer()
{
return LoadServerAsync().ToObservable();
}
}
EDIT: I've changed this again to remove any chaining of caches - I just manage the one cache internally. Lesson is not to post too early.

WCFService calling a Singleton instance from windows service returning empty

I have a static (large)data from database that i need to send to clients so i created a singleton class that get the data from the database and populate the list.
I start the service host inside the windows service, so when an outside call the wcf the data comes empty, what should i do?
[ServiceBehavior(InstanceContextMode = InstanceContextMode.Single)]
public class CacheDataService : ICacheDataService
{
public List<Sale> GetDataFromImobDateById(int idImob, DateTime date)
{
return SalesHelper.Instance.GetDataFromImobDate(idImob, date);
}
}
public class SalesHelper
{
static Lazy<SalesHelper> singleton = new Lazy<SalesHelper>(() => new SalesHelper());
public static SalesHelper Instance { get { return singleton.Value; } }
ICacheData<Sale> _cache;
List<Sale> CacheData = new List<Sale>();
public void SetCache(ICacheData<Sale> cacheData)
{
_cache = cacheData;
}
public void ReloadCache()
{
CacheData.Clear();
GetAllData();
}
public void GetAllData()
{
CacheData = _cache.GetAllData();
}
public List<Sale> GetDataFromImobDate(int idImob, DateTime date)
{
var result = (from r in CacheData
where r.Data_Alteracao.Equals(date)
&& r.Id_Imobiliaria.Equals(idImob)
select r).ToList();
return result;
}
}
and in the Service i start the ServiceHost and the cache
_tempSales = new SalesHelper();
ICacheData<Sale> _cacheSale = new Sale();
_tempSales.SetCache(_cacheSale);
_tempSales.GetAllData();
_service = new ServiceHost(typeof(CacheDataService));

I think your are missing the initialization of SalesHelper.Instance.
doing this new Lazy<SalesHelper>(() => new SalesHelper()); leads to get an intance of _cache not initialized.
So we have a couple of workaround to chose.
One of them is initilize the Intance:
SalesHelper.Instance.SetCache(_cacheSale);
It should look like this:
//_tempSales = new SalesHelper();
ICacheData<Sale> _cacheSale = new Sale();
//_tempSales.SetCache(_cacheSale);
//_tempSales.GetAllData();
SalesHelper.Instance.SetCache(_cacheSale);
SalesHelper.Instance.GetAllData(); //Now it should return the info
_service = new ServiceHost(typeof(CacheDataService));
The other one is replace your prop Intance with a factory method GetInstance() which should receive the cache and set it if it is needed.
Let me know if the first workaround solve your problem.

Testing async method does not give consistent result

I am trying to figure out the reason why my unit test would fail when run together with other unit tests in the solution but pass when run alone. Can anyone show me what am I missing?
The SUT is a class called CompositeClient that is essentially a wrapper class around two other clients. It's main idea is to give priority to one of those clients to be called.
public class CompositeClient : IReceiverChannel
{
private static readonly List<IReceiverChannel> ReceiverChannels = new List<IReceiverChannel>();
public CompositeClient(IReceiverChannel priority, IReceiverChannel normal)
{
ReceiverChannels.Add(priority);
ReceiverChannels.Add(normal);
}
public async Task<IEnumerable<Request>> ReceiveBatchAsync(int batchSize)
{
var req = new List<Request>();
foreach (var channel in ReceiverChannels)
{
req.AddRange(await channel.ReceiveBatchAsync(batchSize - req.Count).ConfigureAwait(false));
if (req.Count >= batchSize)
{
break;
}
}
return req;
}
}
Running the unit test below with all the other unit tests in the solution yield me a failed result. But if I run this test alone, it will pass.
[TestMethod]
public async Task ReceivedRequestShouldComeFromPriorityClientFirst()
{
var normalPriorityClient = GetNormalClientMock();
var highPriorityClient = GetPriorityClientMock();
var compositeClient = new CompositeClient(highPriorityClient, normalPriorityClient);
var requests = await compositeClient.ReceiveBatchAsync(1);
requests.Should().HaveCount(1);
requests.First().Origin.Should().BeSameAs("priority");
normalPriorityClient.CallCount.Should().Be(1); // It will fail here with actual CallCount = 0.
highPriorityClient.CallCount.Should().Be(0);
}
private static ReceiverChannelMock GetNormalClientMock()
{
return new ReceiverChannelMock("normal");
}
private static ReceiverChannelMock GetPriorityClientMock()
{
return new ReceiverChannelMock("priority");
}
private class ReceiverChannelMock : IReceiverChannel
{
private readonly string name;
public ReceiverChannelMock(string name)
{
this.name = name;
}
public int CallCount { get; private set; }
public Task<IEnumerable<Request>> ReceiveBatchAsync(int batchSize)
{
this.CallCount++;
return Task.FromResult<IEnumerable<Request>>(
new List<Request>
{
new Request
{
Origin = this.name
}
});
}
}
Tools used:
Visual Studio 2013
.NET Framework 4.5.2
Resharper 9.2
FluentAssertion

As David pointed out, I overlooked the static field that I declared in the CompositeClient class. Removing the static keyword solved the issue.

Consume HttpContext Cache with threads

I have the following cache implementation for my application:
public static class Keys
{
public const string CacheKey = "cachekey";
}
public interface ICache
{
string QueryCachedData(string param);
}
the data is loaded when the application starts in Global.asax
//Global.asax
protected void Application_Start(object sender, EventArgs e)
{
//instantiates the repository
HttpContext.Current.Application[Keys.CacheKey] = repository.getDataView();
}
the implementation recover the data from HttpContext.Current
public class Cache : ICache
{
private Cache() { }
private static Cache _instance = null;
public static Cache GetInstance()
{
if (_instance == null)
_instance = new Cache();
return _instance;
}
private System.Data.DataView GetCachedData()
{
if (HttpContext.Current.Application[Keys.CacheKey] == null)
{
//instantiates the repository
HttpContext.Current.Application[Keys.CacheKey] = repository.getDataView();
}
return HttpContext.Current.Application[Keys.CacheKey] as System.Data.DataView;
}
private readonly Object _lock = new Object();
public string QueryCachedData(string param)
{
lock (_lock)
{
var data = GetCachedData();
//Execute query
return result;
}
}
}
at some point i need consume some third party web service with the following class using the cache...
public class ThirdPartyWebserviceConsumer
{
ICache _cache;
int _provider;
public ThirdPartyWebserviceConsumer(int provider, ICache cache)
{
_cache = cache;
_provider = provider;
}
public result DoSomething()
{
var info = _cache.QueryCachedData(param);
}
}
...using multi-thread:
public List<Result> Foo(ICache cache, List<int> collectionOfProviders)
{
List<Result> results = new List<Result>();
List<Task> taskList = new List<Task>();
foreach (var provider in collectionOfProviders)
{
var task = new Task<Result>(() => new ThirdPartyWebserviceConsumer(provider, cache).DoSomething());
task.Start();
task.ContinueWith(task =>
{
results.Add(task.Result);
});
taskList.Add(task);
}
Task.WaitAll(taskList.ToArray());
return results;
}
My problem is that HttpContext.Current.Application is null in the thead context.
What options do I have? there are some form to access the HttpContext in thread? or maybe another type of cache that could be shared between the threads?

My problem is that HttpContext.Current.Application is null in the thead context. What options do I have?
HttpContext.Current is bound to the managed thread processing the current request.
If you need data from the current context for another thread, you need to copy that data out of the current context first and pass it to your separate thread.

Caching WebAPI 2

EDIT: For each request, a new instance of controller is created. However, this is not true with Attribute classes. Once they are created, it is used for multiple requests. I hope it helps.
I wrote my own WebAPI (using latest version of WebAPI and .net framework) caching action filter. I am aware about CacheCow & this. However, i wanted mine anyways.
However, there is some issue with my code because i don't get exepected output when i use it in my project on live server. On local machine everything works fine.
I used below code in my blog RSS generator and i cache the data for each category. There are around 5 categories (food, tech, personal etc).
Issue: When i navigate to say api/GetTech it returns me the rss feed items from personal blog category. When i navigate to say api/GetPersonal , it returns me api/Food
I am not able to find the root cause but I think this is due to use of static method/variable. I have double checked that my _cachekey has unique value for each category of my blog.
Can someone point out any issues with this code esp when we have say 300 requests per minute ?
public class WebApiOutputCacheAttribute : ActionFilterAttribute
{
// Cache timespan
private readonly int _timespan;
// cache key
private string _cachekey;
// cache repository
private static readonly MemoryCache _webApiCache = MemoryCache.Default;
/// <summary>
/// Initializes a new instance of the <see cref="WebApiOutputCacheAttribute"/> class.
/// </summary>
/// <param name="timespan">The timespan in seconds.</param>
public WebApiOutputCacheAttribute(int timespan)
{
_timespan = timespan;
}
public override void OnActionExecuting(HttpActionContext ac)
{
if (ac != null)
{
_cachekey = ac.Request.RequestUri.PathAndQuery.ToUpperInvariant();
if (!_webApiCache.Contains(_cachekey)) return;
var val = (string)_webApiCache.Get(_cachekey);
if (val == null) return;
ac.Response = ac.Request.CreateResponse();
ac.Response.Content = new StringContent(val);
var contenttype = (MediaTypeHeaderValue)_webApiCache.Get("response-ct") ?? new MediaTypeHeaderValue("application/rss+xml");
ac.Response.Content.Headers.ContentType = contenttype;
}
else
{
throw new ArgumentNullException("ac");
}
}
public override void OnActionExecuted(HttpActionExecutedContext actionExecutedContext)
{
if (_webApiCache.Contains(_cachekey)) return;
var body = actionExecutedContext.Response.Content.ReadAsStringAsync().Result;
if (actionExecutedContext.Response.StatusCode == HttpStatusCode.OK)
{
lock (WebApiCache)
{
_wbApiCache.Add(_cachekey, body, DateTime.Now.AddSeconds(_timespan));
_webApiCache.Add("response-ct", actionExecutedContext.Response.Content.Headers.ContentType, DateTimeOffset.UtcNow.AddSeconds(_timespan));
}
}
}
}

The same WebApiOutputCacheAttribute instance can be used to cache multiple simultaneous requests, so you should not store cache keys on the instance of the attribute. Instead, regenerate the cache key during each request / method override. The following attribute works to cache HTTP GET requests.
using System;
using System.Linq;
using System.Net.Http;
using System.Net.Http.Headers;
using System.Web.Http.Controllers;
using System.Web.Http.Filters;
using Newtonsoft.Json;
// based on strathweb implementation
// http://www.strathweb.com/2012/05/output-caching-in-asp-net-web-api/
public class CacheHttpGetAttribute : ActionFilterAttribute
{
public int Duration { get; set; }
public ILogExceptions ExceptionLogger { get; set; }
public IProvideCache CacheProvider { get; set; }
private bool IsCacheable(HttpRequestMessage request)
{
if (Duration < 1)
throw new InvalidOperationException("Duration must be greater than zero.");
// only cache for GET requests
return request.Method == HttpMethod.Get;
}
private CacheControlHeaderValue SetClientCache()
{
var cachecontrol = new CacheControlHeaderValue
{
MaxAge = TimeSpan.FromSeconds(Duration),
MustRevalidate = true,
};
return cachecontrol;
}
private static string GetServerCacheKey(HttpRequestMessage request)
{
var acceptHeaders = request.Headers.Accept;
var acceptHeader = acceptHeaders.Any() ? acceptHeaders.First().ToString() : "*/*";
return string.Join(":", new[]
{
request.RequestUri.AbsoluteUri,
acceptHeader,
});
}
private static string GetClientCacheKey(string serverCacheKey)
{
return string.Join(":", new[]
{
serverCacheKey,
"response-content-type",
});
}
public override void OnActionExecuting(HttpActionContext actionContext)
{
if (actionContext == null) throw new ArgumentNullException("actionContext");
var request = actionContext.Request;
if (!IsCacheable(request)) return;
try
{
// do NOT store cache keys on this attribute because the same instance
// can be reused for multiple requests
var serverCacheKey = GetServerCacheKey(request);
var clientCacheKey = GetClientCacheKey(serverCacheKey);
if (CacheProvider.Contains(serverCacheKey))
{
var serverValue = CacheProvider.Get(serverCacheKey);
var clientValue = CacheProvider.Get(clientCacheKey);
if (serverValue == null) return;
var contentType = clientValue != null
? JsonConvert.DeserializeObject<MediaTypeHeaderValue>(clientValue.ToString())
: new MediaTypeHeaderValue(serverCacheKey.Substring(serverCacheKey.LastIndexOf(':') + 1));
actionContext.Response = actionContext.Request.CreateResponse();
// do not try to create a string content if the value is binary
actionContext.Response.Content = serverValue is byte[]
? new ByteArrayContent((byte[])serverValue)
: new StringContent(serverValue.ToString());
actionContext.Response.Content.Headers.ContentType = contentType;
actionContext.Response.Headers.CacheControl = SetClientCache();
}
}
catch (Exception ex)
{
ExceptionLogger.Log(ex);
}
}
public override void OnActionExecuted(HttpActionExecutedContext actionExecutedContext)
{
try
{
var request = actionExecutedContext.Request;
// do NOT store cache keys on this attribute because the same instance
// can be reused for multiple requests
var serverCacheKey = GetServerCacheKey(request);
var clientCacheKey = GetClientCacheKey(serverCacheKey);
if (!CacheProvider.Contains(serverCacheKey))
{
var contentType = actionExecutedContext.Response.Content.Headers.ContentType;
object serverValue;
if (contentType.MediaType.StartsWith("image/"))
serverValue = actionExecutedContext.Response.Content.ReadAsByteArrayAsync().Result;
else
serverValue = actionExecutedContext.Response.Content.ReadAsStringAsync().Result;
var clientValue = JsonConvert.SerializeObject(
new
{
contentType.MediaType,
contentType.CharSet,
});
CacheProvider.Add(serverCacheKey, serverValue, new TimeSpan(0, 0, Duration));
CacheProvider.Add(clientCacheKey, clientValue, new TimeSpan(0, 0, Duration));
}
if (IsCacheable(actionExecutedContext.Request))
actionExecutedContext.ActionContext.Response.Headers.CacheControl = SetClientCache();
}
catch (Exception ex)
{
ExceptionLogger.Log(ex);
}
}
}
Just replace the CacheProvider with your MemoryCache.Default. In fact, the code above uses the same by default during development, and uses azure cache when deployed to a live server.
Even though your code resets the _cachekey instance field during each request, these attributes are not like controllers where a new one is created for each request. Instead, the attribute instance can be repurposed to service multiple simultaneous requests. So don't use an instance field to store it, regenerate it based on the request each and every time you need it.