Develop Reference

Rx.Net GroupBy, How to subscribe multiple observers to a specific Group

I am taking my first leap into the world of Rx and finding it difficult to get the desired results, especially with the GroupBy operator, so any help would be much appreciated.
How can I subscribe multiple observers to a specific group?
My requirements are:
I have a DataProvider class that makes http Api requests at regular intervals.
The http response is a List<Item>. Each Item has a unique Id property.
I need to process each Item as a separate stream based on its Id, which looks like a case for GroupBy.
Each group needs its own pipeline where:
It Starts with a specific value (StartWith operator)
It Buffers the previous Item for comparison with the current Item (Buffer(2,1) operator)
If the current Item is different to the previous (Where) emit the current Item
The result is an IObservable<Item> of changes (ChangeStream). I am no longer dealing with a specific group.
How can I stay within the group pipeline and allow multiple subscribers to Subscribe to a specific group?
Observers can subscribe early (before the Item.Id has appeared on the response stream and before the group is created)
Observers can subscribe late (after the Item.Id has appeared on the response stream and the group has been created)
Late subscribers should receive the last change for the Item.Id (Replay(1)) but I can’t seem to figure this part out either.
What is the Rx way to Multicast a specific group? Any help / advice would be much appreciated. I have provided sample code below.
using System;
using System.Collections.Generic;
using System.Linq;
using System.Reactive.Linq;
using System.Reactive.Subjects;
using System.Threading;
namespace RxTest
{
class Program
{
static void Main(string[] args)
{
var dataService = new MockDataService();
// How do I subscribe to a specific group?
// Eg. I am only interested in changes to Items where Id == 1
// Subscribers can be early (before the stream is hot)
var item1Stream = dataService.SubscribeToItem(1);
// There can be multiple subscribers to a group
var item1Stream2 = dataService.SubscribeToItem(1);
Console.WriteLine("Press Any Key to Start");
Console.ReadLine();
dataService.Start();
// Subscribers can be late (Eg. Subscriber to Item Id == 2 after it has emitted items)
Thread.Sleep(2000);
var item2Stream = dataService.SubscribeToItem(2);
// Subscribers can be early (After connect but before the Item Id appears on the Stream (before group creation))
// Eg. Subscribe to group 4 (Group 4 doesn't get created until 20s after connect in this example)
var item4Stream = dataService.SubscribeToItem(4);
// What is the Rx way to Multicast a Group?
Console.WriteLine("Press Any Key to Exit");
Console.ReadLine();
dataService.Stop();
}
}
public class MockDataService
{
private readonly IConnectableObservable<Item> _itemsStream;
private IDisposable _itemsSubscription;
private readonly IObservable<Item> _changeStream;
private IDisposable _changeSubscription;
public MockDataService()
{
// Simulate Http response pipeline.
//// Time: 1s...............10s..............20s.....etc
//// stream: [[1][2]]repeat...[[2][3]]repeat...[[3][4]]repeat...
IObservable<List<Item>> responseStream = Observable.Interval(TimeSpan.FromSeconds(1))
.Take(50)
.Select(tick =>
{
// Every 10 ticks an item drops off the stream and a new one starts
// Every 2 ticks the Item value increases to generate a change.
int rangeStart = Math.Min(((int)tick / 10) + 1, 5);
return Enumerable.Range(rangeStart, 2).Select(id => new Item(id, (int)tick / 2)).ToList();
});
// Flatten the list into IObservable<Item>
//// Time: 1s.............10s............20s.....etc
//// stream: [1][2]repeat...[2][3]repeat...[3][4]repeat...
_itemsStream = responseStream
.SelectMany(list => list)
.Publish();
// Split into groups by Item.Id and process each group for changes
// ChangeStream is an IObservable<Item> that have changes.
_changeStream = _itemsStream
.GroupBy(item => item.Id)
.SelectMany(grp =>
grp
// Pipeline for each group.
.StartWith(new Item(grp.Key, -1)) // Initial item from Db
//.TakeUntil(Item => Item.IsComplete()) // Logic to complete the group
.LogConsoleWithThread($"Group: {grp.Key}")
.Buffer(2, 1)
.Where(buffer => buffer.Count == 2 && buffer[0].HasChanges(buffer[1]))
.Select(buffer => buffer[1])
.LogConsoleWithThread($"Group.Change : {grp.Key}")
// How do I push changes in this group to Zero..Many subscribed Observers?
// I would also like to Replay(1) to all late subscribers to a group.
);
}
/// <summary>
/// How to get the IObservable for a specific group?
/// </summary>
/// <param name="itemId"></param>
/// <returns></returns>
public IObservable<Item> SubscribeToItem(int itemId)
{
// ????
return null;
}
public void Start()
{
_changeSubscription = _changeStream.SubscribeConsole("ChangeStream");
_itemsSubscription = _itemsStream.Connect();
}
public void Stop()
{
_changeSubscription.Dispose();
_itemsSubscription.Dispose();
}
}
public class Item
{
public int Id { get; private set; }
public int Value { get; private set; }
public Item(int id, int value)
{
Id = id;
Value = value;
}
public bool HasChanges(Item compareItem)
{
return this.Value != compareItem.Value;
}
public override string ToString()
{
return $"Item: Id={Id} Value={Value}";
}
}
public static class RxExtensions
{
public static IDisposable SubscribeConsole<T>(this IObservable<T> observable, string name = "")
{
return observable.Subscribe(new ConsoleObserver<T>(name));
}
/// <summary>
/// Logs to the Console the subscriptions and emissions done on/by the observable
/// each log message also includes the thread it happens on
/// </summary>
/// <typeparam name="T">The Observable Type</typeparam>
/// <param name="observable">The Observable to log.</param>
/// <param name="name">An optional name prefix that will be added before each notification</param>
/// <returns></returns>
public static IObservable<T> LogConsoleWithThread<T>(this IObservable<T> observable, string name = "")
{
return Observable.Defer(() =>
{
Console.WriteLine("{0} Subscription happened on Thread: {1}", name, Thread.CurrentThread.ManagedThreadId);
return observable.Do(
x => Console.WriteLine("{0} - OnNext({1}) Thread: {2}", name, x, Thread.CurrentThread.ManagedThreadId),
ex =>
{
Console.WriteLine("{0} - OnError Thread:{1}", name, Thread.CurrentThread.ManagedThreadId);
Console.WriteLine("\t {0}", ex);
},
() => Console.WriteLine("{0} - OnCompleted() Thread {1}", name, Thread.CurrentThread.ManagedThreadId));
});
}
}
/// <summary>
/// An observer that outputs to the console each time the OnNext, OnError or OnComplete occurs
/// </summary>
/// <typeparam name="T"></typeparam>
public class ConsoleObserver<T> : IObserver<T>
{
private readonly string _name;
public ConsoleObserver(string name = "")
{
_name = name;
}
public void OnNext(T value)
{
Console.WriteLine("{0} - OnNext({1})", _name, value);
}
public void OnError(Exception error)
{
Console.WriteLine("{0} - OnError:", _name);
Console.WriteLine("\t {0}", error);
}
public void OnCompleted()
{
Console.WriteLine("{0} - OnCompleted()", _name);
}
}
}

You probably need a specialized publishing operator, because the existing ones (Publish, PublishLast and Replay) are too narrow or too broad for your needs. So you'll need to use the Multicast operator, supplied with a custom replay subject that buffers only the last element per key. Here is a basic implementation of such a subject:
public class ReplayLastPerKeySubject<T, TKey> : ISubject<T>
{
private readonly Func<T, TKey> _keySelector;
private readonly ReplaySubject<ReplaySubject<T>> _subjects;
private readonly IObservable<T> _mergedSubjects;
private readonly Dictionary<TKey, ReplaySubject<T>> _perKey;
public ReplayLastPerKeySubject(Func<T, TKey> keySelector)
{
_keySelector = keySelector;
_subjects = new ReplaySubject<ReplaySubject<T>>();
_mergedSubjects = _subjects.Merge();
_perKey = new Dictionary<TKey, ReplaySubject<T>>();
}
public void OnNext(T value)
{
var key = _keySelector(value);
ReplaySubject<T> subject;
if (!_perKey.TryGetValue(key, out subject))
{
subject = new ReplaySubject<T>(1);
_perKey.Add(key, subject);
_subjects.OnNext(subject);
}
subject.OnNext(value);
}
public void OnCompleted()
{
// All subjects, inner and outer, must be completed
_subjects.OnCompleted();
_subjects.Subscribe(subject => subject.OnCompleted());
}
public void OnError(Exception error)
{
// Faulting the master (outer) subject is enough
_subjects.OnError(error);
}
public IDisposable Subscribe(IObserver<T> observer)
{
return _mergedSubjects.Subscribe(observer);
}
}
This implementation is based on an answer of a similar question, written by an RX expert. The original answer uses a Concat observable for subscribing the observers, while this one uses a Merge observable, so I am not 100% sure about its correctness and efficiency.
Having such an implementation in place, the rest is easy. You first create a published version of your original observable:
var published = YourObservable
.Multicast(new ReplayLastPerKeySubject<Item, int>(x => x.Id)))
.RefCount();
And finally you can create a change stream for a specific key, by using the Where operator:
var changeStream13 = published.Where(x => x.Id == 13);

Asp.Net Core Rules Engine with Programmatic Dependency Injection - Constructor on type 'type' not found

I'v developed a Rules Engine library, called RulesChain, that works perfectly when the rules doesn't need that any dependency to be injected.
The primary goal with this library is to simplify writing business rules in .NET environment based on Rules Design Pattern and Chain of Responsability Pattern to work like ASP.Net Core middlewares works.
When I need to have any dependency injected I get this error:
System.MissingMethodException: 'Constructor on type 'AspNetCoreRulesChainSample.Rules.ShoppingCartRules.IsValidCupomRule' not found.'
What is the problem?
My abstract Rule class needs to receive the next rule to be called on it's constructor. But I can't add the put an specific Rule on constructor because the chain is resolved on RuleChain class
How it Works?
Basically all rules have a ShouldRun method that defines if the run method should be called a Run method that applies the Business Rule. And the Invoke method that is called inside the rule when it needs to call the next rule.
This is the rule with dependency injection that throws an error:
public class IsValidCupomRule : Rule<ApplyDiscountContext>
{
private ISalesRepository _salesRepository;
public IsValidCupomRule(Rule<ApplyDiscountContext> next, ISalesRepository salesRepository) : base(next)
{
_salesRepository = salesRepository;
}
public override ApplyDiscountContext Run(ApplyDiscountContext context)
{
// Gets 7% of discount;
var myDiscount = context.Context.Items.Sum(i => i.Price * 0.07M);
context = _next.Invoke(context) ?? context;
// Only apply first order disccount if the discount applied by the other rules are smaller than this
if (myDiscount > context.DiscountApplied)
{
context.DiscountApplied = myDiscount;
context.DiscountTypeApplied = "Cupom";
}
return context;
}
public override bool ShouldRun(ApplyDiscountContext context)
{
return !string.IsNullOrWhiteSpace(context.Context.CupomCode)
&& context.Context.Items?.Count > 1
&& _salesRepository.IsCupomAvaliable(context.Context.CupomCode);
}
}
A basic rule without dependency is like that.
public class BirthdayDiscountRule : Rule<ApplyDiscountContext>
{
public BirthdayDiscountRule(Rule<ApplyDiscountContext> next) : base(next)
{ }
public override ApplyDiscountContext Run(ApplyDiscountContext context)
{
// Gets 10% of discount;
var birthDayDiscount = context.Context.Items.Sum(i => i.Price * 0.1M);
context = _next.Invoke(context);
// Only apply birthday disccount if the discount applied by the other rules are smaller than this
if (birthDayDiscount > context.DiscountApplied)
{
context.DiscountApplied = birthDayDiscount;
context.DiscountTypeApplied = "Birthday Discount";
}
return context;
}
public override bool ShouldRun(ApplyDiscountContext context)
{
var dayAndMonth = context.ClientBirthday.ToString("ddMM");
var todayDayAndMonth = DateTime.Now.ToString("ddMM");
return dayAndMonth == todayDayAndMonth;
}
}
And the abstract rule is that:
public abstract class Rule<T> : IRule<T>
{
protected readonly Rule<T> _next;
protected Rule(Rule<T> next)
{
_next = next;
}
/// <summary>
/// Valides if the rules should be executed or not
/// </summary>
/// <returns></returns>
public abstract bool ShouldRun(T context);
/// <summary>
/// Executes the rule
/// </summary>
/// <returns></returns>
public abstract T Run(T context);
public virtual T Invoke(T context)
{
if(ShouldRun(context))
return Run(context);
else
return _next != null
? _next.Invoke(context)
: context;
}
}
To create my chain of rules I just need to do this:
public ShoppingCart ApplyDiscountOnShoppingCart(ShoppingCart shoppingCart)
{
// Create the chain
var shoppingCartRuleChain = new RuleChain<ApplyDiscountContext>()
.Use<IsValidCupomRule>()
.Use<BirthdayDiscountRule>()
.Use<FirstOrderDiscountRule>()
.Build();
// Create the chain context
var shoppingCartRuleContext = new ApplyDiscountContext(shoppingCart);
shoppingCartRuleContext.Properties["IsFirstOrder"] = true;
shoppingCartRuleContext.ClientBirthday = DateTime.UtcNow;
// Invoke the RulesChain
shoppingCartRuleContext = shoppingCartRuleChain.Invoke(shoppingCartRuleContext);
// Get data form the Chain result and return a ShoppingCart with new data.
shoppingCart.Discount = shoppingCartRuleContext.DiscountApplied;
shoppingCart.DiscountType = shoppingCartRuleContext.DiscountTypeApplied;
return shoppingCart;
}
The principal point for me here is that I can put any Rule in the .Use<IRule>() call and it permits that the rules does not depends on each other and the chain can be changed without the need of refactoring each rule. I'm doing this on Build() method.
This methos just invert the order of each rule on the chain and creates a new instance of each rule, and adds the last Rule instance as the next Rule of he new Rule.
This is the RuleChain class
public class RuleChain<T> : IRuleChain<T>
{
private readonly IList<Type> _components = new List<Type>();
public IRuleChain<T> Use<TRule>()
{
_components.Add(typeof(TRule));
return this;
}
public IRule<T> Build()
{
IRule<T> app = EndOfChainRule<T>.EndOfChain();
foreach (var component in _components.Reverse())
{
app = (IRule<T>)Activator.CreateInstance(component,app);
}
return app;
}
}
Here is how I instantiate new Rules with the next Rule: app = (IRule<T>)Activator.CreateInstance(component,app);
Other information that may be useful:
This is how I resolve Dependencies on IoC module
public static class Modules
{
public static void AddRepository(this IServiceCollection services)
{
services.AddScoped<ISalesRepository, SalesRepository>();
}
public static void AddRules(this IServiceCollection services)
{
services.AddScoped<IsValidCupomRule>();
services.AddScoped<FirstOrderDiscountRule>();
services.AddScoped<BirthdayDiscountRule>();
services.AddScoped<ShoppingCartRulesChain>();
}
}
My startup.cs Configure is this:
public void ConfigureServices(IServiceCollection services)
{
services.AddRepository();
services.AddRules();
services.Configure<CookiePolicyOptions>(options =>
{
// This lambda determines whether user consent for non-essential cookies is needed for a given request.
options.CheckConsentNeeded = context => true;
options.MinimumSameSitePolicy = SameSiteMode.None;
});
services.AddMvc().SetCompatibilityVersion(CompatibilityVersion.Version_2_1);
}
Whats my question?
How can I instantiate a new class based on the same Rule<T> class and with dependencies of IServiceCollection?
The RulesChain source code is available at: https://github.com/lutticoelho/RulesChain
This sample source code is available at: https://github.com/lutticoelho/AspNetCoreRulesChainSample
If anyone needs more information about, or to put more code on the question feel free to ask at the comments and I'll provide any changes needed in this question.

Now there is a lot to unpack here. First observation would be with the RuleChain class.
If the intention is to allow for Dependency Injection via constructor injection then the current design of the class will need to be refactored to allow for that.
Since the current design is modeled behind the Asp.Net Core Middleware pipeline, I would suggest using delegates to store and handle the desired invocation.
First define a delegate to handle the rule processing
/// <summary>
/// A function that can process a <see cref="TContext"/> dependent rule.
/// </summary>
/// <typeparam name="TContext"></typeparam>
/// <param name="context"></param>
/// <returns>A task that represents the completion of rule processing</returns>
public delegate Task RuleHandlingDelegate<TContext>(TContext context);
The advantage of using the delegate is that it can be late bound to an actual implementation after all the necessary dependencies have be resolved.
Also note that this generic delegate definition uses Task to allow for asynchronous operations
This does require a change to the IRuleChain<T> definition.
/// <summary>
/// Defines a class that provides the mechanisms to configure an application's rules pipeline execution.
/// </summary>
/// <typeparam name="TContext">The context shared by all rules in the chain</typeparam>
public interface IRuleChain<TContext> {
/// <summary>
/// Adds a rule to the application's request chain.
/// </summary>
/// <returns>The <see cref="IRuleChain{T}"/>.</returns>
IRuleChain<TContext> Use<TRule>();
/// <summary>
/// Builds the delegate used by this application to process rules executions.
/// </summary>
/// <returns>The rules handling delegate.</returns>
RuleHandlingDelegate<TContext> Build();
}
And the implementation.
In order to allow other arguments to be injected into the rule implementation, the chain will need to be able to resolve constructor argument.
public abstract class RuleChain<TContext> : IRuleChain<TContext> {
private readonly Stack<Func<RuleHandlingDelegate<TContext>, RuleHandlingDelegate<TContext>>> components =
new Stack<Func<RuleHandlingDelegate<TContext>, RuleHandlingDelegate<TContext>>>();
private bool built = false;
public RuleHandlingDelegate<TContext> Build() {
if (built) throw new InvalidOperationException("Chain can only be built once");
var next = new RuleHandlingDelegate<TContext>(context => Task.CompletedTask);
while (components.Any()) {
var component = components.Pop();
next = component(next);
}
built = true;
return next;
}
public IRuleChain<TContext> Use<TRule>() {
components.Push(createDelegate<TRule>);
return this;
}
protected abstract object GetService(Type type, params object[] args);
private RuleHandlingDelegate<TContext> createDelegate<TRule>(RuleHandlingDelegate<TContext> next) {
var ruleType = typeof(TRule);
MethodInfo methodInfo = getValidInvokeMethodInfo(ruleType);
//Constructor parameters
object[] constructorArguments = new object[] { next };
object[] dependencies = getDependencies(ruleType, GetService);
if (dependencies.Any())
constructorArguments = constructorArguments.Concat(dependencies).ToArray();
//Create the rule instance using the constructor arguments (including dependencies)
object rule = GetService(ruleType, constructorArguments);
//return the delegate for the rule
return (RuleHandlingDelegate<TContext>)methodInfo
.CreateDelegate(typeof(RuleHandlingDelegate<TContext>), rule);
}
private MethodInfo getValidInvokeMethodInfo(Type type) {
//Must have public method named Invoke or InvokeAsync.
var methodInfo = type.GetMethod("Invoke") ?? type.GetMethod("InvokeAsync");
if (methodInfo == null)
throw new InvalidOperationException("Missing invoke method");
//This method must: Return a Task.
if (!typeof(Task).IsAssignableFrom(methodInfo.ReturnType))
throw new InvalidOperationException("invalid invoke return type");
//and accept a first parameter of type TContext.
ParameterInfo[] parameters = methodInfo.GetParameters();
if (parameters.Length != 1 || parameters[0].ParameterType != typeof(TContext))
throw new InvalidOperationException("invalid invoke parameter type");
return methodInfo;
}
private object[] getDependencies(Type middlewareType, Func<Type, object[], object> factory) {
var constructors = middlewareType.GetConstructors().Where(c => c.IsPublic).ToArray();
var constructor = constructors.Length == 1 ? constructors[0]
: constructors.OrderByDescending(c => c.GetParameters().Length).FirstOrDefault();
if (constructor != null) {
var ctorArgsTypes = constructor.GetParameters().Select(p => p.ParameterType).ToArray();
return ctorArgsTypes
.Skip(1) //Skipping first argument since it is suppose to be next delegate
.Select(parameter => factory(parameter, null)) //resolve other parameters
.ToArray();
}
return Array.Empty<object>();
}
}
With this abstract chain, it is now the responsibility of it's implementation to define how to resolve any dependencies.
Following the sample context, that is simple enough. Since using the default DI extension, then the chain should depend on IServiceProvider for types whose arguments are not known and Activator for those with provided constructor arguments.
public class DiscountRuleChain : RuleChain<ApplyDiscountContext> {
private readonly IServiceProvider services;
public DiscountRuleChain(IServiceProvider services) {
this.services = services;
}
protected override object GetService(Type type, params object[] args) =>
args == null || args.Length == 0
? services.GetService(type)
: Activator.CreateInstance(type, args);
}
With all of the above provided so far, there were some changes that allowed for a cleaner design.
Specifically IRule<TContext> and its default implementation.
public interface IRule<TContext> {
Task Invoke(TContext context);
}
public abstract class Rule<TContext> : IRule<TContext> {
protected readonly RuleHandlingDelegate<TContext> next;
protected Rule(RuleHandlingDelegate<TContext> next) {
this.next = next;
}
public abstract Task Invoke(TContext context);
}
Any Context specific rules can now be abstracted to target a specific domain
For example
public abstract class DiscountRule : Rule<ApplyDiscountContext> {
protected DiscountRule(RuleHandlingDelegate<ApplyDiscountContext> next) : base(next) {
}
}
This simplified the implementations specific to discounts in the sample and allowed for dependencies to be injected
IsValidCupomRule
public class IsValidCupomRule : DiscountRule {
private readonly ISalesRepository _salesRepository;
public IsValidCupomRule(RuleHandlingDelegate<ApplyDiscountContext> next, ISalesRepository salesRepository)
: base(next) {
_salesRepository = salesRepository;
}
public override async Task Invoke(ApplyDiscountContext context) {
if (cupomAvailable(context)) {
// Gets 7% of discount;
var myDiscount = context.Context.Items.Sum(i => i.Price * 0.07M);
await next.Invoke(context);
// Only apply discount if the discount applied by the other rules are smaller than this
if (myDiscount > context.DiscountApplied) {
context.DiscountApplied = myDiscount;
context.DiscountTypeApplied = "Cupom";
}
} else
await next(context);
}
private bool cupomAvailable(ApplyDiscountContext context) {
return !string.IsNullOrWhiteSpace(context.Context.CupomCode)
&& context.Context.Items?.Count > 1
&& _salesRepository.IsCupomAvaliable(context.Context.CupomCode);
}
}
FirstOrderDiscountRule
public class FirstOrderDiscountRule : DiscountRule {
public FirstOrderDiscountRule(RuleHandlingDelegate<ApplyDiscountContext> next) : base(next) { }
public override async Task Invoke(ApplyDiscountContext context) {
if (shouldRun(context)) {
// Gets 5% of discount;
var myDiscount = context.Context.Items.Sum(i => i.Price * 0.05M);
await next.Invoke(context);
// Only apply discount if the discount applied by the other rules are smaller than this
if (myDiscount > context.DiscountApplied) {
context.DiscountApplied = myDiscount;
context.DiscountTypeApplied = "First Order Discount";
}
} else
await next.Invoke(context);
}
bool shouldRun(ApplyDiscountContext context) {
return (bool)(context.Properties["IsFirstOrder"] ?? false);
}
}
The following test was used to verify expected behavior of the rules engine.
[TestClass]
public class RulesEngineTests {
[TestMethod]
public async Task Should_Apply_Cupom_Discount() {
//Arrange
var cupomCode = "cupomCode";
var services = new ServiceCollection()
.AddSingleton<ISalesRepository>(sp =>
Mock.Of<ISalesRepository>(_ => _.IsCupomAvaliable(cupomCode) == true)
)
.BuildServiceProvider();
// Create the chain
var shoppingCartRuleChain = new DiscountRuleChain(services)
.Use<IsValidCupomRule>()
.Use<FirstOrderDiscountRule>()
.Build();
ShoppingCart shoppingCart = new ShoppingCart {
CupomCode = cupomCode,
Items = new List<ShoppingCartItem> {
new ShoppingCartItem { Price = 10M },
new ShoppingCartItem { Price = 10M },
}
};
var expectedDiscountType = "Cupom";
var expectedDiscountApplied = 1.40M;
// Create the chain context
var shoppingCartRuleContext = new ApplyDiscountContext(shoppingCart);
shoppingCartRuleContext.Properties["IsFirstOrder"] = true;
shoppingCartRuleContext.ClientBirthday = DateTime.UtcNow;
//Act
await shoppingCartRuleChain.Invoke(shoppingCartRuleContext);
// Get data from the context result and verify new data.
shoppingCart.Discount = shoppingCartRuleContext.DiscountApplied;
shoppingCart.DiscountType = shoppingCartRuleContext.DiscountTypeApplied;
//Assert (using FluentAssertions)
shoppingCart.DiscountType.Should().Be(expectedDiscountType);
shoppingCart.Discount.Should().Be(expectedDiscountApplied);
}
}
Note how the dependency to be injected was mocked to test the expected behavior in isolation.

How to return an empty column with iQueryable

I've a method what obtains a list of Users filtering by an specific field. This method is useful for a lot of functionalities of my application, but in other cases is too slow.
The user's table have a field what contains data of pictures, so it's a heavy field. Now i'm developing a functionality what not need this field and i'm searching the way to don't return it, or return it as empty for streamline the process
I'm working with c#, obtaining a filteredList of Users from UnitOfWork repository with "GetByFilter" function.
UserController.cs
/// <summary>
/// Get by Filter
/// </summary>
/// <param name="filter">user filters</param>
/// <returns></returns>
[Route("functionRoute")]
[HttpPost]
public IHttpActionResult GetUsersByFilter([FromBody] UserFilter filter)
{
try
{
UserService service= new UserService ();
List<User> list = service.GetByFilter(filter).ToList();
List<UserCE> listCE = Mapper.Map<List<UserCE>>(list);
return Ok(listCE);
}
catch (Exception ex)
{
TraceManager.LogError(ex);
return InternalServerError(ex);
}
}
UserService.cs
public List<User> GetByFilter(UserFilter filter)
{
return _unitOfWork.UserRepository.GetByFilter(filter).ToList();
}
UserRepository.cs
public IQueryable<User> GetByFilter(UserFilter filter)
{
return Get_internal(filter);
}
private IQueryable<User> Get_internal(UserFilter filter)
{
IQueryable<User> users = _context.Users;
if (filter.Deleted != null)
{
users = users.Where(u => u.Deleted == filter.Deleted );
}
return users;
}
I try to clear column later, but the process continues being too heavy. How i can streamline this process?

I would change your Get_internal() function like so:
(have not tested this, I wrote the code purely from memory, but it should probably work with a few minor tweaks)
private IQueryable<User> Get_internal(UserFilter filter)
{
IQueryable<User> users = _context.Users;
if (filter.Deleted != null)
{
users = users.Where(u => u.Deleted == filter.Deleted );
}
return users.Select(x => new User() {
Id = x.Id,
Name = x.Name,
//every property except your image property
});
}
This will return a new User object for every row, but will only fill the properties you are selecting explicitly. So if you don't select your image property, this will not get selected.
If you check the generated SQL, you should also see that this column is never selected.

By convention, all public properties with a getter and a setter will be included in the model.
To exclude properties you will need to either use Data Annotations or the Fluent API
Data Annotations:
public class User
{
public int Id { get; set; }
[NotMapped]
public string IgnoredField { get; set; }
}
Fluent API:
public class MyContext : DbContext
{
public DbSet<User> Users{ get; set; }
protected override void OnModelCreating(ModelBuilder modelBuilder)
{
modelBuilder.Entity<User>()
.Ignore(b => b.IgnoredField);
}
}

Finnaly after try some things, I decided to apply the solution proposed by Steven, but with few minor tweaks. It gave problems in "Get_Internal", so I decided (although it's not the best option) do it in the controller.
the code looks (more or less) like this:
/// <summary>
/// Get by Filter
/// </summary>
/// <param name="filter">user filters</param>
/// <returns></returns>
[Route("functionRoute")]
[HttpPost]
public IHttpActionResult GetUsersByFilter([FromBody] UserFilter filter)
{
try
{
UserService service= new UserService ();
List<User> list = service.GetByFilter(filter).Select(x => new User() {
Id = x.Id,
Name = x.Name,
//every property except your image property
}).ToList();
List<UserCE> listCE = Mapper.Map<List<UserCE>>(list);
return Ok(listCE);
}
catch (Exception ex)
{
TraceManager.LogError(ex);
return InternalServerError(ex);
}
}

Reactive services. Grouping and caching streams

New: Entire source code with tests is now at https://github.com/bboyle1234/ReactiveTest
Let's imagine we have a view state object that is able to be updated by small partial view change events. Here are some example models of the total view, the incremental view update events and the accumulator function Update that builds the total view:
interface IDeviceView : ICloneable {
Guid DeviceId { get; }
}
class DeviceTotalView : IDeviceView {
public Guid DeviceId { get; set; }
public int Voltage { get; set; }
public int Currents { get; set; }
public object Clone() => this.MemberwiseClone();
}
class DeviceVoltagesUpdateView : IDeviceView {
public Guid DeviceId { get; set; }
public int Voltage { get; set; }
public object Clone() => this.MemberwiseClone();
}
class DeviceCurrentsUpdateView : IDeviceView {
public Guid DeviceId { get; set; }
public int Current { get; set; }
public object Clone() => this.MemberwiseClone();
}
class DeviceUpdateEvent {
public DeviceTotalView View;
public IDeviceView LastUpdate;
}
static DeviceUpdateEvent Update(DeviceUpdateEvent previousUpdate, IDeviceView update) {
if (update.DeviceId != previousUpdate.View.DeviceId) throw new InvalidOperationException("Device ids do not match (numskull exception).");
var view = (DeviceTotalView)previousUpdate.View.Clone();
switch (update) {
case DeviceVoltagesUpdateView x: {
view.Voltage = x.Voltage;
break;
}
case DeviceCurrentsUpdateView x: {
view.Currents = x.Current;
break;
}
}
return new DeviceUpdateEvent { View = view, LastUpdate = update };
}
Next, let's imagine we already have an injectable service that is able to produce an observable stream of the small update events for all devices, and that we want to create a service that can produce an aggregated view stream for individual devices.
Here is the interface of the service we want to create:
interface IDeviceService {
/// <summary>
/// Gets an observable that produces aggregated update events for the device with the given deviceId.
/// On subscription, the most recent event is immediately pushed to the subscriber.
/// There can be multiple subscribers.
/// </summary>
IObservable<DeviceUpdateEvent> GetDeviceStream(Guid deviceId);
}
How can I implement this interface and its requirements using the reactive extensions in the System.Reactive v4 library, targeting .netstandard2.0? Here's my boiler code with comments and that's as far as I've been able to get.
class DeviceService : IDeviceService {
readonly IObservable<IDeviceView> Source;
public DeviceService(IObservable<IDeviceView> source) { // injected parameter
/// When injected here, "source" is cold in the sense that it won't produce events until the first time it is subscribed.
/// "source" will throw an exception if its "Subscribe" method is called more than once as it is intended to have only one observer and
/// be read all the way from the beginning.
Source = source;
/// Callers of the "Subscribe" method below will expect data to be preloaded and will expect to be immediately delivered the most
/// recent event. So we need to immediately subscribe to "source" and start preloading the aggregate streams.
/// I'm assuming there is going to need to be a groupby to split the stream by device id.
var groups = source.GroupBy(x => x.DeviceId);
/// Now somehow we need to perform the aggregrate function on each grouping.
/// And create an observable that immediately delivers the most recent aggregated event when "Subscribe" is called below.
}
public IObservable<DeviceUpdateEvent> GetDeviceStream(Guid deviceId) {
/// How do we implement this? The observable that we return must be pre-loaded with the latest update
throw new NotImplementedException();
}
}

You have some weird code in that gist. Here's what I got working:
public class DeviceService : IDeviceService, IDisposable
{
readonly IObservable<IDeviceView> Source;
private readonly Dictionary<Guid, IObservable<DeviceUpdateEvent>> _updateStreams = new Dictionary<Guid, IObservable<DeviceUpdateEvent>>();
private readonly IObservable<(Guid, IObservable<DeviceUpdateEvent>)> _groupedStream;
private readonly CompositeDisposable _disposable = new CompositeDisposable();
public DeviceService(IObservable<IDeviceView> source)
{
Source = source;
_groupedStream = source
.GroupBy(v => v.DeviceId)
.Select(o => (o.Key, o
.Scan(new DeviceUpdateEvent { View = DeviceTotalView.GetInitialView(o.Key), LastUpdate = null }, (lastTotalView, newView) => lastTotalView.Update(newView))
.Replay(1)
.RefCount()
));
var groupSubscription = _groupedStream.Subscribe(t =>
{
_updateStreams[t.Item1] = t.Item2;
_disposable.Add(t.Item2.Subscribe());
});
_disposable.Add(groupSubscription);
}
public void Dispose()
{
_disposable.Dispose();
}
public IObservable<DeviceUpdateEvent> GetDeviceStream(Guid deviceId)
{
/// How do we implement this? The observable that we return must be pre-loaded with the latest update
if(this._updateStreams.ContainsKey(deviceId))
return this._updateStreams[deviceId];
return _groupedStream
.Where(t => t.Item1 == deviceId)
.Select(t => t.Item2)
.Switch();
}
}
The meat here is the _groupedStream piece. You group by DeviceId, as you said, then you use Scan to update state. I also moved Update to a static class and made it an extension method. You'll need an initial state, so I modified your DeviceTotalView class to get that. Modify accordingly:
public class DeviceTotalView : IDeviceView
{
public Guid DeviceId { get; set; }
public int Voltage { get; set; }
public int Currents { get; set; }
public object Clone() => this.MemberwiseClone();
public static DeviceTotalView GetInitialView(Guid deviceId)
{
return new DeviceTotalView
{
DeviceId = deviceId,
Voltage = 0,
Currents = 0
};
}
}
Next, the .Replay(1).Refcount() serves to remember the most recent update then provide that on subscription. We then stuff all of these child observables into a dictionary for easy retrieval on the method call. The dummy subscriptions (_disposable.Add(t.Item2.Subscribe())) are necessary for Replay to work.
In the event that there's an early request for a DeviceId that doesn't yet have an update, we subscribe to the _groupedStream which will wait for the first update, producing that Id's observable, then .Switch subscribes to that child observable.
However, all of this failed against your test code, I'm guessing because of the ConnectableObservableForAsyncProducerConsumerQueue class. I didn't want to debug that, because I wouldn't recommend doing something like that. In general it's not recommended to mix TPL and Rx code. They problems they solve largely overlap and they get in each other's way. So I modified your test code replacing that connectable observable queue thing with a Replay subject.
I also added the test-case for an early request (before an updates for that Device have arrived):
DeviceUpdateEvent deviceView1 = null;
DeviceUpdateEvent deviceView2 = null;
DeviceUpdateEvent deviceView3 = null;
var subject = new ReplaySubject<IDeviceView>();
var id1 = Guid.NewGuid();
var id2 = Guid.NewGuid();
var id3 = Guid.NewGuid();
subject.OnNext(new DeviceVoltagesUpdateView { DeviceId = id1, Voltage = 1 });
subject.OnNext(new DeviceVoltagesUpdateView { DeviceId = id1, Voltage = 2 });
subject.OnNext(new DeviceVoltagesUpdateView { DeviceId = id2, Voltage = 100 });
var service = new DeviceService(subject);
service.GetDeviceStream(id1).Subscribe(x => deviceView1 = x);
service.GetDeviceStream(id2).Subscribe(x => deviceView2 = x);
service.GetDeviceStream(id3).Subscribe(x => deviceView3 = x);
/// I believe there is no need to pause here because the Subscribe method calls above
/// block until the events have all been pushed into the subscribers above.
Assert.AreEqual(deviceView1.View.DeviceId, id1);
Assert.AreEqual(deviceView2.View.DeviceId, id2);
Assert.AreEqual(deviceView1.View.Voltage, 2);
Assert.AreEqual(deviceView2.View.Voltage, 100);
Assert.IsNull(deviceView3);
subject.OnNext(new DeviceVoltagesUpdateView { DeviceId = id2, Voltage = 101 });
Assert.AreEqual(deviceView2.View.Voltage, 101);
subject.OnNext(new DeviceVoltagesUpdateView { DeviceId = id3, Voltage = 101 });
Assert.AreEqual(deviceView3.View.DeviceId, id3);
Assert.AreEqual(deviceView3.View.Voltage, 101);
That passes fine and can be run without async.
Also, as a general tip, I would recommend doing unit tests for Rx code with the Microsoft.Reactive.Testing package, rather than time-gapping things.

A huge thanks to #Shlomo for the answer above.
The implementation given in the accepted answer, whilst a magical education for me, had a couple of issues that also needed to be solved in turn. The first was a threadrace problem, and the second was performance when a large number of devices were in the system. I ended up solving the threadrace AND dramatically improving performance with this modified implementation:
In the constructor, the grouped and scanned device stream is subscribed directly to a BehaviorSubject, which implements the Replay(1).RefCount() functionality required to immediately notify new subscribers of the latest value in the stream.
In the GetDeviceStream method, we continue to use a dictionary lookup to find the device stream, creating a preloaded BehaviorSubject if it doesn't already exist in the dictionary. We have removed the Where search that existed in the previous implementation in the question above. Using the where search caused a threadrace problem that was solved by making the grouped stream replayable. That caused an expontial performance issue. Replacing it with FirstOrDefault reduced the time take by half, and then removing it completely in favor of the GetCreate dictionary technique gave perfect perfomance O(1) instead of O(n2).
GetCreateSubject uses the Lazy proxy object as the dictionary value because the ConcurrentDictionary can sometimes call the Create method more than once for a single key. Supplying a Lazy to the dictionary ensures that the Value property is only called on one of the lazies, and therefore only one BehaviorSubject is created per device.
class DeviceService : IDeviceService, IDisposable {
readonly CompositeDisposable _disposable = new CompositeDisposable();
readonly ConcurrentDictionary<Guid, Lazy<BehaviorSubject<DeviceUpdateEvent>>> _streams = new ConcurrentDictionary<Guid, Lazy<BehaviorSubject<DeviceUpdateEvent>>>();
BehaviorSubject<DeviceUpdateEvent> GetCreateSubject(Guid deviceId) {
return _streams.GetOrAdd(deviceId, Create).Value;
Lazy<BehaviorSubject<DeviceUpdateEvent>> Create(Guid id) {
return new Lazy<BehaviorSubject<DeviceUpdateEvent>>(() => {
var subject = new BehaviorSubject<DeviceUpdateEvent>(DeviceUpdateEvent.GetInitialView(deviceId));
_disposable.Add(subject);
return subject;
});
}
}
public DeviceService(IConnectableObservable<IDeviceView> source) {
_disposable.Add(source
.GroupBy(x => x.DeviceId)
.Subscribe(deviceStream => {
_disposable.Add(deviceStream
.Scan(DeviceUpdateEvent.GetInitialView(deviceStream.Key), DeviceUtils.Update)
.Subscribe(GetCreateSubject(deviceStream.Key)));
}));
_disposable.Add(source.Connect());
}
public void Dispose() {
_disposable.Dispose();
}
public IObservable<DeviceUpdateEvent> GetDeviceStream(Guid deviceId) {
return GetCreateSubject(deviceId).AsObservable();
}
}
[TestMethod]
public async Task Test2() {
var input = new AsyncProducerConsumerQueue<IDeviceView>();
var source = new ConnectableObservableForAsyncProducerConsumerQueue<IDeviceView>(input);
var service = new DeviceService(source);
var ids = Enumerable.Range(0, 100000).Select(i => Guid.NewGuid()).ToArray();
var idsRemaining = ids.ToHashSet();
var t1 = Task.Run(async () => {
foreach (var id in ids) {
await input.EnqueueAsync(new DeviceVoltagesUpdateView { DeviceId = id, Voltage = 1 });
}
});
var t2 = Task.Run(() => {
foreach (var id in ids) {
service.GetDeviceStream(id).Subscribe(x => idsRemaining.Remove(x.View.DeviceId));
}
});
await Task.WhenAll(t1, t2);
var sw = Stopwatch.StartNew();
while (idsRemaining.Count > 0) {
if (sw.Elapsed.TotalSeconds > 600) throw new Exception("Failed");
await Task.Delay(100);
}
}
See entire problem source code and test code here: https://github.com/bboyle1234/ReactiveTest

Writing a generic FluentValidation custom validator to check unique constraint

Really new to C#, ASP.NET MVC and FluentValidation.
i have a user model like:
public class UserDetails{
public int ID { get; set; }
public string UserName { get; set; }
public string Email { get; set; }
}
for now, i've been validating the UserName and Email using FluentValidation, something like:
public AdminDetailsValidator(){
RuleFor(ad => ad.UserName).NotNull().Must(UniqueUserName(UserName)).WithMessage("UserName not Available");
RuleFor(ad => ad.Email).NotNull().Must(UniqueEmail(Email)).WithMessage("This Email id has already been registered"); ;
}
public bool UniqueUserName(string un)
{
if (UserDbContext.userDetails.SingleOrDefault(p => p.UserName == un) == null)
{
return true;
}
else
{
return false;
}
}
public bool UniqueEmail(string em)
{
if (UserDbContext.userDetails.SingleOrDefault(p => p.Email == em) == null)
{
return true;
}
else
{
return false;
}
}
But i'd rather want a more generic UniqueValidator, that i can use with multiple classes and properties. Or Atleast, i don't have to make a separate function for each property. So i looked into the custom validators. But i have no idea, how i can use that feature for my needs.
I want to do something like this:
RuleFor(ad => ad.Email).NotNull().SetValidator(new UniquePropertyValidator<UserDbContext>(userDetails.Email).WithMessage("This Email id has already been registered");
Is that even possible to do that? I want to pass the DbContext as type parameter and property as an argument(or some variation of it, whichever works). and the method can check the property against the table and return if it's unique or not.

Have you looked into using lambdas and generics? I haven't used FluentValidation so this might not be the correct method for a validator.
var dbContext = new UserDbContext();
RuleFor(ud => ud.Email)
.NotNull()
.SetValidator(
new UniquePropertyValidator<UserDetails>
(ud, ud => ud.Email, () => dbcontext.userDetails)
.WithMessage("This Email id has already been registered");
public class UniquePropertyValidator<T> {
public UniquePropertyValidator(T entity, Func<T,string> propertyAccessorFunc, Func<IEnumerable<T>> collectionAccessorFunc) {
_entity = entity;
_propertyAccessorFunc = propertyAccessorFunc;
_collectionAccessorFunc =collectionAccessorFunc;
}
public bool Validate(){
//Get all the entities by executing the lambda
var entities = _collectionAccessorFunc();
//Get the value of the entity that we are validating by executing the lambda
var propertyValue = _propertyAccessorFunc(_entity);
//Find the matching entity by executing the propertyAccessorFunc against the
//entities in the collection and comparing that with the result of the entity
//that is being validated. Warning SingleOrDefault will throw an exception if
//multiple items match the supplied predicate
//http://msdn.microsoft.com/en-us/library/vstudio/bb342451%28v=vs.100%29.aspx
var matchingEntity = entities.SingleOrDefault(e => _propertyAccessorFunc(e) == propertyValue);
return matchingEntity == null;
}
}

I have also been trying to find an elegant solution for this validator, but the solution provided so far seems to fetch all the data and then check for uniqueness. That is not very good in my opinion.
When trying to use the implementation proposed below, I get an error that LINQ to Entities does not support Invoke (i.e. executing a Func<> inside the Where clause). Is there any workaround?
public class UniqueFieldValidator<TObject, TViewModel, TProperty> : PropertyValidator where TObject : Entity where TViewModel : Entity
{
private readonly IDataService<TObject> _dataService;
private readonly Func<TObject, TProperty> _property;
public UniqueFieldValidator(IDataService<TObject> dataService, Func<TObject, TProperty> property)
: base("La propiedad {PropertyName} tiene que ser unica.")
{
_dataService = dataService;
_property = property;
}
protected override bool IsValid(PropertyValidatorContext context)
{
var model = context.Instance as TViewModel;
var value = (TProperty)context.PropertyValue;
if (model != null && _dataService.Where(t => t.Id != model.Id && Equals(_property(t), value)).Any())
{
return false;
}
return true;
}
}
public class ArticuloViewModelValidator : AbstractValidator<ArticuloViewModel>
{
public ArticuloViewModelValidator(IDataService<Articulo> articuloDataService)
{
RuleFor(a => a.Codigo).SetValidator(new UniqueFieldValidator<Articulo, ArticuloViewModel, int>(articuloDataService, a => a.Codigo));
}
}

We can solve this problem simply by working with LINQ to Entities.
Here is a static method used to determine whether the given value is unique in the specified DbSet:
static class ValidationHelpers
{
/// <summary>
/// Determines whether the specified <paramref name="newValue"/> is unique inside of
/// the given <paramref name="dbSet"/>.
/// </summary>
/// <param name="dbSet"></param>
/// <param name="getColumnSelector">
/// Determines the column, with which we will compare <paramref name="newValue"/>
/// </param>
/// <param name="newValue">
/// Value, that will be checked for uniqueness
/// </param>
/// <param name="cancellationToken"></param>
/// <typeparam name="TEntity"></typeparam>
/// <typeparam name="TColumn"></typeparam>
/// <returns></returns>
public static async Task<bool> IsColumnUniqueInsideOfDbSetAsync<TEntity, TColumn>(DbSet<TEntity> dbSet,
Expression<Func<TEntity, TColumn>> getColumnSelector,
TColumn newValue,
CancellationToken cancellationToken)
where TEntity : class
{
return !await dbSet
.Select(getColumnSelector)
.AnyAsync(column => column.Equals(newValue), cancellationToken);
}
}
Example of using
We have the following entity:
public class Category
{
// ...
public string Title { get; set; }
// ...
}
And a DbContext class:
public interface ApplicationDbContext
{
// ...
public DbSet<Category> Category { get; set; }
// ...
}
Let's say a user wants to create a new category. We want to validate the title of this category for uniqueness:
RuleFor(c => c.Title)
.MustAsync
(
(newTitle, token) => ValidationHelpers.IsColumnUniqueInsideOfDbSetAsync
(_context.Category, c => c.Title, newTitle, token)
)
.WithMessage("{PropertyName} must be unique");
Note: _context is an object of type ApplicationDbContext.