How to group and Throttle Object by ID Rx - c#

I have incoming objects of the same type, but if An Object property IsThrottlable is set to false regardless of the ID I DON'T want to throttle it but if IsThrottlable is set to true I would like to throttle the object every 3 seconds by ID. So if an object with the same ID comes in 50 times with 3 seconds I would like to send the HTTPSend for the last Object.
namespace BoatThrottle
{
class MData
{
public int ID { get; set; }
public bool IsThrottlable { get; set; }
public string Description { get; set; }
}
class Program
{
static void Main(string[] args)
{
Random rand = new Random();
while (true)
{
var data = GenerateRandomObj(rand);
SendData(data);
Task.Delay(rand.Next(100, 2000));
}
}
static MData GenerateRandomObj(Random rand)
{
return new MData() { ID = rand.Next(1, 20), Description = "Notification....", IsThrottlable = (rand.Next(2) == 1) };
}
static void SendData(MData mData)
{
if (mData.IsThrottlable)
{
_doValues.OnNext(mData);
var dd = ThrottledById(DoValues);
var observable =
dd
.Throttle(TimeSpan.FromMilliseconds(3000.0))
.ObserveOn(Scheduler.ThreadPool.DisableOptimizations());
_subscription =
observable
.ObserveOn(Scheduler.ThreadPool.DisableOptimizations())
.Subscribe(y =>
{
HTTPSend(y);
});
}
else
{
// MData object coming in IsThrottlable set to false always send this data NO throttling
HTTPSend(mData);
}
}
private static IDisposable? _subscription = null;
public static IObservable<MData> ThrottledById(IObservable<MData> observable)
{
return observable.Buffer(TimeSpan.FromSeconds(3))
.SelectMany(x =>
x.GroupBy(y => y.ID)
.Select(y => y.Last()));
}
private static readonly Subject<MData> _doValues = new Subject<MData>();
public static IObservable<MData> DoValues { get { return _doValues; } }
static void HTTPSend(MData mData)
{
Console.WriteLine("===============HTTP===>> " + mData.ID + " " + mData.Description + " " + mData.IsThrottlable);
}
}
}
EDIT:
e.g ALL received within 3 seconds
MData ID = 1, IsThrottlable = False, Description = "Notify"
MData ID = 2, IsThrottlable = True, Description = "Notify1"
MData ID = 2, IsThrottlable = True, Description = "Notify2"
MData ID = 9, IsThrottlable = False, Description = "Notify2"
MData ID = 2, IsThrottlable = True, Description = "Notify3"
MData ID = 2, IsThrottlable = True, Description = "Notify4"
MData ID = 3, IsThrottlable = True, Description = "Notify"
MData ID = 4, IsThrottlable = True, Description = "Notify"
MData ID = 5, IsThrottlable = True, Description = "Notify1"
MData ID = 5, IsThrottlable = True, Description = "Notify2"
MData ID = 8, IsThrottlable = True, Description = "Notify1"
MData ID = 8, IsThrottlable = True, Description = "Notify2"
MData ID = 8, IsThrottlable = True, Description = "Notify3"
MData ID = 8, IsThrottlable = True, Description = "Notify4"
MData ID = 8, IsThrottlable = True, Description = "Notify5"
MData ID = 8, IsThrottlable = True, Description = "Notify6"
Expected at the First 3 seconds:
MData ID = 1, IsThrottlable = False, Description = "Notify"
MData ID = 9, IsThrottlable = False, Description = "Notify2"
MData ID = 2, IsThrottlable = True, Description = "Notify4"
MData ID = 3, IsThrottlable = True, Description = "Notify"
MData ID = 4, IsThrottlable = True, Description = "Notify"
MData ID = 5, IsThrottlable = True, Description = "Notify2"
MData ID = 8, IsThrottlable = True, Description = "Notify6"

I decided to take your final implementation, as posted in your question, but it should be as an answer, and clean up the query for you in a way that is the most idiomatic Rx kind of way.
Here's my version of your code:
public MainWindow()
{
InitializeComponent();
Debug.Print("========================");
_subscription =
Observable
.Generate(0, x => true, x => x + 1,
x => new MData() { ID = Random.Shared.Next(1, 3), Description = "Notification....", IsThrottlable = Random.Shared.Next(2) == 1 },
x => TimeSpan.FromMilliseconds(Random.Shared.Next(100, 2000)))
.GroupBy(m => m.IsThrottlable)
.SelectMany(g =>
g.Key
? g.GroupBy(x => x.ID).SelectMany(g2 => g2.Throttle(TimeSpan.FromSeconds(3.0)))
: g)
.SelectMany(m => Observable.Start(() => HTTPSend(m)))
.Subscribe();
}
The final .SelectMany(m => Observable.Start(() => HTTPSend(m))) might need to be written as .Select(m => Observable.Start(() => HTTPSend(m))).Merge(1).

One way to do it is to group the sequence by the IsThrottlable property. This way you'll get a nested sequence that contains two subsequences, one containing the throttleable elements and one containing the non-throttleable elements. You can then transform each of the two subsequences accordingly, and finally use the SelectMany operator to flatten the nested sequence back to a flat sequence that contains the elements emitted by the two transformed subsequences.
The subsequence that contains the non-throttleable elements needs no transformation, so you can return it as is.
The subsequence that contains the throttleable elements needs to be grouped further by the ID property, producing even thinner subsequences that contain only throttleable elements having the same id. These are the sequences that need to be throttled:
IObservable<MData> throttled = source
.GroupBy(x => x.IsThrottlable)
.SelectMany(g1 =>
{
if (!g1.Key) return g1; // Not throttleable, return it as is.
return g1
.GroupBy(x => x.ID)
.SelectMany(g2 => g2.Throttle(TimeSpan.FromSeconds(3)));
});
At the end you'll get a flat sequence that contains both the throttleable and the non-throttleable items, with the throttleable items already throttled by id.
The SelectMany operator is essentially a combination of the Select+Merge operators.

Related

How to group the Similar value array in c#

class Program
{
static void Main(string[] args)
{
int[] arr1 = { 1, 2, 3, 4 };
int[] arr2 = { 1, 2, 3 };
int[] arr3 = { 1, 2, 3 };
int[] arr4 = { 1, 2, 3, 4 };
int[] arr5 = { 1, 2, 3, 4 };
int[] arr6 = { 1, 2, 3, 4, 5 };
Order Order1 = new Order { OrderId = 1, Deatils = arr1 };
Order Order2 = new Order { OrderId = 2, Deatils = arr2 };
Order Order3 = new Order { OrderId = 3, Deatils = arr3 };
Order Order4 = new Order { OrderId = 4, Deatils = arr4 };
Order Order5 = new Order { OrderId = 5, Deatils = arr5 };
Order Order6 = new Order { OrderId = 6, Deatils = arr6 };
// I want to Output like this based on same values in details object.
string similarOrderDetailsOrderIds_1 = "1,4,5";
string similarOrderDetailsOrderIds_2 = "2,3";
string similarOrderDetailsOrderIds_3 = "5";
}
}
public class Order
{
public int OrderId { get; set; }
public int[] Deatils { get; set; }
}
I want to output like this because OrderId-1,4,5 have the same values in Details.
string similarOrderDetailsOrderIds_1 = "1,4,5";
string similarOrderDetailsOrderIds_2 = "2,3";
string similarOrderDetailsOrderIds_3 = "5";
This extension method is a generic generalization of the solution provided by Michael Liu here which orders elements prior to hash code generation and will work for any T which itself correctly implements GetHashCode() and Equals():
static class Extensions
{
public static int GetCollectionHashCode<T>(this IEnumerable<T> e)
{
return ((IStructuralEquatable)e.OrderByDescending(x => x).ToArray())
.GetHashCode(EqualityComparer<T>.Default);
}
}
You can utilize it like so:
var orders = new[] { Order1, Order2, Order3, Order4, Order5, Order6 };
var groups = orders
.Where(x => x.Deatils != null)
.GroupBy(x => x.Deatils.GetCollectionHashCode())
.Select(x => x.Select(y => y.OrderId));
Note: Deatils typo above is intentional as per OP.
You could use LINQ as the other answer provided or create a helper method to group them for you.
public static IEnumerable<IEnumerable<T>> Aggregate<T>(IEnumerable<T> ungrouped, Func<T, T, bool> Expression)
{
// create a place to put the result
List<List<T>> groupedItems = new();
// create a mutable storage to keep track which ones we shouldn't compare
List<T> mutableBag = ungrouped.ToArray().ToList();
// n^2 isn't great but it was easy to implement, consider creating a hash from all elements and use those instead of manual comparisons
foreach (var left in ungrouped)
{
// create a place to store any similar items
List<T> similarItems = new();
// compare element to every remaining element, if they are similar add the other to the similar list
foreach (var right in mutableBag)
{
if (Expression(left, right))
{
similarItems.Add(right);
}
}
// if we didn't find any similar items continue and let GC collect similar items
if (similarItems.Count == 0)
{
continue;
}
// since we found items remove the from the mutable bag so we don't have duplicate entries
foreach (var item in similarItems)
{
mutableBag.Remove(item);
}
// add the similar items to the result
groupedItems.Add(similarItems);
}
return groupedItems;
}
public static bool Compare<T>(T[] Left, T[] Right)
{
// this compare method could be anything I would advise against doing member comparison like this, this is simplified to provide how to use the Aggregate method and what the CompareMethod might look like for comlpex types
if (Left.Length != Right.Length)
{
return false;
}
foreach (var item in Left)
{
if (Right.Contains(item) is false)
{
return false;
}
}
return true;
}
For the given example
Order Order1 = new Order { OrderId = 1, Deatils = arr1 };
Order Order2 = new Order { OrderId = 2, Deatils = arr2 };
Order Order3 = new Order { OrderId = 3, Deatils = arr3 };
Order Order4 = new Order { OrderId = 4, Deatils = arr4 };
Order Order5 = new Order { OrderId = 5, Deatils = arr5 };
Order Order6 = new Order { OrderId = 6, Deatils = arr6 };
Order[] arrs = { Order1, Order2, Order3, Order4, Order5, Order6 };
We could use this new helper method like this:
var groupedItems = Aggregate(arrs, (left, right) => Compare(left.Deatils, right.Deatils));

Linq group by date range

I have collection that I need to group if the parent key is common AND if the date field is within n (e.g. 2) hours of each other.
Sample data:
List<DummyObj> models = new List<DummyObj>()
{
new DummyObj { ParentKey = 1, ChildKey = 1, TheDate = DateTime.Parse("01/01/2020 00:00:00"), Name = "Single item - not grouped" },
new DummyObj { ParentKey = 2, ChildKey = 2, TheDate = DateTime.Parse("01/01/2020 01:00:00"), Name = "Should be grouped with line below" },
new DummyObj { ParentKey = 2, ChildKey = 3, TheDate = DateTime.Parse("01/01/2020 02:00:00"), Name = "Grouped with above" },
new DummyObj { ParentKey = 2, ChildKey = 4, TheDate = DateTime.Parse("01/01/2020 04:00:00"), Name = "Separate item as greater than 2 hours" },
new DummyObj { ParentKey = 2, ChildKey = 5, TheDate = DateTime.Parse("01/01/2020 05:00:00"), Name = "Grouped with above" },
new DummyObj { ParentKey = 3, ChildKey = 6, TheDate = DateTime.Parse("01/01/2020 05:00:00"), Name = "Single item - not grouped" }
};
private class DummyObj
{
public int ParentKey { set; get; }
public int ChildKey { set; get; }
public DateTime TheDate { set; get; }
public string Name { set; get; }
}
The resulting grouping should be (child keys):
{[1]}, {[2,3]}, {[4,5]}, {[6]}
I could group by parent key first then loop through comparing the individual items within the groups but hoping for a more elegant solution.
As always, thank you very much.
public static void Test()
{
var list = GetListFromDb(); //returns List<DummyObj>;
var sortedList = new List<DummyObj>();
foreach(var g in list.GroupBy(x => x.ParentKey))
{
if(g.Count() < 2)
{
sortedList.Add(g.First());
}
else
{
var datesInGroup = g.Select(x => x.TheDate);
var hoursDiff = (datesInGroup.Max() - datesInGroup.Min()).TotalHours;
if(hoursDiff <= 2)
{
string combinedName = string.Join("; ", g.Select(x => x.Name));
g.First().Name = combinedName;
sortedList.Add(g.First());
}
else
{
//now it's the mess
DateTime earliest = g.Select(x => x.TheDate).Min();
var subGroup = new List<DummyObj>();
foreach(var line in g)
{
if((line.TheDate - earliest).TotalHours > 2)
{
//add the current subgroup entry to the sorted group
subGroup.First().Name = string.Join("; ", subGroup.Select(x => x.Name));
sortedList.Add(subGroup.First());
//new group needed and new earliest date to start the group
sortedList = new List<DummyObj>();
sortedList.Add(line);
earliest = line.TheDate;
}
else
{
subGroup.Add(line);
}
}
//add final sub group, i.e. when there's none that are over 2 hours apart or the last sub group
if(subGroup.Count > 1)
{
subGroup.First().Name = string.Join("; ", subGroup.Select(x => x.Name));
sortedList.Add(subGroup.First());
}
else if(subGroup.Count == 1)
{
sortedList.Add(subGroup.First());
}
}
}
}
}
Here you go:
List<DummyObj> models = new List<DummyObj>()
{
new DummyObj { ParentKey = 1, ChildKey = 1, TheDate = DateTime.Parse("01/01/2020 00:00:00"), Name = "Single item - not grouped" },
new DummyObj { ParentKey = 2, ChildKey = 2, TheDate = DateTime.Parse("01/01/2020 01:00:00"), Name = "Should be grouped with line below" },
new DummyObj { ParentKey = 2, ChildKey = 3, TheDate = DateTime.Parse("01/01/2020 02:00:00"), Name = "Grouped with above" },
new DummyObj { ParentKey = 2, ChildKey = 4, TheDate = DateTime.Parse("01/01/2020 04:00:00"), Name = "Separate item as greater than 2 hours" },
new DummyObj { ParentKey = 2, ChildKey = 5, TheDate = DateTime.Parse("01/01/2020 05:00:00"), Name = "Grouped with above" },
new DummyObj { ParentKey = 3, ChildKey = 6, TheDate = DateTime.Parse("01/01/2020 05:00:00"), Name = "Single item - not grouped" }
};
List<List<DummyObj>> groups =
models
.GroupBy(x => x.ParentKey)
.Select(xs => xs.OrderBy(x => x.TheDate).ToList())
.SelectMany(xs => xs.Skip(1).Aggregate(new[] { xs.Take(1).ToList() }.ToList(), (a, x) =>
{
if (x.TheDate.Subtract(a.Last().Last().TheDate).TotalHours < 2.0)
{
a.Last().Add(x);
}
else
{
a.Add(new [] { x }.ToList());
}
return a;
}))
.ToList();
string output =
String.Join(", ",
groups.Select(x =>
$"{{[{String.Join(",", x.Select(y => $"{y.ChildKey}"))}]}}"));
That gives me:
{[1]}, {[2,3]}, {[4,5]}, {[6]}

C# sort object list with start position and loop

I have a strange question :)
I have a object list looking like this:
var list = new []
{
new { Id = 1, Name = "Marcus" },
new { Id = 2, Name = "Mattias" },
new { Id = 3, Name = "Patric" },
new { Id = 4, Name = "Theodor" },
};
I would like to sort the list providing a "start id"
For example, if I provide "start id" 3, the result should look like this:
Id
Name
3
Patric
4
Theodor
1
Marcus
2
Mattias
I have no idea where to start, so I really need some help from you coding gods
The list is from a sql table, but it does not matter for me where the sort take place (in sql query or in c# code)
Try this:
var list = new []
{
new { Id = 1, Name = "Marcus" },
new { Id = 2, Name = "Mattias" },
new { Id = 3, Name = "Patric" },
new { Id = 4, Name = "Theodor" },
};
var start_id = 3;
var max_id = list.Max(y => y.Id);
var result =
from x in list
orderby (x.Id + max_id - start_id) % max_id
select x;
I get:
With LINQ to objects you can do something like that:
var list = new []
{
new { Id = 1, Name = "Marcus" },
new { Id = 2, Name = "Mattias" },
new { Id = 3, Name = "Patric" },
new { Id = 4, Name = "Theodor" },
};
var startId = 3;
var result = list
.GroupBy(i => i.Id >= startId ? 1 : 0) // split in two groups
.OrderByDescending(g => g.Key) // sort to have the group with startId first
.Select(g => g.OrderBy(i => i.Id)) // sort each group
.SelectMany(i => i) // combine result
.ToList();
Console.WriteLine(string.Join(", ", result.Select(i => i.Id))); // prints "3, 4, 1, 2"
You require 2 criteria to apply:
Order ascending by Id.
Return the Ids greater than threshold before the Ids less than threshold.
You can try:
var offset = 3;
var sorted1 = list
.OrderBy(item => item.Id < offset)
.ThenBy(item => item.Id);
The OrderBy condition yields true if Id is less than offset and false otherwise.
true is greater than false and therefore is returned later
A dirty way could also be:
var offset = 3;
var sorted2 = list
.OrderBy(item => unchecked((uint)(item.Id - offset)));
Here the offset is subtracted from Id and the result converted to unsigned int to make the negative values become very large positive ones. A little hacky. Might not work with queries against SQL providers.
Here's a toy Non-Linq Version
object[] ShiftList(int id)
{
var list = new dynamic[]
{
new { Id = 1, Name = "Marcus" },
new { Id = 2, Name = "Mattias" },
new { Id = 3, Name = "Patric" },
new { Id = 4, Name = "Theodor" },
};
Span<dynamic> listSpan = list;
int indexFound = -1;
for (int i = 0; i < list.Length; i++)
{
if (listSpan[i].Id == id)
{
indexFound = i;
}
}
if (indexFound is -1)
{
return list;
}
var left = listSpan.Slice(0, indexFound);
var right = listSpan[indexFound..];
object[] objs = new object[list.Length];
Span<object> objSpan = objs;
right.CopyTo(objSpan);
left.CopyTo(objSpan[right.Length..]);
return objs;
}
Try using foreach and iterate over each object in your list:
foreach (var item in list)
{
}
from here you should be able to use some of the collection methods for a list to reorder your list.

How to merge a single list using C# lambda

I have a class like the following:
public class MyData
{
public int Key { get; set; }
public string MyString { get; set; }
public bool MyFlag { get; set; }
}
I have a list of these classes:
var data = new List<MyData>();
Which is populated as follows:
data.Add(new MyData() { Key = 1, MyFlag = true, MyString = "Hello" });
data.Add(new MyData() { Key = 1, MyFlag = false, MyString = "Goodbye" });
data.Add(new MyData() { Key = 2, MyFlag = true, MyString = "Test" });
data.Add(new MyData() { Key = 2, MyFlag = false, MyString = "Merge" });
data.Add(new MyData() { Key = 3, MyFlag = false, MyString = "Data" });
What I want is a list as follows:
Key true false
1 Hello Goodbye
2 Test Merge
3 Data
I need an anonymous type that reflects the three values above. I found a few posts and articles that seemed to suggest GroupJoin, but I'm unsure how I could use that in this case as it seems to allow joining two separate lists.
I suggest grouping (by Key property). If you want true and false properties we have to put it as #true and #false since true and false are keywords:
var result = data
.GroupBy(item => item.Key)
.Select(chunk => new {
Key = chunk.Key,
#true = chunk.FirstOrDefault(item => item.MyFlag)?.MyString,
#false = chunk.FirstOrDefault(item => !item.MyFlag)?.MyString,
});
Please note that I am not in front of a PC right now (so I could not test it), but this should give you an idea at the very least.
data
.GroupBy(x => x.Key)
.Select(
x => new {
Key = x.Key,
True = data.SingleOrDefault(y1 => y1.Key == x.Key && y1.MyFlag)?.MyString,
False = data.SingleOrDefault(y2 => y2.Key == x.Key && !y2.MyFlag)?.MyString
});

Linq query to group by field1, count field2 and filter by count between values of joined collection

I'm having trouble with getting a my linq query correct. I've been resisting doing this with foreach loops because I'm trying to better understand linq.
I have following data in LinqPad.
void Main()
{
var events = new[] {
new {ID = 1, EventLevel = 1, PatientID = "1", CodeID = "2", Occurences = 0 },
new {ID = 2, EventLevel = 2, PatientID = "1", CodeID = "2", Occurences = 0 },
new {ID = 3, EventLevel = 1, PatientID = "2", CodeID = "1", Occurences = 0 },
new {ID = 4, EventLevel = 3, PatientID = "2", CodeID = "2", Occurences = 0 },
new {ID = 5, EventLevel = 1, PatientID = "3", CodeID = "3", Occurences = 0 },
new {ID = 6, EventLevel = 3, PatientID = "1", CodeID = "4", Occurences = 0 }
};
var filter = new FilterCriterion();
var searches = new List<FilterCriterion.Occurence>();
searches.Add(new FilterCriterion.Occurence() { CodeID = "1", MinOccurences = 2, MaxOccurences = 3 });
searches.Add(new FilterCriterion.Occurence() { CodeID = "2", MinOccurences = 2, MaxOccurences = 3 });
filter.Searches = searches;
var summary = from e in events
let de = new
{
PatientID = e.PatientID,
CodeID = e.CodeID
}
group e by de into t
select new
{
PatientID = t.Key.PatientID,
CodeID = t.Key.CodeID,
Occurences = t.Count(d => t.Key.CodeID == d.CodeID)
};
var allCodes = filter.Searches.Select(i => i.CodeID);
summary = summary.Where(e => allCodes.Contains(e.CodeID));
// How do I find the original ID property from the "events" collection and how do I
// eliminate the instances where the Occurences is not between MinOccurences and MaxOccurences.
foreach (var item in summary)
Console.WriteLine(item);
}
public class FilterCriterion
{
public IEnumerable<Occurence> Searches { get; set; }
public class Occurence
{
public string CodeID { get; set; }
public int? MinOccurences { get; set; }
public int? MaxOccurences { get; set; }
}
}
The problem I have is that need to filter the results by the MinOccurences and MaxOccurences filter property and in the end I want the "events" objects where the IDs are 1,2,3 and 4.
Thanks in advance if you can provide help.
To access event.ID at the end of processing you need to pass it with your first query. Alter select to this:
// ...
group e by de into t
select new
{
PatientID = t.Key.PatientID,
CodeID = t.Key.CodeID,
Occurences = t.Count(d => t.Key.CodeID == d.CodeID),
// taking original items with us
Items = t
};
Having done that, your final query (including occurrences filter) might look like this:
var result = summary
// get all necessary data, including filter that matched given item
.Select(Item => new
{
Item,
Filter = searches.FirstOrDefault(f => f.CodeID == Item.CodeID)
})
// get rid of those without matching filter
.Where(i => i.Filter != null)
// this is your occurrences filtering
.Where(i => i.Item.Occurences >= i.Filter.MinOccurences
&& i.Item.Occurences <= i.Filter.MaxOccurences)
// and finally extract original events IDs
.SelectMany(i => i.Item.Items)
.Select(i => i.ID);
This produces 1, 2 as result. 3 and 4 are left out as they don't get past occurrences filtering.
I have run your program in linqpad.
My understanding is that you want to filter using filter.MinOccurences and filter.MaxOccurences on Occurences count of result data set.
You can add additional filters using Where clause.
if (filter.MinOccurences.HasValue)
summary = summary.Where (x=> x.Occurences >= filter.MinOccurences);
if (filter.MaxOccurences.HasValue)
summary = summary.Where (x=> x.Occurences <= filter.MaxOccurences);

Categories

Resources