I am trying to get the 20 latest values of an observable and exposing it as a property without blocking occurring. At the moment, my code looks like:
class Foo
{
private IObservable<int> observable;
public Foo(IObservable<int> bar)
{
this.observable = bar;
}
public IEnumerable<int> MostRecentBars
{
get
{
return this.observable.TakeLast(20).ToEnumerable();
}
}
}
However, when the MostRecentBars getter is called, this is blocking, presumably because ToEnumerable will not return until there are at least 20 observed values.
Is there a built-in way to expose up to a maximum of 20 most recent values of the observable without blocking? If there are less than 20 observed values then it should just return all of them.
I'll give you two choices. One uses the Rx Scan operator, but I think that one makes it a little more complicated to read. The other uses a standard Queue with locking. You can choose.
(1)
class Foo
{
private int[] bars = new int[] { };
public Foo(IObservable<int> bar)
{
bar
.Scan<int, int[]>(
new int[] { },
(ns, n) =>
ns
.Concat(new [] { n, })
.TakeLast(20)
.ToArray())
.Subscribe(ns => bars = ns);
}
public IEnumerable<int> MostRecentBars
{
get
{
return bars;
}
}
}
(2)
class Foo
{
private Queue<int> queue = new Queue<int>();
public Foo(IObservable<int> bar)
{
bar.Subscribe(n =>
{
lock (queue)
{
queue.Enqueue(n);
if (queue.Count > 20)
{
queue.Dequeue();
}
}
});
}
public IEnumerable<int> MostRecentBars
{
get
{
lock (queue)
{
return queue.ToArray();
}
}
}
}
I hope these help.
I can't think of a built-in Rx operator(s) that fits your requirements. You could implement it this way:
class Foo
{
private IObservable<int> observable;
private Queue<int> buffer = new Queue<int>();
public Foo(IObservable<int> bar)
{
this.observable = bar;
this.observable
.Subscribe(item =>
{
lock (buffer)
{
if (buffer.Count == 20) buffer.Dequeue();
buffer.Enqueue(item);
}
});
}
public IEnumerable<int> MostRecentBars
{
get
{
lock (buffer)
{
return buffer.ToList(); // Create a copy.
}
}
}
}
Although you have already got your answer, I was thinking of solving this using Replay Subject with buffer and came up with something like:
class Foo
{
private ReplaySubject<int> replay = new ReplaySubject<int>(20);
public Foo(IObservable<int> bar)
{
bar.Subscribe(replay);
}
public IEnumerable<int> MostRecentBars
{
get
{
var result = new List<int>();
replay.Subscribe(result.Add); //Replay fill in the list with buffered items on same thread
return result;
}
}
}
Let me know if this fits into your problem.
I have a few extensions I tend to attach to any project I build with the reactive extensions, one of them is a sliding window:
public static IObservable<IEnumerable<T>> SlidingWindow<T>(this IObservable<T> o, int length)
{
Queue<T> window = new Queue<T>();
return o.Scan<T, IEnumerable<T>>(new T[0], (a, b) =>
{
window.Enqueue(b);
if (window.Count > length)
window.Dequeue();
return window.ToArray();
});
}
This returns an array of the most recent N items (or less, if there have not been N items yet).
For your case, you should be able to do:
class Foo
{
private IObservable<int> observable;
private int[] latestWindow = new int[0];
IDisposable slidingWindowSubscription;
public Foo(IObservable<int> bar)
{
this.observable = bar;
slidingWindowSubscription = this.observable.SlidingWindow(20).Subscribe(a =>
{
latestWindow = a;
});
}
public IEnumerable<int> MostRecentBars
{
get
{
return latestWindow;
}
}
}
Seriously, doesn't this all depend on how long you willing to wait? How do you know the observable has completed when you have 19 items in hand? One second later you are sure it's done? ten years later? You are sure it's done? How do you know? It's Observable, so you have to keep observing it until your operators or whatever monadic transforms you apply do some useful transform with the incoming stream.
I would think the (possibly new addition) Window or Buffer with the TimeSpan overload would work. Especially Window, it releases an Observable of Observable so once the outer Observable is created, you can actually listen for the First of 20 items, and then you need to listen carefully for the OnCompleted, or you lose the whole point of the Window operator, but you get the idea.
Related
I need to make a class that stores a List of int and read/write from it asynchronously.
here's my class:
public class ConcurrentIntegerList
{
private readonly object theLock = new object();
List<int> theNumbers = new List<int>();
public void AddNumber(int num)
{
lock (theLock)
{
theNumbers.Add(num);
}
}
public List<int> GetNumbers()
{
lock (theLock)
{
return theNumbers;
}
}
}
But it is not thread-safe until here. when I do multiple operations from different threads I get this error:
Collection was modified; enumeration operation may not execute.
What I missed here?
public List<int> GetNumbers()
{
lock (theLock)
{
// return theNumbers;
return theNumbers.ToList();
}
}
But the performance won't be very good this way, and GetNumbers() now returns a snapshot copy.
I've got a public static List<MyDoggie> DoggieList;
DoggieList is appended to and written to by multiple processes throughout my application.
We run into this exception pretty frequently:
Collection was modified; enumeration operation may not execute
Assuming there are multiple classes writing to DoggieList how do we get around this exception?
Please note that this design is not great, but at this point we need to quickly fix it in production.
How can we perform mutations to this list safely from multiple threads?
I understand we can do something like:
lock(lockObject)
{
DoggieList.AddRange(...)
}
But can we do this from multiple classes against the same DoggieList?
you can also create you own class and encapsulate locking thing in that only, you can try like as below ,
you can add method you want like addRange, Remove etc.
class MyList {
private object objLock = new object();
private List<int> list = new List<int>();
public void Add(int value) {
lock (objLock) {
list.Add(value);
}
}
public int Get(int index) {
int val = -1;
lock (objLock) {
val = list[0];
}
return val;
}
public void GetAll() {
List<int> retList = new List<int>();
lock (objLock) {
retList = new List<T>(list);
}
return retList;
}
}
Good stuff : Concurrent Collections very much in detail :http://www.albahari.com/threading/part5.aspx#_Concurrent_Collections
making use of concurrent collection ConcurrentBag Class can also resolve issue related to multiple thread update
Example
using System.Collections.Concurrent;
using System.Threading.Tasks;
public static class Program
{
public static void Main()
{
var items = new[] { "item1", "item2", "item3" };
var bag = new ConcurrentBag<string>();
Parallel.ForEach(items, bag.Add);
}
}
Using lock a the disadvantage of preventing concurrent readings.
An efficient solution which does not require changing the collection type is to use a ReaderWriterLockSlim
private static readonly ReaderWriterLockSlim _lock = new ReaderWriterLockSlim();
With the following extension methods:
public static class ReaderWriterLockSlimExtensions
{
public static void ExecuteWrite(this ReaderWriterLockSlim aLock, Action action)
{
aLock.EnterWriteLock();
try
{
action();
}
finally
{
aLock.ExitWriteLock();
}
}
public static void ExecuteRead(this ReaderWriterLockSlim aLock, Action action)
{
aLock.EnterReadLock();
try
{
action();
}
finally
{
aLock.ExitReadLock();
}
}
}
which can be used the following way:
_lock.ExecuteWrite(() => DoggieList.Add(new Doggie()));
_lock.ExecuteRead(() =>
{
// safe iteration
foreach (MyDoggie item in DoggieList)
{
....
}
})
And finally if you want to build your own collection based on this:
public class SafeList<T>
{
private readonly ReaderWriterLockSlim _lock = new ReaderWriterLockSlim();
private readonly List<T> _list = new List<T>();
public T this[int index]
{
get
{
T result = default(T);
_lock.ExecuteRead(() => result = _list[index]);
return result;
}
}
public List<T> GetAll()
{
List<T> result = null;
_lock.ExecuteRead(() => result = _list.ToList());
return result;
}
public void ForEach(Action<T> action) =>
_lock.ExecuteRead(() => _list.ForEach(action));
public void Add(T item) => _lock.ExecuteWrite(() => _list.Add(item));
public void AddRange(IEnumerable<T> items) =>
_lock.ExecuteWrite(() => _list.AddRange(items));
}
This list is totally safe, multiple threads can add or get items in parallel without any concurrency issue. Additionally, multiple threads can get items in parallel without locking each other, it's only when writing than 1 single thread can work on the collection.
Note that this collection does not implement IEnumerable<T> because you could get an enumerator and forget to dispose it which would leave the list locked in read mode.
make DoggieList of type ConcurrentStack and then use pushRange method. It is thread safe.
using System.Collections.Concurrent;
var doggieList = new ConcurrentStack<MyDoggie>();
doggieList.PushRange(YourCode)
I have a stream with live data, and a stream which basically delimits parts of the live data that belong together. Now when someone subscribes to the live data stream, I would like to replay them the live data. However I don't want to remember all the live data, only the part since the last time the other stream emitted a value.
There is an issue which would solve my problem, since there is a replay operator which does exactly what I want (or at least I think).
What is currently the way to do this easily? Is there a better way than something like the following?
private class ReplayWithLimitObservable<TItem, TDelimiter> : IConnectableObservable<TItem>
{
private readonly List<TItem> cached = new List<TItem>();
private readonly IObservable<TDelimiter> delimitersObservable;
private readonly IObservable<TItem> itemsObservable;
public ReplayWithLimitObservable(IObservable<TItem> itemsObservable, IObservable<TDelimiter> delimitersObservable)
{
this.itemsObservable = itemsObservable;
this.delimitersObservable = delimitersObservable;
}
public IDisposable Subscribe(IObserver<TItem> observer)
{
lock (cached)
{
cached.ForEach(observer.OnNext);
}
return itemsObservable.Subscribe(observer);
}
public IDisposable Connect()
{
var delimiters = delimitersObservable.Subscribe(
p =>
{
lock (cached)
{
cached.Clear();
}
});
var items = itemsObservable.Subscribe(
p =>
{
lock (cached)
{
cached.Add(p);
}
});
return Disposable.Create(
() =>
{
items.Dispose();
delimiters.Dispose();
lock (cached)
{
cached.Clear();
}
});
}
public static IConnectableObservable<TItem> ReplayWithLimit<TItem, TDelimiter>(IObservable<TItem> items, IObservable<TDelimiter> delimiters)
{
return new ReplayWithLimitObservable<TItem, TDelimiter>(items, delimiters);
}
Does this do what you want? It has the advantage of leaving all of the locking and race conditions to the Rx pros :)
private class ReplayWithLimitObservable<T, TDelimiter> : IConnectableObservable<T>
{
private IConnectableObservable<IObservable<T>> _source;
public ReplayWithLimitObservable(IObservable<T> source, IObservable<TDelimiter> delimiter)
{
_source = source
.Window(delimiter) // new replay window on delimiter
.Select<IObservable<T>,IObservable<T>>(window =>
{
var replayWindow = window.Replay();
// immediately connect and start memorizing values
replayWindow.Connect();
return replayWindow;
})
.Replay(1); // remember the latest window
}
IDisposable Connect()
{
return _source.Connect();
}
IDisposable Subscribe(IObserver<T> observer)
{
return _source
.Concat()
.Subscribe(observer);
}
}
public static IConnectableObservable<TItem> ReplayWithLimit<TItem, TDelimiter>(IObservable<TItem> items, IObservable<TDelimiter> delimiters)
{
return new ReplayWithLimitObservable<TItem, TDelimiter>(items, delimiters);
}
So I have this recursive factorial function in c#. I am using it to deal with BigInteger. The problem arises when I want to deal with large integers and because my function is recursive it will cause a StackOverflow exception. Now the simple solution is to not make the function recursive. I am wondering if there is a way to get around this? I'm thinking along the lines of more ram allocated the the stack?
BigInteger Factorial(BigInteger n)
{
return n == 1 ? 1 : n * Factorial(n - 1);
}
I understand it is nice if you could express recursive functions in c# without worrying about the stack. But unfortunately that is not directly possible, and no matter how big you make the stack there will always be situations where you run out of stack space. Furthermore your performance will likely be pretty horrendous. If you have a tail recursive function like this factorial something can be done, that pretty much lets you express your function in the original recursive way, without the huge penalty.
Unfortunately c# does not directly support tail recursive calls, but workarounds are possible using a so-called "trampoline" construction.
See for example: http://bartdesmet.net/blogs/bart/archive/2009/11/08/jumping-the-trampoline-in-c-stack-friendly-recursion.aspx and http://www.thomaslevesque.com/2011/09/02/tail-recursion-in-c/
From the last blog, comes the following code that will allow you to perform the factorial as a tail recursive function without stack problems.
public static class TailRecursion
{
public static T Execute<T>(Func<RecursionResult<T>> func)
{
do
{
var recursionResult = func();
if (recursionResult.IsFinalResult)
return recursionResult.Result;
func = recursionResult.NextStep;
} while (true);
}
public static RecursionResult<T> Return<T>(T result)
{
return new RecursionResult<T>(true, result, null);
}
public static RecursionResult<T> Next<T>(Func<RecursionResult<T>> nextStep)
{
return new RecursionResult<T>(false, default(T), nextStep);
}
}
public class RecursionResult<T>
{
private readonly bool _isFinalResult;
private readonly T _result;
private readonly Func<RecursionResult<T>> _nextStep;
internal RecursionResult(bool isFinalResult, T result, Func<RecursionResult<T>> nextStep)
{
_isFinalResult = isFinalResult;
_result = result;
_nextStep = nextStep;
}
public bool IsFinalResult { get { return _isFinalResult; } }
public T Result { get { return _result; } }
public Func<RecursionResult<T>> NextStep { get { return _nextStep; } }
}
class Program
{
static void Main(string[] args)
{
BigInteger result = TailRecursion.Execute(() => Factorial(50000, 1));
}
static RecursionResult<BigInteger> Factorial(int n, BigInteger product)
{
if (n < 2)
return TailRecursion.Return(product);
return TailRecursion.Next(() => Factorial(n - 1, n * product));
}
}
You can create a new thread with the stacksize you want...
var tcs = new TaskCompletionSource<BigInteger>();
int stackSize = 1024*1024*1024;
new Thread(() =>
{
tcs.SetResult(Factorial(10000));
},stackSize)
.Start();
var result = tcs.Task.Result;
But as mentioned in comments, an iterative way for this would be better..
I am searching for right thread-safe collection (concurrent collection) for the following scenario:
I may have requests from an external source which generates GUIDs (so it is unique and non-recurring). I need to store (say the last 100 requests) and check if duplicate GUIDs are delivered or not. I may not save all GUIDs more than 100 or so due to some limitations.
Now the problem is that when this mechanism is used in a service, it must be bound to 100 items and searching based on GUIDs is vital.
I decided to use ConcurrentDictionary yet I doubt it is a good decision since I may change the keys after using the whole 100 slots. I may find a good mechanism to replace the oldest requests when dictionary is full.
Any idea is much appreciated.
A code snippet is provided to show my incomplete implementation
public static ConcurrentDictionary<string, TimedProto> IncidentsCreated = new ConcurrentDictionary<string, TimedProto>(20, 100);
private static bool AddTo_AddedIncidents(proto ReceivedIncident)
{
try
{
int OldestCounter = 0;
DateTime OldestTime = DateTime.Now;
if (IncidentsCreated.Count < 100)
{
TimedProto tp = new TimedProto();
tp.IncidentProto = ReceivedIncident;
tp.time = DateTime.Now;
IncidentsCreated.AddOrUpdate(ReceivedIncident.IncidentGUID, tp,
(s,i) => i);
return true;
}
else //array is full, a replace oldest mechanism is required
{
}
return true;
}
catch (Exception ex)
{
LogEvent("AddTo_AddedIncidents\n"+ex.ToString(), EventLogEntryType.Error);
return false;
}
}
public struct proto
{
public string IncidentGUID;
//other variables
}
public struct TimedProto
{
public proto IncidentProto;
public DateTime time;
}
Thanks
Try this one: http://ayende.com/blog/162529/trivial-lru-cache-impl?key=02e8069c-62f8-4042-a7d2-d93806369824&utm_source=feedburner&utm_medium=feed&utm_campaign=Feed%3A+AyendeRahien+%28Ayende+%40+Rahien%29
Your implementation is flawed since you do use DateTime which has a granularity of 15ms. This means that you can accidentally delete even your most recent guids if you have a high inflow.
public class LruCache<TKey, TValue>
{
private readonly int _capacity;
private readonly Stopwatch _stopwatch = Stopwatch.StartNew();
class Reference<T> where T : struct
{
public T Value;
}
private class Node
{
public TValue Value;
public volatile Reference<long> Ticks;
}
private readonly ConcurrentDictionary<TKey, Node> _nodes = new ConcurrentDictionary<TKey, Node>();
public LruCache(int capacity)
{
Debug.Assert(capacity > 10);
_capacity = capacity;
}
public void Set(TKey key, TValue value)
{
var node = new Node
{
Value = value,
Ticks = new Reference<long> { Value = _stopwatch.ElapsedTicks }
};
_nodes.AddOrUpdate(key, node, (_, __) => node);
if (_nodes.Count > _capacity)
{
foreach (var source in _nodes.OrderBy(x => x.Value.Ticks).Take(_nodes.Count / 10))
{
Node _;
_nodes.TryRemove(source.Key, out _);
}
}
}
public bool TryGet(TKey key, out TValue value)
{
Node node;
if (_nodes.TryGetValue(key, out node))
{
node.Ticks = new Reference<long> {Value = _stopwatch.ElapsedTicks};
value = node.Value;
return true;
}
value = default(TValue);
return false;
}
}
I would use a Circular Buffer for this - there are plenty of implementations around, including this one, and making a thread-safe wrapper for one of them wouldn't be hard.
With only 100 or so slots, looking up by key would be reasonably efficient, and inserting would be extremely efficient (no reallocation as old items are discarded and replaced by new ones).