Develop Reference

concurrect oprating with limit on thread count in a infinity loop

I write an infinity loop for pulling from queue(RabbitMQ) and processing each pulled item in concurrent threads with limited count on running threads.
Now i want a solution for make a limit in thread execution count.see an example of my loop:
public class ThreadWorker<T>
{
public List<T> _lst;
private int _threadCount;
private int _maxThreadCount;
public ThreadWorker(List<T> lst, int maxThreadCount)
{
_lst = lst;
_maxThreadCount = maxThreadCount;
}
public void Start()
{
var i = 0;
while (i < _lst.Count)
{
i++;
var pull = _lst[i];
Process(pull);
}
}
public void Process(T item)
{
if (_threadCount > _maxThreadCount)
{
//wait any opration be done
// How to wait for one thread?
Interlocked.Decrement(ref _threadCount);
}
var t = new Thread(() => Opration(item));
t.Start();
Interlocked.Increment(ref _threadCount);
}
public void Opration(T item)
{
Console.WriteLine(item.ToString());
}
}
Notice that when i use a semaphore for limitation, Start() method don't wait for all running threads. my loop should after running threads with _maxThreadCount, be wait until release a thread and then push new thread for concurrent processing.

I would use Semaphore this way to control the number of threads:
public class ThreadWorker<T>
{
SemaphoreSlim _sem = null;
List<T> _lst;
public ThreadWorker(List<T> lst, int maxThreadCount)
{
_lst = lst;
_sem = new SemaphoreSlim(maxThreadCount);
}
public void Start()
{
var i = 0;
while (i < _lst.Count)
{
i++;
var pull = _lst[i];
_sem.Wait(); /*****/
Process(pull);
}
}
public void Process(T item)
{
var t = new Thread(() => Opration(item));
t.Start();
}
public void Opration(T item)
{
Console.WriteLine(item.ToString());
_sem.Release(); /*****/
}
}

Read-write synchronization class implementation

I am writing a read-write synchronization class, and would like some advice on what I to do next. For some reason, it sometimes allows a Read to happen in the middle of a Write, and I cannot find the reason.
This is what I want from this class:
Reads not allowed at the same time as writes.
Multiples reads can happen at the same time.
Only one write can happen at a time.
When a write is needed, all already executing reads continue,
no new reads are allowed, when all reads finish the write executes.
I know that .Net framework has a class to do this... but what I want is to understand and to reproduce something like that. I'm not reinventing the wheel, I am trying to understand it by making my own wheel... happens that my wheel is kinda squared a bit.
What I have currently is this:
public class ReadWriteSync
{
private ManualResetEvent read = new ManualResetEvent(true);
private volatile int readingBlocks = 0;
private AutoResetEvent write = new AutoResetEvent(true);
private object locker = new object();
public IDisposable ReadLock()
{
lock (this.locker)
{
this.write.Reset();
Interlocked.Increment(ref this.readingBlocks);
this.read.WaitOne();
}
return new Disposer(() =>
{
if (Interlocked.Decrement(ref this.readingBlocks) == 0)
this.write.Set();
});
}
public IDisposable WriteLock()
{
lock (this.locker)
{
this.read.Reset();
this.write.WaitOne();
}
return new Disposer(() =>
{
this.read.Set();
if (this.readingBlocks == 0)
this.write.Set();
});
}
class Disposer : IDisposable
{
Action disposer;
public Disposer(Action disposer) { this.disposer = disposer; }
public void Dispose() { this.disposer(); }
}
}
This is my test program... when something goes wrong it prints the lines in red.
class Program
{
static ReadWriteSync sync = new ReadWriteSync();
static void Main(string[] args)
{
Console.BackgroundColor = ConsoleColor.DarkGray;
Console.ForegroundColor = ConsoleColor.Gray;
Console.Clear();
Task readTask1 = new Task(() => DoReads("A", 20));
Task readTask2 = new Task(() => DoReads("B", 30));
Task readTask3 = new Task(() => DoReads("C", 40));
Task readTask4 = new Task(() => DoReads("D", 50));
Task writeTask1 = new Task(() => DoWrites("E", 500));
Task writeTask2 = new Task(() => DoWrites("F", 200));
readTask1.Start();
readTask2.Start();
readTask3.Start();
readTask4.Start();
writeTask1.Start();
writeTask2.Start();
Task.WaitAll(
readTask1, readTask2, readTask3, readTask4,
writeTask1, writeTask2);
}
static volatile bool reading;
static volatile bool writing;
static void DoWrites(string name, int interval)
{
for (int i = 1; i < int.MaxValue; i += 2)
{
using (sync.WriteLock())
{
Console.ForegroundColor = (writing || reading) ? ConsoleColor.Red : ConsoleColor.Gray;
writing = true;
Console.WriteLine("WRITE {1}-{0} BEGIN", i, name);
Thread.Sleep(interval);
Console.WriteLine("WRITE {1}-{0} END", i, name);
writing = false;
}
Thread.Sleep(interval);
}
}
static void DoReads(string name, int interval)
{
for (int i = 0; i < int.MaxValue; i += 2)
{
using (sync.ReadLock())
{
Console.ForegroundColor = (writing) ? ConsoleColor.Red : ConsoleColor.Gray;
reading = true;
Console.WriteLine("READ {1}-{0} BEGIN", i, name);
Thread.Sleep(interval * 3);
Console.WriteLine("READ {1}-{0} END", i, name);
reading = false;
}
Thread.Sleep(interval);
}
}
}
What is wrong with all this... any advice on how to do it correctly?

The primary issue that I see is that you are trying to make reset events encompass both the meanings of a read/write and the handling of their current state, without synchronizing in a consistent manner.
Here's an example of how the inconsistent synchronization may bite you in your specific code.
A write is disposing and a read is coming in.
The read acquires the lock
The write sets the read ManualResetEvent (MRE)
The write checks the current read count, finding 0
The read resets the write AutoResetEvent (ARE)
The read increments the read count
The read finds its MRE has been set and begins to read
All is fine so far, but the write hasn't finished yet...
A second write comes in and acquires the lock
The second write resets the read MRE
The first write finishes by setting the write ARE
The second write finds its ARE has been set and begins to write
When thinking about multiple threads, unless you are within a lock of some sort, you must take the view that all other data is wildly fluctuating and cannot be trusted.
A naive implementation of this may split out the queueing logic from the state logic and synchronize appropriately.
public class ReadWrite
{
private static int readerCount = 0;
private static int writerCount = 0;
private int pendingReaderCount = 0;
private int pendingWriterCount = 0;
private readonly object decision = new object();
private class WakeLock:IDisposable
{
private readonly object wakeLock;
public WakeLock(object wakeLock) { this.wakeLock = wakeLock; }
public virtual void Dispose() { lock(this.wakeLock) Monitor.PulseAll(this.wakeLock); }
}
private class ReadLock:WakeLock
{
public ReadLock(object wakeLock) : base(wakeLock) { Interlocked.Increment(ref readerCount); }
public override void Dispose()
{
Interlocked.Decrement(ref readerCount);
base.Dispose();
}
}
private class WriteLock:WakeLock
{
public WriteLock(object wakeLock) : base(wakeLock) { Interlocked.Increment(ref writerCount); }
public override void Dispose()
{
Interlocked.Decrement(ref writerCount);
base.Dispose();
}
}
public IDisposable TakeReadLock()
{
lock(decision)
{
pendingReaderCount++;
while (pendingWriterCount > 0 || Thread.VolatileRead(ref writerCount) > 0)
Monitor.Wait(decision);
pendingReaderCount--;
return new ReadLock(this.decision);
}
}
public IDisposable TakeWriteLock()
{
lock(decision)
{
pendingWriterCount++;
while (Thread.VolatileRead(ref readerCount) > 0 || Thread.VolatileRead(ref writerCount) > 0)
Monitor.Wait(decision);
pendingWriterCount--;
return new WriteLock(this.decision);
}
}
}

Await doesn't work with BeginInvoke?

I was experimenting with the new C# await feature. I made a custom awaiter implementation as follows:
using System;
using System.Runtime.CompilerServices;
using System.Threading;
namespace ConsoleApplication1
{
internal class Program
{
private static void Main(string[] args)
{
T1();
Console.WriteLine("After t1");
}
private static async void T1()
{
CustomAwaitable a = new Sleeper().Sleep();
object r = await a;
Console.WriteLine("sleeper awakes " + r);
}
}
internal class CustomAwaitable
{
private readonly Sleeper m_sleeper;
public CustomAwaitable(Sleeper s)
{
m_sleeper = s;
}
public MyAwaiter GetAwaiter()
{
return new MyAwaiter(m_sleeper);
}
}
internal class Sleeper
{
public ManualResetEvent Handle = new ManualResetEvent(false);
public bool Awake { get; set; }
public int Result
{
get { return Environment.TickCount; }
}
public CustomAwaitable Sleep()
{
new Thread(() =>
{
Thread.Sleep(5000);
Awake = true;
Handle.Set();
}).Start();
Console.WriteLine("begin sleeping " + Result);
return new CustomAwaitable(this);
}
}
internal class MyAwaiter : INotifyCompletion
{
private readonly Sleeper m_sleeper;
public MyAwaiter(Sleeper sleeper)
{
m_sleeper = sleeper;
}
public bool IsCompleted
{
get { return m_sleeper.Awake; }
}
public void OnCompleted(Action continuation)
{
// This works!!
//new Thread(() =>
//{
// m_sleeper.Handle.WaitOne();
// continuation();
//}).Start();
// This doesn't work!!
Action k = () =>
{
m_sleeper.Handle.WaitOne();
continuation();
};
k.BeginInvoke(null, null);
}
public object GetResult()
{
return m_sleeper.Result;
}
}
}
The problem is that, in the OnCompleted method, when I schedule the continuation code execution using BeginInvoke, the GetResult method is never called. But when I create a thread manually to do the same thing, everything works as expected. I know that BeginInvoke uses the thread pool internally, which should work the same way as the thread approach (I know that there is a thread count limit with thread pool, but it is negligible in this case as I am not running anything else).
What are your ideas? Thanks!

Troubleshoting while realising robin round

I'm trying to create a Sheduler (for fun) but it fails. Strange is that when i'm debugging in step-over style, my programm works fine, but when i'm removing all breakpoints it freezes after printing last value. So question: why does it freezes? Second: i'm using Thread.Resume and Thread.Suspend, but they are marked as obsolete. How can i avoid it?
Code is below:
using System;
using System.Collections.Generic;
using System.Threading;
namespace ConsoleApplication143
{
internal class Program
{
private static void Main()
{
var rrs = new RobinRoundSheduler(2, () =>
{
for (int i = 0; i < 2; i++)
{
Console.WriteLine("{0} {1}", i,
Thread.CurrentThread.ManagedThreadId);
}
}) {TimeForTask = new TimeSpan(1)};
rrs.Start();
Console.ReadKey();
}
}
internal class RobinRoundSheduler
{
private readonly LinkedList<Thread> _threads;
public TimeSpan TimeForTask { get; set; }
public RobinRoundSheduler(int taskCount, ThreadStart start)
{
TimeForTask = TimeSpan.FromSeconds(1);
_threads = new LinkedList<Thread>();
for (int i = 0; i < taskCount; i++)
{
_threads.AddLast(new Thread(start));
}
}
public void Start()
{
while (_threads.Count > 0)
{
var list = new List<Thread>();
foreach (var thread in _threads)
{
lock (thread)
{
if (thread.ThreadState == ThreadState.Unstarted)
thread.Start();
else
thread.Resume();
}
thread.Join(TimeForTask);
lock (thread)
{
if (thread.ThreadState == ThreadState.Stopped || thread.ThreadState == ThreadState.Aborted)
list.Add(thread);
else
{
thread.Suspend();
}
}
}
list.ForEach(thread => _threads.Remove(thread));
}
}
}
}
it seems having a deadlock problem due to Thread.Suspend() method but i dunno another alternative to suspend a thread withoud adding checks for ManualResetEvents in calling methods. But i want to call method knows nothing about multithreading.

Can this code be refactored by using the reactive framework?

copy paste the following code in new C# console app.
class Program
{
static void Main(string[] args)
{
var enumerator = new QueuedEnumerator<long>();
var listenerWaitHandle = Listener(enumerator);
Publisher(enumerator);
listenerWaitHandle.WaitOne();
}
private static AutoResetEvent Listener(IEnumerator<long> items)
{
var #event = new AutoResetEvent(false);
ThreadPool.QueueUserWorkItem((o) =>
{
while (items.MoveNext())
{
Console.WriteLine("Received : " + items.Current);
Thread.Sleep(2 * 1000);
}
(o as AutoResetEvent).Set();
}, #event);
return #event;
}
private static void Publisher(QueuedEnumerator<long> enumerator)
{
for (int i = 0; i < 10; i++)
{
enumerator.Set(i);
Console.WriteLine("Sended : " + i);
Thread.Sleep(1 * 1000);
}
enumerator.Finish();
}
class QueuedEnumerator<T> : IEnumerator<T>
{
private Queue _internal = Queue.Synchronized(new Queue());
private T _current;
private bool _finished;
private AutoResetEvent _setted = new AutoResetEvent(false);
public void Finish()
{
_finished = true;
_setted.Set();
}
public void Set(T item)
{
if (_internal.Count > 3)
{
Console.WriteLine("I'm full, give the listener some slack !");
Thread.Sleep(3 * 1000);
Set(item);
}
else
{
_internal.Enqueue(item);
_setted.Set();
}
}
public T Current
{
get { return _current; }
}
public void Dispose()
{
}
object System.Collections.IEnumerator.Current
{
get { return _current; }
}
public bool MoveNext()
{
if (_finished && _internal.Count == 0)
return false;
else if (_internal.Count > 0)
{
_current = (T)_internal.Dequeue();
return true;
}
else
{
_setted.WaitOne();
return MoveNext();
}
}
public void Reset()
{
}
}
}
2 threads (A,B)
A thread can provide one instance at a time and calls the Set method
B thread wants to receive a sequence of instances (provided by thread A)
So literally transforming an Add(item), Add(item), .. to a IEnumerable between different threads
Other solutions also welcome of course!

Sure - this code might not be the best way to do it, but here was my initial stab at it:
Subject<Item> toAddObservable;
ListObservable<Item> buffer;
void Init()
{
// Subjects are an IObservable we can trigger by-hand, they're the
// mutable variables of Rx
toAddObservable = new Subject(Scheduler.TaskPool);
// ListObservable will hold all our items until someone asks for them
// It will yield exactly *one* item, but only when toAddObservable
// is completed.
buffer = new ListObservable<Item>(toAddObservable);
}
void Add(Item to_add)
{
lock (this) {
// Subjects themselves are thread-safe, but we still need the lock
// to protect against the reset in FetchResults
ToAddOnAnotherThread.OnNext(to_add);
}
}
IEnumerable<Item> FetchResults()
{
IEnumerable<Item> ret = null;
buffer.Subscribe(x => ret = x);
lock (this) {
toAddObservable.OnCompleted();
Init(); // Recreate everything
}
return ret;
}