A robust method of tracking failed workers with ThreadPool - c#

I'm looking for a good method of tracking (counting) which workers have failed when queued with a Threadpool and using WaitHandle.WaitAll() for all threads to finish.
Is Interlocking a counter a good technique or is there a more robust strategy?

Okay, here's an approach that you could take. I've encapsulated the data that we want to track into a class TrackedWorkers. There is a constructor on this class that enables you to set how many workers will be working. Then, the workers are launched using LaunchWorkers which requires a delegate that eats an object and returns a bool. The object represents the input to the worker and the bool represents success or failure depending on true or false being the return value, respectively.
So basically what we do we have an array to track worker state. We launch the workers and set the status corresponding to that worker depending on the return value from the worker. When the worker returns, we set an AutoResetEvent and WaitHandle.WaitAll for all the AutoResetEvents to be set.
Note that there is an nested class to track the work (the delegate) the worker is supposed to do, the input to that work, and an ID used to set the status AutoResetEvent corresponding to that thread.
Note very carefully that once the work is done we are not holding a reference to the work delegate func nor to the input. This is important so that we don't accidentally prevent stuff from being garbage collected.
There are methods for getting the status of a particular worker, as well as all the indexes of the workers that succeeded and all the indexes of the workers that failed.
One last note: I do not consider this code production ready. It is merely a sketch of the approach that I would take. You need to take care to add testing, exception handling and other such details.
class TrackedWorkers {
class WorkerState {
public object Input { get; private set; }
public int ID { get; private set; }
public Func<object, bool> Func { get; private set; }
public WorkerState(Func<object, bool> func, object input, int id) {
Func = func;
Input = input;
ID = id;
}
}
AutoResetEvent[] events;
bool[] statuses;
bool _workComplete;
int _number;
public TrackedWorkers(int number) {
if (number <= 0 || number > 64) {
throw new ArgumentOutOfRangeException(
"number",
"number must be positive and at most 64"
);
}
this._number = number;
events = new AutoResetEvent[number];
statuses = new bool[number];
_workComplete = false;
}
void Initialize() {
_workComplete = false;
for (int i = 0; i < _number; i++) {
events[i] = new AutoResetEvent(false);
statuses[i] = true;
}
}
void DoWork(object state) {
WorkerState ws = (WorkerState)state;
statuses[ws.ID] = ws.Func(ws.Input);
events[ws.ID].Set();
}
public void LaunchWorkers(Func<object, bool> func, object[] inputs) {
Initialize();
for (int i = 0; i < _number; i++) {
WorkerState ws = new WorkerState(func, inputs[i], i);
ThreadPool.QueueUserWorkItem(this.DoWork, ws);
}
WaitHandle.WaitAll(events);
_workComplete = true;
}
void ThrowIfWorkIsNotDone() {
if (!_workComplete) {
throw new InvalidOperationException("work not complete");
}
}
public bool GetWorkerStatus(int i) {
ThrowIfWorkIsNotDone();
return statuses[i];
}
public IEnumerable<int> SuccessfulWorkers {
get {
return WorkersWhere(b => b);
}
}
public IEnumerable<int> FailedWorkers {
get {
return WorkersWhere(b => !b);
}
}
IEnumerable<int> WorkersWhere(Predicate<bool> predicate) {
ThrowIfWorkIsNotDone();
for (int i = 0; i < _number; i++) {
if (predicate(statuses[i])) {
yield return i;
}
}
}
}
Sample usage:
class Program {
static Random rg = new Random();
static object lockObject = new object();
static void Main(string[] args) {
int count = 64;
Pair[] pairs = new Pair[count];
for(int i = 0; i < count; i++) {
pairs[i] = new Pair(i, 2 * i);
}
TrackedWorkers workers = new TrackedWorkers(count);
workers.LaunchWorkers(SleepAndAdd, pairs.Cast<object>().ToArray());
Console.WriteLine(
"Number successful: {0}",
workers.SuccessfulWorkers.Count()
);
Console.WriteLine(
"Number failed: {0}",
workers.FailedWorkers.Count()
);
}
static bool SleepAndAdd(object o) {
Pair pair = (Pair)o;
int timeout;
double d;
lock (lockObject) {
timeout = rg.Next(1000);
d = rg.NextDouble();
}
Thread.Sleep(timeout);
bool success = d < 0.5;
if (success) {
Console.WriteLine(pair.First + pair.Second);
}
return (success);
}
}
The above program is going to launch sixty-four threads. The ith thread has the task of adding the numbers i and 2 * i and printing the result to the console. However, I have added a random amount of sleep (less than one second) to simulate busyness and I flip a coin to determine success or failure of the thread. Those that succeed print the sum they were tasked with and return true. Those that fail print nothing and return false.
Here I have used
struct Pair {
public int First { get; private set; }
public int Second { get; private set; }
public Pair(int first, int second) : this() {
this.First = first;
this.Second = second;
}
}

Related

Thread executes "slowly"

I am starting with threads and wrote for the sake of learning the following simple program, which later would be used to calculate about 100,000 times a formula (it is a relatively simple one but which takes an iterated range of values).
The problem with it is that I expected every thread to execute in almost no time and thus the complete program to finish nearly immediately, but the fact is that everything runs too slow (about 10s)...
static readonly double TotalIterations = 1000;
public static Iterations ActualIterations = new Iterations();
public static void Main()
{
var par1 = "foo";
var par2 = "boo";
var par3 = 3;
for (int i = 0; i < TotalIterations; i++)
{
new Thread(() => new Calculations().Calculate(par1, par2, par3)).Start();
}
AwaitingThreads();
}
static void AwaitThreads()
{
Console.WriteLine("Awaiting threads to finished...");
while (true)
{
lock (ActualIterations)
{
if (ActualIterations.Progress() == TotalIterations) break;
}
Thread.Sleep(1 * 1000);
}
Console.WriteLine("All threads finished!");
}
public class Calculations {
public bool Calculate(string par1, string par2, int par3)
{
// ...
bool result = false;
lock (ActualIterations)
{
ActualIterations.Incr();
}
return result;
}
}
public class Iterations
{
int progress = 0;
public void Incr()
{
progress++;
}
public int Progress()
{
return progress;
}
}
I also tried using a ThreadPool like this, but there was no improvement...
static readonly double TotalIterations = 1000;
static string par1 = "foo";
static string par2 = "boo";
static int par3 = 3;
public static Iterations ActualIterations = new Iterations();
public static void Main()
{
ThreadPool.QueueUserWorkItem(MyThreadPool);
AwaitThreads();
}
static void AwaitThreads()
{
Console.WriteLine("Awaiting threads to finished...");
while (true)
{
lock (ActualIterations)
{
if (ActualIterations.Progress() == TotalIterations) break;
}
Thread.Sleep(1 * 1000);
}
Console.WriteLine("All threads finished!");
}
static void MyThreadPool(Object stateInfo)
{
for (int i = 0; i < TotalIterations; i++)
{
new Thread(() => new Calculations().Calculate(par1, par2, par3)).Start();
}
}
public class Calculations {
public bool Calculate(string par1, string par2, int par3)
{
// ...
bool result = false;
lock (ActualIterations)
{
ActualIterations.Incr();
}
return result;
}
}
public class Iterations
{
int progress = 0;
public void Incr()
{
progress++;
}
public int Progress()
{
return progress;
}
}
When I quit using threads in this example and use a static method, executing it sequentially in my for loop, the program finishes in 1s...
Can anybody enlighten me what I am doing wrong here with those threads?
The problem with it is that I expected every thread to execute in almost no time
Right. You're ignoring the fact that creating a new thread is a relatively expensive operation. Far, far more expensive than "acquiring a lock and incrementing an integer" which is the work you're doing in the thread.
To give a real world comparison, it's a little like ordering a new car, waiting it to be delivered, and then driving it 1km. That's going to be slower than just walking 1km.
Using the thread pool would be faster, but you're not using it correctly - you're launching one thread pool task which then creates all the other threads again.
I would encourage you to look at using Task<T> instead, which normally uses the thread pool under the hood, and is a generally more modern abstraction for this sort of work.
This is the way to proceed doing what you wanted to do:
class Program
{
static void Main(string[] args)
{
List<Task> tasks = new List<Task>();
for (int i = 0; i < 1000; i++)
{
tasks.Add(Task.Run(() =>
{
Console.WriteLine("Calculations " + DateTime.Now);
}));
}
Task.WaitAll(tasks.ToArray());
}
}
Tasks are actually optimized and programmer-friendly to use if you need to work with threads.
Another advice i want to give you is to create an Object just for locking purposes, example:
class Program
{
private static Object _locker = new Object();
static void Main(string[] args)
{
List<Task> tasks = new List<Task>();
for (int i = 0; i < 1000; i++)
{
tasks.Add(Task.Run(() =>
{
lock (_locker)
{
Console.WriteLine("Calculations " + DateTime.Now);
}
}));
}
Task.WaitAll(tasks.ToArray());
}
}
I see the problem in the AwaitThreads method.
It uses the same lock (ActualIterations) as working thread and it makes working threads to wait for shared resource additionally.
Also (as it was mentioned by #Euphoric) the thread working code you have shown is just about single increment and it uses the shared resource between all threads.
You have to change it in some another way and try to avoid shared resource usage in multi threaded environment.
For example, if you need to make some calculation on huge data array you have to feed each thread own data part to be processed and then wait for all tasks to be finished. There is Task concept and Task.WaitAll

c# Threading Lock and Monitor

I'm currently running into a problem with multithreading and accessing a static list. A static list holds all items with several properties. The items are identified with a Guid. A main work thread changes some properties for any item in the static list. The child threads all have their own Guid, with this Guid they read their own item in the static list. And after a specific event they remove their assigned element from the static list.
To get to the source I have broken down my code to the essential methods and classes. The work thread has the following simplified code
public void RunWork()
{
Random random = new Random();
Int32 index = -1;
while (!Kill)
{
Thread.Sleep(1);
if (MainWindow.Clients != null)
{
index = random.Next(0, MainWindow.Clients.Count);
MainWindow.Clients[index].State = MainWindow.RandomString(9);
}
}
}
Each child thread has the following simplified code
public void RunChild()
{
Random random = new Random();
while (!Kill)
{
Thread.Sleep(100);
if (MainWindow.Clients.Any(x => x.Id == Id))
{
this.State = MainWindow.Clients.First(x => x.Id == Id).State;
}
Thread.Sleep(random.Next(50));
if (random.Next(100) % 90 == 0)
{
Kill = true;
MainWindow.Clients.RemoveAll(x => x.Id == Id);
}
}
}
If a child removes itself from the MainWindow.Clients list the work thread throws a exception, that the index it is trying to access does not exist.
I have added lock statments around every access of MainWindow.Clients but this does not prevent the work thread from accessing a deleted item. I have also tried Monitor.Enter(MainWindow.Clients) and Monitor.Exit(MainWindow.Clients) but with the same result as with lock.
The static list MainWindow.Clients is created before any thread runs and never gets recreated or disposed.
If the lock statement is set around this block of code in the RunWork() method
lock (MainWindow.Clients)
{
Thread.Sleep(1);
if (MainWindow.Clients != null)
{
index = random.Next(0, MainWindow.Clients.Count);
MainWindow.Clients[index].State = MainWindow.RandomString(9);
}
}
Why does it not block the child threads from changing the list between the lines
where the random index is set and the list gets accessed?
Update 1:
The following code still throws a IndexOutOfRangeException at MainWindow.Clients[index].State = MainWindow.RandomString(9);:
public void RunWork()
{
Random random = new Random();
Int32 index = -1;
while (!Kill)
{
Thread.Sleep(1);
if (MainWindow.Clients != null)
{
lock (MainWindow.Clients)
{
index = random.Next(0, MainWindow.Clients.Count);
MainWindow.Clients[index].State = MainWindow.RandomString(9);
}
}
}
}
public void RunChild()
{
Random random = new Random();
while (!Kill)
{
Thread.Sleep(100);
if (MainWindow.Clients.Any(x => x.Id == Id))
{
this.State = MainWindow.Clients.First(x => x.Id == Id).State;
}
Thread.Sleep(random.Next(50));
if (random.Next(100) % 90 == 0)
{
Kill = true;
lock (MainWindow.Clients)
{
MainWindow.Clients.RemoveAll(x => x.Id == Id);
}
}
}
}
Update 2: Here is the complete code for the quick sample application
Update 3: I have edited my code and wrapped all accesses of MainWindow.Clients with lock statements. But still the threads access the variable while it is locked:
I'm not sure what exactly you are trying to achieve, but I've written something that might help you find the correct solution. Sorry for the lack of correctness - tight schedule ;-)
using System;
using System.Collections.Concurrent;
using System.Collections.Generic;
using System.Diagnostics;
using System.Linq;
using System.Threading;
namespace ConcurrentCollectionTest
{
internal class Client
{
public string State
{
get; set;
}
public string Name
{
get;
internal set;
}
}
internal class MainWindow
{
private ConcurrentDictionary<int, Client> _dict = new ConcurrentDictionary<int, Client>();
public IDictionary<int, Client> Clients
{
get
{
return _dict;
}
}
}
internal class Program
{
private static bool killAll = false;
private static MainWindow mainWindow = new MainWindow();
private static int id = -100;
private static string state = "Initial";
private static Random random = new Random();
private static object lockObject = new object();
internal static string RandomString(int v)
{
int k = random.Next(0, v);
return k.ToString();
}
public static void RunChild()
{
Debug.WriteLine($"child running {Thread.CurrentThread.Name}");
bool killThis = false;
while (!killThis && !killAll)
{
Thread.Sleep(100);
Client client = null;
if (mainWindow.Clients.TryGetValue(id, out client))
{
state = client.State;
}
Thread.Sleep(random.Next(50));
if (random.Next(100) % 90 == 0)
{
Debug.WriteLine($"killing {Thread.CurrentThread.Name}");
killThis = true;
lock (lockObject)
{
mainWindow.Clients.Remove(id);
}
}
}
}
public static void RunWork()
{
Console.WriteLine("RunWork");
Random random = new Random();
Int32 index = -1;
while (!killAll)
{
if (!mainWindow.Clients.Any())
{
killAll = true;
break;
}
Thread.Sleep(100);
// edit: still need lock here as count can change in between
Client client = null;
lock (lockObject)
{
index = random.Next(0, mainWindow.Clients.Count);
client = mainWindow.Clients[index];
}
Debug.WriteLine($"Changing {client.Name}");
client.State = RandomString(9);
}
Console.WriteLine("Worker killed");
}
private static void Main(string[] args)
{
Console.WriteLine("Starting. Enter id or kill");
for (int i = 0; i < 100; i++)
{
mainWindow.Clients.Add(i, new Client
{
Name = $"Client {i:000}",
State = "Unknown"
});
}
var worker = new Thread(RunWork);
worker.Start();
var threadList = new List<Thread>();
threadList.Add(worker);
for (int i = 0; i < 10; i++)
{
var thread = new Thread(RunChild)
{
Name = $"Child {i:00}"
};
threadList.Add(thread);
thread.Start();
}
while (!killAll)
{
var str = Console.ReadLine();
if (str.Equals("kill", StringComparison.InvariantCultureIgnoreCase))
{
killAll = true;
break;
}
int enteredId = -1;
if (int.TryParse(str, out enteredId))
{
id = enteredId;
}
}
foreach (var thread in threadList)
{
thread.Join();
}
Console.WriteLine("all dead");
}
}
}

Cost of ExecutionContext.Capture() and ExecutionContext.Run(context, work, state)

Does anybody know if it is expensive to ExecutionContext.Capture() and to ExecutionContext.Run(context, work, state)?
Does it decrease performance and so is recommended to use carefully?
I'm asking since I have an ContextItem where I save Context work and state in to execute later. Since I want to be able to react on an exception that might be thrown while executing work, I have a fallback that is executed if an exceptrion is thrown in work. And I also have final work, that is executed in any case regardlessly if an exception was thrown or not.
Since I can use an ExecutionContext only once I would have to ExecutionContext.Capture() three times for one of these ContextItems...
Or does this sound like a totally wrong approach?
As recommended by #Alois Kraus I ran a test with the following code comparing locking to capturing & lined up excecution:
class Program
{
private static readonly object _lock = new object();
private static readonly int numberOfItems = 1000000;
private static readonly int _numberOfIterations = 1000000;
private static void Main(string[] args)
{
MeasureTimeWithLocking();
MeasureTimeWithCapuringContext();
Console.WriteLine();
MeasureTimeWithLocking();
MeasureTimeWithCapuringContext();
Console.WriteLine();
MeasureTimeWithLocking();
MeasureTimeWithCapuringContext();
Console.ReadKey();
}
private static void MeasureTimeWithLocking()
{
List<ContextItem> items = new List<ContextItem>();
Stopwatch stopwatch = Stopwatch.StartNew();
for (int i = 0; i < numberOfItems; i++)
{
ContextItem item = new ContextItem();
item.Work1 = DoSomeWorkWithLock;
item.Work2 = DoSomeWorkWithLock;
item.Work3 = DoSomeWorkWithLock;
}
Parallel.ForEach(items, (item) =>
{
item.Work1(null);
item.Work2(null);
item.Work3(null);
});
stopwatch.Stop();
Console.WriteLine("Time elapsed with locking: " + stopwatch.Elapsed);
}
private static void MeasureTimeWithCapuringContext()
{
List<ContextItem> items = new List<ContextItem>();
Stopwatch stopwatch = Stopwatch.StartNew();
for (int i = 0; i < numberOfItems; i++)
{
ContextItem item = new ContextItem();
item.Context1 = ExecutionContext.Capture();
item.Context2 = ExecutionContext.Capture();
item.Context3 = ExecutionContext.Capture();
item.Work1 = DoSomeWork;
item.Work2 = DoSomeWork;
item.Work3 = DoSomeWork;
}
foreach (ContextItem item in items)
{
ExecutionContext.Run(item.Context1, item.Work1, null);
ExecutionContext.Run(item.Context2, item.Work2, null);
ExecutionContext.Run(item.Context3, item.Work3, null);
}
stopwatch.Stop();
Console.WriteLine("Time elapsed with capturing context: " + stopwatch.Elapsed);
}
private static void DoSomeWork(object ignored)
{
Work();
}
private static void DoSomeWorkWithLock(object ignored)
{
lock (_lock)
{
Work();
}
}
private static void Work()
{
int count = 0;
for (int i = 0; i < _numberOfIterations; i++)
{
count ++;
}
}
private class ContextItem
{
public ExecutionContext Context1 { get; set; }
public ExecutionContext Context2 { get; set; }
public ExecutionContext Context3 { get; set; }
public ContextCallback Work1 { get; set; }
public ContextCallback Work2 { get; set; }
public ContextCallback Work3 { get; set; }
}
}
Results are:
So if I did this right, capturing & executing lined up is in average round about 5 times more expensive than locking.
To also answer the part of my question:
Or does this sound like a totally wrong approach?
I read in this article that
if you have to know they’re there, either you’re doing something super advanced, or something’s gone wrong.
The article was recommended on SO as the best source if you want to know about ExecutionContext.
After going through it and running some tests with a colleague I realized that I was using ExecutionContext where it didn’t make sense, plus it is less performant then than locks and so it probably also is less performant than other threading functionalities / constructs.

Randomly Timeout(TimeSpan) doesn't work in Rx

The earlier post seems not very clear, so after some testing, I reopened this post with much more simplified words, hope somebody could help.
My singleton observable was turned from multiple source of I/O events, means they're concurrently raised up in underlying, based on testing (to prove Rx is not thread safe) and RX design guideline, I made it serialized, see that lock(...):
public class EventFireCenter
{
public static event EventHandler<GTCommandTerminalEventArg> OnTerminalEventArrived;
private static object syncObject = new object();
public static void TestFireDummyEventWithId(int id)
{
lock (syncObject)
{
var safe = OnTerminalEventArrived;
if (safe != null)
{
safe(null, new GTCommandTerminalEventArg(id));
}
}
}
}
This is the singleton Observable:
public class UnsolicitedEventCenter
{
private readonly static IObservable<int> publisher;
static UnsolicitedEventCenter()
{
publisher = Observable.FromEventPattern<GTCommandTerminalEventArg>(typeof(EventFireCenter), "OnTerminalEventArrived")
.Select(s => s.EventArgs.Id);
}
private UnsolicitedEventCenter() { }
/// <summary>
/// Gets the Publisher property to start observe an observable sequence.
/// </summary>
public static IObservable<int> Publisher { get { return publisher; } }
}
The scenario of Subscribe(...) can be described by following code, you can see the Subscribe(...) could be called concurrently in different threads:
for (var i = 0; i < concurrentCount; i++)
{
var safe = i;
Scheduler.Default.Schedule(() =>
{
IDisposable dsp = null;
dsp = UnsolicitedEventCenter.Publisher
.Timeout(TimeSpan.FromMilliseconds(8000))
.Where(incomingValue => incomingValue == safe)
.ObserveOn(Scheduler.Default)
//.Take(1)
.Subscribe((incomingEvent) =>
{
Interlocked.Increment(ref onNextCalledTimes);
dsp.Dispose();
}
, ex =>
{
Interlocked.Increment(ref timeoutExceptionOccurredTimes);
lock (timedOutEventIds)
{
// mark this id has been timed out, only for unit testing result check.
timedOutEventIds.Add(safe);
}
dsp.Dispose();
});
Interlocked.Increment(ref threadPoolQueuedTaskCount);
});
}
As pointed out times by experienced people, call Dispose() in OnNext(...) is not recommended, but let's ignore it here since the code was from production.
Now the problem is randomly that .Timeout(TimeSpan.FromMilliseconds(8000)) is not working, the ex was never called, anyone could see any abnormal in the code?
for testing, I setup the stress testing, but so far, I didn't reproduced it, while in production, it appeared several times per day. Just in case, I pasted all the testing code:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Reactive.Concurrency;
using System.Reactive.Linq;
using System.Text;
using System.Threading;
using System.Threading.Tasks;
namespace Rx
{
class Program
{
static void Main(string[] args)
{
// avoid thread creation delay in thread pool.
ThreadPool.SetMinThreads(200, 50);
// let the test run for 100 times
for (int t = 0; t < 100; t++)
{
Console.WriteLine("");
Console.WriteLine("======Current running times: " + t);
// at meantime, 150 XXX.Subscribe(...) will be called.
const int concurrentCount = 150;
// how many fake event will be fire to santisfy that 150 XXX.Subscribe(...).
const int fireFakeEventCount = 40;
int timeoutExceptionOccurredTimes = 0;
var timedOutEventIds = new List<int>();
int onNextCalledTimes = 0;
int threadPoolQueuedTaskCount = 0;
for (var i = 0; i < concurrentCount; i++)
{
var safe = i;
Scheduler.Default.Schedule(() =>
{
IDisposable dsp = null;
dsp = UnsolicitedEventCenter.Publisher
.Timeout(TimeSpan.FromMilliseconds(8000))
.Where(incomingValue => incomingValue == safe)
.ObserveOn(Scheduler.Default)
//.Take(1)
.Subscribe((incomingEvent) =>
{
Interlocked.Increment(ref onNextCalledTimes);
dsp.Dispose();
}
, ex =>
{
Interlocked.Increment(ref timeoutExceptionOccurredTimes);
lock (timedOutEventIds)
{
// mark this id has been timed out, only for unit testing result check.
timedOutEventIds.Add(safe);
}
dsp.Dispose();
});
Interlocked.Increment(ref threadPoolQueuedTaskCount);
});
}
Console.WriteLine("Starting fire event: " + DateTime.Now.ToString("HH:mm:ss.ffff"));
int threadPoolQueuedTaskCount1 = 0;
// simulate a concurrent event fire
for (int i = 0; i < fireFakeEventCount; i++)
{
var safe = i;
Scheduler.Default.Schedule(() =>
{
EventFireCenter.TestFireDummyEventWithId(safe);
Interlocked.Increment(ref threadPoolQueuedTaskCount1);
});
}
// make sure all proceeding task has been done in threadPool.
while (threadPoolQueuedTaskCount < concurrentCount)
{
Thread.Sleep(1000);
}
// make sure all proceeding task has been done in threadPool.
while (threadPoolQueuedTaskCount1 < fireFakeEventCount)
{
Thread.Sleep(100);
}
Console.WriteLine("Finished fire event: " + DateTime.Now.ToString("HH:mm:ss.ffff"));
// sleep a time which >3000ms.
Thread.Sleep(8000);
Console.WriteLine("timeoutExceptionOccurredTimes: " + timeoutExceptionOccurredTimes);
Console.WriteLine("onNextCalledTimes: " + onNextCalledTimes);
if ((concurrentCount - fireFakeEventCount) != timeoutExceptionOccurredTimes)
{
try
{
Console.WriteLine("Non timeout fired for these ids: " +
Enumerable.Range(0, concurrentCount)
.Except(timedOutEventIds).Except(Enumerable.Range(0, fireFakeEventCount)).Select(i => i.ToString())
.Aggregate((acc, n) => acc + "," + n));
}
catch (Exception ex) { Console.WriteLine("faild to output timedout ids..."); }
break;
}
if (fireFakeEventCount != onNextCalledTimes)
{
Console.WriteLine("onNextOccurredTimes assert failed");
break;
}
if ((concurrentCount - fireFakeEventCount) != timeoutExceptionOccurredTimes)
{
Console.WriteLine("timeoutExceptionOccurredTimes assert failed");
break;
}
}
Console.WriteLine("");
Console.WriteLine("");
Console.WriteLine("DONE!");
Console.ReadLine();
}
}
public class EventFireCenter
{
public static event EventHandler<GTCommandTerminalEventArg> OnTerminalEventArrived;
private static object syncObject = new object();
public static void TestFireDummyEventWithId(int id)
{
lock (syncObject)
{
var safe = OnTerminalEventArrived;
if (safe != null)
{
safe(null, new GTCommandTerminalEventArg(id));
}
}
}
}
public class UnsolicitedEventCenter
{
private readonly static IObservable<int> publisher;
static UnsolicitedEventCenter()
{
publisher = Observable.FromEventPattern<GTCommandTerminalEventArg>(typeof(EventFireCenter), "OnTerminalEventArrived")
.Select(s => s.EventArgs.Id);
}
private UnsolicitedEventCenter() { }
/// <summary>
/// Gets the Publisher property to start observe an observable sequence.
/// </summary>
public static IObservable<int> Publisher { get { return publisher; } }
}
public class GTCommandTerminalEventArg : System.EventArgs
{
public GTCommandTerminalEventArg(int id)
{
this.Id = id;
}
public int Id { get; private set; }
}
}
Most likely the Timeout is not triggering because you have it before the Where filter. This means that all events are flowing through and resetting the timer, and then most of the events get filtered by the Where clause. To your subscribing observer, it will seem like it never gets a result and the timeout never triggers. Move the Timeout to be after the Where and you should now have a system that times out individual observers if they do not get their expected event on time.

Mutithreading with sequence

I have a main task that is spawning threads to do some work. When the work is completed it will write to the console.
My problem is that some of the threads that are created later will finish faster than those created earlier. However I need the writing to the console to be done in the same exact sequence as the thread was created.
So if a thread had completed its task, while some earlier threads had not, it has to wait till those earlier threads complete too.
public class DoRead
{
public DoRead()
{
}
private void StartReading()
{
int i = 1;
while (i < 10000)
{
Runner r = new Runner(i, "Work" + i.ToString());
r.StartThread();
i += 1;
}
}
}
internal class Runner : System.IDisposable
{
int _count;
string _work = "";
public Runner(int Count, string Work)
{
_count = Count;
_work = Work;
}
public void StartThread()
{
ThreadPool.QueueUserWorkItem(new WaitCallback(runThreadInPool), this);
}
public static void runThreadInPool(object obj)
{
((Runner)obj).run();
}
public void run()
{
try
{
Random r = new Random();
int num = r.Next(1000, 2000);
DateTime end = DateTime.Now.AddMilliseconds(num);
while (end > DateTime.Now)
{
}
Console.WriteLine(_count.ToString() + " : Done!");
}
catch
{
}
finally
{
_work = null;
}
}
public void Dispose()
{
this._work = null;
}
}
There may be a simpler way to do this than I used, (I'm used to .Net 4.0).
using System;
using System.Collections.Generic;
using System.Text;
using System.Threading;
namespace ConsoleApplication5
{
class Program
{
public static readonly int numOfTasks = 100;
public static int numTasksLeft = numOfTasks;
public static readonly object TaskDecrementLock = new object();
static void Main(string[] args)
{
DoRead dr = new DoRead();
dr.StartReading();
int tmpNumTasks = numTasksLeft;
while ( tmpNumTasks > 0 )
{
Thread.Sleep(1000);
tmpNumTasks = numTasksLeft;
}
List<string> strings = new List<string>();
lock( DoRead.locker )
{
for (int i = 1; i <= Program.numOfTasks; i++)
{
strings.Add( DoRead.dicto[i] );
}
}
foreach (string s in strings)
{
Console.WriteLine(s);
}
Console.ReadLine();
}
public class DoRead
{
public static readonly object locker = new object();
public static Dictionary<int, string> dicto = new Dictionary<int, string>();
public DoRead()
{
}
public void StartReading()
{
int i = 1;
while (i <= Program.numOfTasks )
{
Runner r = new Runner(i, "Work" + i.ToString());
r.StartThread();
i += 1;
}
}
}
internal class Runner : System.IDisposable
{
int _count;
string _work = "";
public Runner(int Count, string Work)
{
_count = Count;
_work = Work;
}
public void StartThread()
{
ThreadPool.QueueUserWorkItem(new WaitCallback(runThreadInPool), this);
}
public static void runThreadInPool(object obj)
{
Runner theRunner = ((Runner)obj);
string theString = theRunner.run();
lock (DoRead.locker)
{
DoRead.dicto.Add( theRunner._count, theString);
}
lock (Program.TaskDecrementLock)
{
Program.numTasksLeft--;
}
}
public string run()
{
try
{
Random r = new Random();
int num = r.Next(1000, 2000);
Thread.Sleep(num);
string theString = _count.ToString() + " : Done!";
return theString;
}
catch
{
}
finally
{
_work = null;
}
return "";
}
public void Dispose()
{
this._work = null;
}
}
}
}
Basically, I store the string you want printed from each task into a dictionary where the index is the task#. (I use a lock to make accessing the dictionary safe).
Next, so that the main program waits until all the background threads are done, I used another locked access to a NumTasksLeft variable.
I added stuff into the callback for the Runner.
It is bad practice to use busy loops, so I changed it to a Thread.Sleep( num ) statement.
Just change numOfTasks to 10000 to match your example.
I pull the return strings out of the dictionary in order, and then print it to the screen.
I'm sure you could refactor this to move or otherwise deal with the global variables, but this works.
Also, you might have noticed I didn't use the lock in the command
tmpNumTasks = numTasksLeft;
That's threadsafe, since numTasksLeft is an int which is read atomically on 32-bit computers and higher.
I don't know much on C#, but the whole idea of multi-threading is that you have multiple thread executing independently and you can never know which one will finish earlier (and you shouldn't expect earlier thread to end earlier).
One workaround is, instead writing out the finish message in the processing thread, have the processing thread setup a flag somewhere (probably a list with no of elements = no of thread spawned), and have a separate thread print out the finish message base on the flags in that list, and report up to the position that previous flag is consecutively "finished".
Honestly I don't feel that reasonable for you to print finish message like this anyway. I think changing the design is way better to have such meaningless "feature".
Typically, such requirements are met with an incrementing sequence number, much as you have already done.
Usually, the output from the processing threads is fed through a filter object that contains a list, (or dictionary), of all out-of-order result objects, 'holding them back' until all results with a lower seqeuence-number have come in. Again, similar to what you have already done.
What is not necessary is any kind of sleep() loop. The work threads themselves can operate the filter object, (which would beed a lock), or the work threads can producer-consumer-queue the results to an 'output thread' that operates the out-of-order filter.
This scheme works fine with pooled work threads, ie. those without continual create/terminate/destroy overhead.

Categories

Resources