There is a global queue of objects that you have to send to your customers. Queue is continually filled with new elements in its flow (one element in a second), that`s why you have to send constantly. Every client is served in a separate thread. After the object is sent to all clients it must be removed from the queue. It seems to be easy, but how to know that all the threads have already sent a particular object?
I do everything on the socket.
Thread threadForClientSending = new Thread(delegate()
{
while (true)
{
try
{
List<SymbolsTable> [] localArrayList ;
//main.que -- global queue
foreach (var eachlist in localArrayList = main.que.ToArray())
{
foreach (var item in eachlist)
{
byte[] message =
encoding.GetBytes((item.GetHashCode()%100).ToString() + " "+item.SDate +"\n\r");
client.Send(message);
}
Thread.Sleep(500);
}
}
catch (Exception ex)
{
Console.WriteLine(ex.Message);
break;
}
}
});
Such code sends everything to everyone, but it doesn`t clean the queue.
How to clean the queue from all of the processed items?
public static ConcurrentQueue<List<SymbolsTable>> que = new ConcurrentQueue<List<SymbolsTable>>();
public partial class SymbolsTable
{
public string SName { get; set; }
public Nullable<double> SPrice { get; set; }
public Nullable<int> SVolume { get; set; }
public System.DateTime SDate { get; set; }
}
NOTE: I highly recommend to define local queue for each client (task in server) to achieve maximum concurrency and cleaner code.
You can achieve what you need by using a CountDownEvent which would hold thread access for each item, We should set it to number of available worker that send data to clients.
here is how we can do it:
Definitions:
public static ConcurrentQueue<List<SymbolsTable>> que = new ConcurrentQueue<List<SymbolsTable>>();
public static CountdownEvent counter = new CountdownEvent(NumberOfThreads);
private const int NumberOfThreads = 3; //for example we have 3 clients here
Thread:
Thread threadForClientSending = new Thread(delegate()
{
while (true)
{
try
{
List<SymbolsTable> list;
var peek = que.TryPeek(out list);
if (!peek)
{
Thread.Sleep(100); //nothing to pull
continue;
}
foreach (var item in list)
{
main.que -- global queue
byte[] message =
encoding.GetBytes((item.GetHashCode() % 100).ToString() + " " + item.SDate + "\n\r");
client.Send(message);
Thread.Sleep(500);
}
counter.Signal(); //the thread would signal itself as finished, and wait for others to finish the task
lock (que)
{
List<SymbolsTable> lastList;
if (que.TryPeek(out lastList) && lastList.Equals(list))
{
//just one of the threads would dequeue the item
que.TryDequeue(out lastList);
counter.Reset(); //reset counter for next iteration
}
}
}
catch (Exception ex)
{
Console.WriteLine(ex.Message);
break;
}
}
});
here we used TryPeek to just access the item in queue so we won't remove it, at the end in:
lock (que)
{
List<SymbolsTable> lastList;
if (que.TryPeek(out lastList) && lastList.Equals(list))
{
//just one of the threads would dequeue the item
que.TryDequeue(out lastList);
counter.Reset(); //reset counter for next iteration
}
}
we would lock the que so only one thread at a time can access it, then we check to see if the processed item has been removed from queue and if not we will remove it here.
More Elegant Solution (in my Humble Opinion):
as you saw in previous solution we're blocking threads to finish the task for each item together,adding a local queue to each thread would remove this blocking mechanism, so we can achieve maximum concurrency.
I suggest something like:
class GlobalQueue
{
private readonly List<IMyTask> _subscribers=new List<IMyTask>();
public void Subscribe(IMyTask task)
{
_subscribers.Add(task);
}
public void Unsubscribe(IMyTask task)
{
_subscribers.Remove(task);
}
public void Enqueue(List<SymbolsTable> table)
{
foreach (var s in _subscribers)
s.Enqueue(table);
}
}
interface IMyTask
{
void Enqueue(List<SymbolsTable> table);
}
which your task would be roughly like:
class MyTask : IMyTask
{
private readonly ConcurrentQueue<List<SymbolsTable>> _localQueue = new ConcurrentQueue<List<SymbolsTable>>();
private readonly Thread _thread;
private bool _started;
public void Enqueue(List<SymbolsTable> table)
{
_localQueue.Enqueue(table);
}
public MyTask()
{
_thread = new Thread(Execute);
}
public void Start()
{
_started = true;
_thread.Start();
}
public void Stop()
{
_started = false;
}
private void Execute()
{
while (_started)
{
try
{
List<SymbolsTable> list;
var peek = _localQueue.TryDequeue(out list);
if (!peek)
{
Thread.Sleep(100); //nothing to pull
continue;
}
foreach (var item in list)
{
byte[] message =
encoding.GetBytes((item.GetHashCode() % 100).ToString() + " " + item.SDate + "\n\r");
client.Send(message);
}
}
catch (Exception ex)
{
Console.WriteLine(ex.Message);
break;
}
}
}
}
Related
I have wpf app. I have 3 labels, 1 button , (Start/Stop). I have databinding. My MainWindowViewModel loooks like beneath:
namespace StartStop
{
public class MainWindowViewModel : BindableBase
{
private Action<Action> RunOnUI;
CancellationTokenSource source;
public DelegateCommand StartCommand { get; set; }
private string startButtonText = "START";
public string StartButtonText
{
get { return startButtonText; }
set { SetProperty(ref startButtonText, value); }
}
private string loop1Value ;
public string Loop1Value
{
get { return loop1Value; }
set { SetProperty(ref loop1Value, value); }
}
private string loop2Value;
public string Loop2Value
{
get { return loop2Value; }
set { SetProperty(ref loop2Value, value); }
}
private string loop3Value;
public string Loop3Value
{
get { return loop3Value; }
set { SetProperty(ref loop3Value, value); }
}
public MainWindowViewModel(Action<Action> runOnUI)
{
RunOnUI = runOnUI;
StartCommand = new DelegateCommand(MainHandler);
}
async void MainHandler()
{
if (StartButtonText == "START")
{
StartButtonText = "STOP";
try
{
source = new CancellationTokenSource();
await Task.Run((Action)Loop1);
}
catch (OperationCanceledException)
{
MessageBox.Show("Test was aborted by user");
}
finally
{
StartButtonText = "START";
source = null;
}
}
else
{
source = null;
StartButtonText = "START";
}
}
private void Loop1()
{
for(int i=0;i<100;i++)
{
Loop1Value = "Loop1: " + i.ToString();
Sleep(1000, source.Token);
Loop2();
}
}
private void Loop2()
{
for (int i = 0; i < 100; i++)
{
Loop2Value = "Loop2: " + i.ToString();
Sleep(1000, source.Token);
Loop3();
}
}
private void Loop3()
{
for (int i = 0; i < 100; i++)
{
Loop3Value = "Loop3: " + i.ToString();
Sleep(1000, source.Token);
}
}
private static void Sleep(int millisecondsTimeout, CancellationToken cancellationToken)
{
Task.WaitAny(Task.Delay(millisecondsTimeout, cancellationToken));
}
}
}
Normally I have more advanced code and in Loops i have serial communication (rs232), I simplified code to show my problem.
When I press Stop button I want the thread stop (exit all loops). I tried to throw exception when Stop is pressed, buy setting source=null, but code doesn't go to catch. Why?
Change your loop method to
private void Loop1(CancellationToken token)
{
for(int i=0;i<100;i++)
{
token.ThrowIfCancellationIsRequested();
Loop1Value = "Loop1: " + i.ToString();
Sleep(1000, token.Token);
Loop2(token);
}
}
Do the same with all other loops. Also change your code for stopping to source.Cancel()
I would assume the Thread.Sleep is a stand in for reading a serial-port. Unfortunately SerialPort.Read does not seem to provide a overload with cancellation support. I'm not really faimilliar with serial ports, so yo might want to read Async Serial Port with CancellationToken and ReadTimeout. Or search for better libraries. Or if nothing else works, set a low timeout and catch the timeout exceptions.
I would also recommend to avoid mutable shared state when dealing with multiple threads. In this case it would mean to take the cancellation token and any other inputs as method parameters, and return some result. Avoid using any mutable fields. Updating the Loop1Value from a background thread looks like a very likely bug to me.
If you do not care if the operation was canceled or completed you could instead check the IsCancellationRequested property and simply return.
I want the thread stop
You need to use cooperative cancellation. Trying to stop a thread non-cooperativly is just not a good idea. See What's wrong with using Thread.Abort() for details.
My program works with a queue and a large file. I work with the queue in several threads and at some point there is OutOfMemoryException because of having too many objects Enqueued in the first thread in the queue. They do not manage to Dequeue in the second thread as fast as required.
P.S. I can use just the primitives of synchronyzing (Thread, Monitor)
I've already written a code which works with not very big data. I know that I can do Thread.Sleep (and this is works, I thried) when in my queue exists definite amount of objects. As far as I know it's not the best solution
class SynchronizedQueue
{
protected readonly object locker = new object();
protected Queue<BlockData> queue = new Queue<BlockData>();
public int Counter { get; set; }
public bool IsClose { get; set; }
public bool TryDequeue(out BlockData blockData)
{
lock (locker)
{
while (queue.Count == 0)
{
if (IsClose)
{
blockData = new BlockData();
return false;
}
Monitor.Wait(locker);
}
blockData = queue.Dequeue();
return true;
}
}
public void Close()
{
lock (locker)
{
IsClose = true;
Monitor.PulseAll(locker);
}
}
public void Enqueue(BlockData blockData)
{
lock (locker)
{
//That's what i want to avoid
if (Counter == 1000)
{
Thread.Sleep(240);
}
if (IsClose)
throw new InvalidOperationException("Work was canceled!");
while (blockData.Id != Counter)
Monitor.Wait(locker);
queue.Enqueue(blockData);
Counter++;
Monitor.PulseAll(locker);
}
}
}
What can you recommend for synchronizing Enqueue/Dequeue and avoid OutOfMemoryException?
I write an infinity loop for pulling from queue(RabbitMQ) and processing each pulled item in concurrent threads with limited count on running threads.
Now i want a solution for make a limit in thread execution count.see an example of my loop:
public class ThreadWorker<T>
{
public List<T> _lst;
private int _threadCount;
private int _maxThreadCount;
public ThreadWorker(List<T> lst, int maxThreadCount)
{
_lst = lst;
_maxThreadCount = maxThreadCount;
}
public void Start()
{
var i = 0;
while (i < _lst.Count)
{
i++;
var pull = _lst[i];
Process(pull);
}
}
public void Process(T item)
{
if (_threadCount > _maxThreadCount)
{
//wait any opration be done
// How to wait for one thread?
Interlocked.Decrement(ref _threadCount);
}
var t = new Thread(() => Opration(item));
t.Start();
Interlocked.Increment(ref _threadCount);
}
public void Opration(T item)
{
Console.WriteLine(item.ToString());
}
}
Notice that when i use a semaphore for limitation, Start() method don't wait for all running threads. my loop should after running threads with _maxThreadCount, be wait until release a thread and then push new thread for concurrent processing.
I would use Semaphore this way to control the number of threads:
public class ThreadWorker<T>
{
SemaphoreSlim _sem = null;
List<T> _lst;
public ThreadWorker(List<T> lst, int maxThreadCount)
{
_lst = lst;
_sem = new SemaphoreSlim(maxThreadCount);
}
public void Start()
{
var i = 0;
while (i < _lst.Count)
{
i++;
var pull = _lst[i];
_sem.Wait(); /*****/
Process(pull);
}
}
public void Process(T item)
{
var t = new Thread(() => Opration(item));
t.Start();
}
public void Opration(T item)
{
Console.WriteLine(item.ToString());
_sem.Release(); /*****/
}
}
CODE UPDATED TO REFLECT ANSWER: SAME PROBLEM STILL OCCURS
This class is supposed to run all tasks in the list, sleep and then wake up and repeat the process infinitely. For some reason though, after the first sleep, the sleepThread.RunWorkerAsync() call gets called twice for some reason. I can obviously solve this by:
if (!sleepThread.IsBusy) { sleepThread.RunWorkerAsync(); }
but that feels like a work around.
Here is the main routine class:
public class ServiceRoutine
{
private static volatile ServiceRoutine instance;
private static object instanceLock = new object();
private static object listLock = new object();
private static readonly List<Task> taskList = new List<Task>()
{
new UpdateWaferQueueTask(),
new UpdateCommentsTask(),
new UpdateFromTestDataTask(),
new UpdateFromTestStationLogsTask(),
new UpdateFromWatchmanLogsTask(),
new UpdateStationsStatusTask()
};
private List<Task> runningTasks;
private BackgroundWorker sleepThread;
private Logger log;
private ServiceRoutine()
{
log = new Logger();
runningTasks = new List<Task>();
sleepThread = new BackgroundWorker();
sleepThread.WorkerReportsProgress = false;
sleepThread.WorkerSupportsCancellation = false;
sleepThread.DoWork += (sender, e) =>
{
int sleepTime = ConfigReader.Instance.GetSleepTime();
log.Log(Logger.LogType.Info,
"service sleeping for " + sleepTime / 1000 + " seconds");
Thread.Sleep(sleepTime);
};
sleepThread.RunWorkerCompleted += (sender, e) => { Run(); };
}
public static ServiceRoutine Instance
{
get
{
if (instance == null)
{
lock (instanceLock)
{
if (instance == null)
{
instance = new ServiceRoutine();
}
}
}
return instance;
}
}
public void Run()
{
foreach (Task task in taskList)
{
lock (listLock)
{
runningTasks.Add(task);
task.TaskComplete += (completedTask) =>
{
runningTasks.Remove(completedTask);
if (runningTasks.Count <= 0)
{
sleepThread.RunWorkerAsync();
}
};
task.Execute();
}
}
}
}
this is called like this:
ServiceRoutine.Instance.Run();
from the service start method. Here is the Task class as well:
public abstract class Task
{
private Logger log;
protected BackgroundWorker thread;
public delegate void TaskPointer(Task task);
public TaskPointer TaskComplete;
public Task()
{
log = new Logger();
thread = new BackgroundWorker();
thread.WorkerReportsProgress = false;
thread.DoWork += WorkLoad;
thread.RunWorkerCompleted += FinalizeTask;
}
protected abstract string Name { get; }
protected abstract void WorkLoad(object sender, DoWorkEventArgs e);
private string GetInnerMostException(Exception ex)
{
string innerMostExceptionMessage = string.Empty;
if (ex.InnerException == null) { innerMostExceptionMessage = ex.Message; }
else
{
while (ex.InnerException != null)
{
innerMostExceptionMessage = ex.InnerException.Message;
}
}
return innerMostExceptionMessage;
}
protected void FinalizeTask(object sender, RunWorkerCompletedEventArgs e)
{
try
{
if (e.Error != null)
{
string errorMessage = GetInnerMostException(e.Error);
log.Log(Logger.LogType.Error, this.Name + " failed: " + errorMessage);
}
else
{
log.Log(Logger.LogType.Info, "command complete: " + this.Name);
}
}
catch (Exception ex)
{
string errorMessage = GetInnerMostException(ex);
log.Log(Logger.LogType.Error, this.Name + " failed: " + errorMessage);
}
finally { TaskComplete(this); }
}
public void Execute()
{
log.Log(Logger.LogType.Info, "starting: " + this.Name);
thread.RunWorkerAsync();
}
}
The question is, why is sleepThread.RunWorkerAsync() getting called twice and is there a better way to get this work without checking if the thread is busy before calling it?
You are facing a race condition here. The problem is in the TaskComplete callback. Last two tasks remove themselves from the runningTasks list before executing the if condition. When it is executed, the list count is zero. You should lock the list before changing its. The lock needs to be taken in the TaskComplete callback:
runningTasks.Add(task);
task.TaskComplete += (completedTask) =>
{
lock (runningTasks)
{
runningTasks.Remove(completedTask);
if (runningTasks.Count <= 0)
{
sleepThread.RunWorkerAsync();
}
}
};
task.Execute();
SOLVED
I tried several different locking techniques on the runningTasks list but nothing worked. After changing runningTasks to a BlockingCollection, everything worked perfectly.
Here is the new add/remove implementation using a BlockingCollection instead of a List:
foreach (Task task in taskList)
{
runningTasks.Add(task);
task.TaskComplete += (completedTask) =>
{
runningTasks.TryTake(out completedTask);
if (runningTasks.Count <= 0 && completedTask != null)
{
sleepThread.RunWorkerAsync();
}
};
task.Execute();
}
I want to know how to stop and restart a thread.
I create N amount of threads, depending on conditions returned from a database. These are long running processes which should never stop but should I get a critical error within the thread I want to completely kill the thread and start it up like new.
The code which I use currently to start the threads:
foreach (MobileAccounts MobileAccount in ReceiverAccounts)
{
Receiver rec = new Receiver();
ThreadStart starterParameters = delegate { rec.StartListener(MobileAccount); };
Thread FeedbackThread = new Thread(starterParameters);
FeedbackThread.Name = MobileAccount.FriendlyName;
FeedbackThread.Start();
Thread.Sleep(1000);
}
You can write your own listener and manage its thread within it.
something like:
public class AccountListener
{
private Thread _worker = null;
private MobileAccount _mobileAccount;
public AccountListener(MobileAccount mobileAccount)
{
_mobileAccount = mobileAccount;
}
protected void Listen()
{
try
{
DoWork();
}
catch (Exception exc)
{
}
}
protected virtual void DoWork()
{
Console.WriteLine(_mobileAccount);
}
public void Start()
{
if (_worker == null)
{
_worker = new Thread(Listen);
}
_worker.Start();
}
public void Stop()
{
try
{
_worker.Abort();
}
catch (Exception)
{
//thrad abort exception
}
finally
{
_worker = null;
}
}
}