I am investigating the design of a work queue processor where the QueueProcessor retrieves a Command Pattern object from the Queue and executes it in a new thread.
I am trying to get my head around a potential Queue lockup scenario where nested Commands may result in a deadlock.
E.G.
A FooCommand object is placed onto the queue which the QueueProcessor then executes in its own thread.
The executing FooCommand places a BarCommand onto the queue.
Assuming that the maximum allowed threads was only 1 thread, the QueueProcessor would be in a deadlocked state since the FooCommand is infinitely waiting for the BarCommand to complete.
How can this situation be managed? Is a queue object the right object for the job? Are there any checks and balances that can be put into place to resolve this issue?
Many thanks. ( application uses C# .NET 3.0 )
You could redesign things so that FooCommand doesn't use the queue to run BarCommand but runs it directly, or you could split FooCommand into two, and have the first half stop immediately after queueing BarCommand, and have BarCommand queue the second have of FooCommand after it's done its work.
Queuing implicitly assumes an asynchronous execution model. By waiting for the command to exit, you are working synchronously.
Maybe you can split up the commands in three parts: FooCommand1 that executes until the BarCommand has to be sent, BarCommand and finally FooCommand2 that continues after BarCommand has finished. These three commands can be queued separately. Of course, BarCommand should make sure that FooCommand2 is queued.
For simple cases like this an additional monitoring thread that can spin off more threads on demand is helpful.
Basically every N seconds check to see if any jobs have been finished, if not, add another thread.
This won't necessarily handle even more complex deadlock problems, but it will solve this one.
My recommendation for the heavier problem is to restrict waits to newly spawned process, in other words, you can only wait on something you started, that way you never get deadlocks, since cycles are impossible in that situation.
If you are building the Queue object yourself there are a few things you can try:
Dynamically add new service threads. Use a timer and add a thread if the available thread count has been zero for too long.
If a command is trying to queue another command and wait for the result then you should synchronously execute the second command in the same thread. If the first thread simply waits for the second you won't get a concurrency benefit anyway.
I assume you want to queue BarCommand so it is able to run in parallel with FooCommand, but BarCommand will need the result at some later point. If this is the case then I would recommend using Future from the Parallel Extensions library.
Bart DeSmet has a good blog entry on this. Basically you want to do:
public void FooCommand()
{
Future<int> BarFuture = new Future<int>( () => BarCommand() );
// Do Foo's Processing - Bar will (may) be running in parallel
int barResult = BarFuture.Value;
// More processing that needs barResult
}
With libararies such as the Parallel Extensions I'd avoid "rolling your own" scheduling.
Related
I've a c# single threaded application and currently working on to make it multi-threaded with the use of thread pools. I am stuck in deciding which model would work for my problem.
Here's my current scenario
While(1)
{
do_sometask();
wait(time);
}
And this is repeated almost forever. The new scenario has multiple threads which does the above. I could easily implement it by spawning number of threads based on the tasks I have to perform, where all the threads perform some task and wait forever.
The issue here is I may not know the number of tasks, so I can't just blindly spawn 500 threads. I thought about using threadpool, but because almost every thread loops forever and won't ever be freed up for new tasks in the queue, am not sure which other model to use.
I am looking for an idea or solution where I could break the loop in the thread and free it up instead of waiting, but come back and resume the same task after the wait(when the time gets elapsed, using something like a timer/checking timestamp of when the last task is performed).
With this I could use a limited number of threads (like in a thread pool) and serve the tasks which are coming in during the time old threads waits(virtually).
Any help is really appreciated.
If all you have is a bunch of things that happen periodically, it sounds what you want is a bunch of timers. Create a timer for each task, to fire when appropriate. So if you have two different tasks:
using System.Threading;
// Task1 happens once per minute
Timer task1Timer = new Timer(
s => DoTask1(),
null,
TimeSpan.FromMinutes(1),
TimeSpan.FromMinutes(1));
// Task2 happens once every 47 seconds
Timer task2Timer = new Timer(
s => DoTask2(),
null,
TimeSpan.FromSeconds(47),
TimeSpan.FromSeconds(47);
The timer is a pretty lightweight object, so having a whole bunch of them isn't really a problem. The timer only takes CPU resources when it fires. The callback method will be executed on a pool thread.
There is one potential problem. If you have a whole lot of timers all with the same period, then the callbacks will all be called at the same time. The threadpool should handle that gracefully by limiting the number of concurrent tasks, but I can't say for sure. But if your wait times are staggered, this is going to work well.
If you have small wait times (less than a second), then you probably need a different technique. I'll detail that if required.
With this design, you only have one thread blocked at any time.
Have one thread (the master thread) waiting on a concurrent blocking collection, such as the BlockingCollection. This thread will be blocked by a call to TryTake until something is placed in the collection, or after a certain amount of time has passed via a timeout passed into the call (more on this later).
Once it is unblocked, it may have a unit of work to be processed. It checks to see if there is one (i.e., the TryTake call didn't time out), then if there is capacity to perform this work, and if so, queues up a thread (pool, Task or whatevs) to service the work. This master thread then goes back to the blocking collection and tries to take another unit of work. The cycle continues.
As a unit of work is begun, it will be noted so that the main thread can see how many threads are working. Once this unit is completed, the notation will be removed. The thread is then freed.
You want to use a timeout so that if it is judged that too many operations are running concurrently, you will be able to re-evaluate this a set period of time down the road. Otherwise, that unit of work sits in the blocking collection until a new unit is added, which is not optimal.
Outside users of this instance can queue up new units of work by simply dropping them in the collection.
You can use a cancellation token to immediately unblock the thread when it's time to shut down operations. Have the worker operations take cancellation tokens as well so they can halt on shutdown.
I could implement it with the help of a threadpool and few conditions to check the last activity of the task before adding it to the threadpool queue.
I'm starting multiple threads and would like to know when any of then finishes. I know the following code:
foreach (Thread t in threads)
t.Join();
But it will only wait for all threads together. That's much too late. I need to know when one thread finishes, even when other threads are still running. I'm looking for something equivalent to WaitAny only for threads. But I can't add code to all threads I'm monitoring, so using signals or other synchronisation objects is not an option.
Some clarification: I'm working on a logging/tracing tool that should log the application's activity. I can insert log statements when a thread starts, but I can't insert a log statement on every possible way out of the thread (multiple exit points, exceptions etc.). So I'd like to register the new thread and then be notified when it finishes to write a log entry. I could asynchronously Join on every thread, but that means a second thread for every monitored thread which may seem a bit much overhead. Threads are used by various means, be it a BackgroundWorker, Task or pool thread. In its essence, it's a thread and I'd like to know when it's done. The exact thread mechanism is defined by the application, not the logging solution.
Instead of Threads use Tasks. It has the method WaitAny.
Task.WaitAny
As you can read here,
More efficient and more scalable use of system resources.
More programmatic control than is possible with a thread or work item.
In my opinion WaitHandle.WaitAny is the best solution, since you don't like to use it for some xyz reason you can try something like this.
Take the advantage of Thread.Join(int) method which takes millisecond timeout and returns true when thread is terminated or false when timed out.
List<Thread> threads = new List<Thread>();
while (!threads.Any(x=> x.Join(100)))
{
}
You can alter the timeout of Join If you know how long it will take.
My answer is based on your clarification that all you have is Thread.Current. Disclaimer: IMO, what you're trying to do is a hack, thus my idea by all means is a hack too.
So, use reflection to obtain the set of native Win32 handles for your desired threads. Your are looking for Thread.GetNativeHandle method which is internal, so you call it like thread.GetType().InvokeMember("GetNativeHandle", BindingFlags.InvokeMethod | BindingFlags.Instance | BindingFlags.NonPublic, ...). Use a reflection tool of your choice or Framework sources to learn more about it. Once you've got the handles, go on with one of the following options:
Set up your own implementation of SynchronizationContext (derive from it) and use SynchronizationContext.WaitHelper(waitAll: false) to wait for your unmanaged handles.
Use the raw Win32 API like WaitForMultipleObjects or CoWaitForMultipleObjects (depending on whether you need to pump messages).
Perform the wait on a separate child or pool thread.
[EDITED] Depending on the execution environment of your target threads, this hack may not work, because one-to-one mapping between managed and unmanaged threads is not guaranteed:
It is possible to determine the Windows thread that is executing the code for a managed thread and to retrieve its handle. However, it still doesn't make sense to call the SetThreadAffinityMask function for this Windows thread, because the managed scheduler can continue the execution of a managed thread in another Windows thread.
It appears however, this may be an implication only for custom CLR hosts. Also, it appears to be possible to control managed thread affinity with Thread.BeginThreadAffinity and Thread.EndThreadAffinity.
You could use a background worker for your working threads.
Then hook all the RunWorkerCompleted events to a method that will wait for them.
If you want that to be synched to the code where you're currently waiting for the join, then the problem is reduced to just synchronizing that single event method to that place in code.
Better yet, I'd suggest to do what you're doing asynchronously without blocking, and just do what you want in the event.
Would you consider wrapping your thread invocations with another 'logging' thread? That way you could log synchronously before & after the thread run.
Something like this pseudo-code:
int threadLogger(<parms>) {
log("starting thread");
retcode = ActualThreadBody(<parms>);
log("exiting thread");
return retcode;
}
If you have more information on the thread started, you could log that as well.
You could also take the thread function as a parameter in the case where you have multiple types of threads to start, which it sounds like you do.
I am writing a Windows Service that receives messages/requests and executes them asynchronously - it does not need to wait for the item to complete, nor care about the result. I am successfully able to execute the requests as Tasks using System.Threading.Tasks.Task. Most of these items execute quickly (less than a second), but some take longer (2-3 minutes).
As a windows service, I need to respond to the "Stop" command and some of the Tasks will still be running. It is preferable to not Cancel the Tasks as the longer running ones might leave data in a bad state (and a rollback is very tricky).
What is the best way to handle this? I thought of keeping a List of the tasks that I have started so that I can do a WaitAll. During the execution of the service it will process tens of thousands of requests. How would I know when to remove completed Tasks from the List so the List doesn't grow wildly? I don't think I should be holding references to that many Task objects.
Thanks in advance.
You can use CancellationToken for this purposes.
Once OnStop event occured, you just call method Cancel() on the CancellationTokenSource and it will be propagated to all tasks that you passed the token in.
There are several techniques how to correctly cancel a task.
You may explicitly check from time to time inside the task if cancellation requested.
Or I beleive there is a property ThrowWhenCancelled on a token itself, and if cancellation has been requested token will throw an CancellationException.
If you don't care about the task results, but want to have a tracking list, just keep the List<Task<T>> (or may be ConcurrentBag<Task<T>>)as a local variable. From time to time start another task that will go through the list and check the Task.Status property, if it's running or else.
I don't think keeping that many references should be an issue as long as you maintain those references correctly.
Also, it depends how do you want to stop your application. If you are fine with the task being killed along with the application you may not track them (unless they hold some resources to hold, and you need to free them) at all. But in most of the cases I would say it should be correctly finalized.
EDIT: just reread your post. RequestAdditionalTime call may help you to wait until long runnning tasks are finished.
Check it on MSDN: ServiceBase.RequestAdditionalTime Method
If you only care about tasks finishing before your service terminates I would suggest using Thread instead of Task.
new Thread(WorkerMethod).Start();
Thread created this way is a so called foreground thread and your application (service) will not end until all foreground threads end. You need to make sure all your foreground threads do not hang under any condition otherwise your application will never terminate by itself. You can achieve the same thing with Task but you would need to keep list of all tasks you have run and use Task.WaitAll to wait for all of them to finish in your Stop event.
If you need to control your threads (i.e. keep reference to them) you need to use some sort of collection.
List<Thread> threads = new List<Thread>();
Thread thrd;
threads.Add(thrd = new Thread(WorkerMethod));
thrd.Start();
But if you actually need to control and cancel you tasks/threads you should rather go with Task which makes cancellation easier.
I've run into a problem while writing an async multi-server network app in c#. I have many jobs being taken care of by the thread pool and these include the writes to the network sockets. This ended up allowing for the case where more than one thread could write to the socket at the same time and discombobulate my outgoing messages. My idea for getting around this was to implement a queue system where whenever data got added to the queue, the socket would write it.
My problem is, I can't quite wrap my head around the architecture of something of this nature. I imagine having a queue object that fires an event on whenever data gets added to the queue. The event then writes the data being held in the queue, but that won't work because if two threads come by and add to the queue simultaneously, even if the queue is made to be thread safe, events will still be fired for both and I'll run into the same problem. So then maybe someway to hold off an event if another is in progress, but then how do I continue that event once the first finishes without simply blocking the thread on some mutex or something. This wouldn't be so hard if I wasn't trying to stay strict with my "block nothing" architecture but this particular application requires that I allow the thread pool threads to keep doing their thing.
Any ideas?
While similar to Porges answer it differs a bit in implementation.
First, I usually don't queue the bytes to send, but objects and seralize them in the sending thread but I guess that's a matter of taste.
But the bigger difference is in the use of ConcurrentQueues (in addition to the BlockingCollection).
So I'd end up with code similar to
BlockingCollection<Packet> sendQueue = new BlockingCollection<Packet>(new ConcurrentQueue<Packet>());
while (true)
{
var packet = sendQueue.Take(); //this blocks if there are no items in the queue.
SendPacket(packet); //Send your packet here.
}
The key-take away here is that you have one thread which loops this code, and all other threads can add to the queue in a thread-safe way (both, BlockingCollection and ConcurrentQueue are thread-safe)
have a look at Processing a queue of items asynchronously in C# where I answered a similar question.
Sounds like you need one thread writing to the socket synchronously and a bunch of threads writing to a queue for that thread to process.
You can use a blocking collection (BlockingCollection<T>) to do the hard work:
// somewhere there is a queue:
BlockingCollection<byte[]> queue = new BlockingCollection<byte[]>();
// in socket-writing thread, read from the queue and send the messages:
foreach (byte[] message in queue.GetConsumingEnumerable())
{
// just an example... obviously you'd need error handling and stuff here
socket.Send(message);
}
// in the other threads, just enqueue messages to be sent:
queue.Add(someMessage);
The BlockingCollection will handle all synchronization. You can also enforce a maximum queue length and other fun things.
I don't know C#, but what I would do is have the event trigger the socket manager to start pulling from the queue and write things out one at a time. If it is already going the trigger won't do anything, and once there is nothing in the queue, it stops.
This solves the problem of two threads writing to the queue simultaneously because the second event would be a no-op.
You could have a thread-safe queue that all your worker thread write their results to. Then have another thread that polls the queue and sends results when it sees them waiting.
I'm creating a webservice+servicebus project where user can do something like
public void ExecuteLongProcess(DateTime fromDate,string aggregateId){}
This method immediately returns but send over the bus a request for the operation.
My problems starts when multiple user ask for the long process over the same aggregateId when another one is already running.
The solution i'm thinking about is a Task that runs continuosly and look in a Queue<LongProcessTask> for a operation that must be executed so I run only one process a time or a future implementation will be multiple process if different aggregateId.
This way I don't overlap long running process over the same aggregate.
Other ideas?
I have created a TaskRunner that instantiate some continuous running Task (number depending on processor cores) that look in a concurrent queue and run each operation pending.
TaskRunner get from Windsor the handler for each operation type in order to leave the processing of each operation in a class aside.
In your answer, you say that multiple threads will take tasks from a concurrent queue. In this case there is a chance that two tasks with the same aggregateId may run at the sametime. I do not know whether this is a problem for you, if so then you must use a different queue for each aggregateId.
If the task order is not an issue then I recommend using a BlockingCollection. Because there is question : What are you planning to do with multiple consumer threads if there is no task in the concurrent queue.
while(some_condition_to_keep_thread_alive)
{
if(!queue.TryDequeue(...))
continue;
else
{
//do the job
}
}
This code will make your cores go crazy if queue is empty. You need a blocking mechanism.
BlockingCollection will do this for you.
Do you insist on using ConcurrentQueue? Ok SemaphoreSlim is your friend.