My team is developing a multi-threaded application using async/await in C# 5.0. In the process of implementing thread synchronization, after several iterations, we came up with a (possibly novel?) new SynchronizationContext implementation with an internal lock that:
When calling Post:
if a lock can be taken, the delegate is executed immediately
if a lock cannot be taken, the delegate is queued
When calling Send:
if a lock can be taken the delegate is executed
if a lock cannot be taken, the thread is blocked
In all cases, before executing the delegate, the context sets itself as the current context and restores the original context when the delegate returns.
It’s an unusual pattern and since we’re clearly not the first people writing such an application I’m wondering:
Is the pattern really safe?
Are there better ways of achieving thread synchronization?
Here’s the source for SerializingSynchronizationContext and a demo on GitHub.
Here’s how it’s used:
Each class wanting protection creates its own instance of the context like a mutex.
The context is awaitable so that statements like the following are possible.
await myContext;
This simply causes the rest of the method to be run under protection of the context.
All methods and properties of the class use this pattern to protect data. Between awaits, only one thread can run on the context at a time and so state will remain consistent. When an await is reached, the next scheduled thread is allowed to run on the context.
The custom SynchronizationContext can be used in combination with AsyncLock if needed to maintain atomicity even with awaited expressions.
Synchronous methods of a class can use custom context for protection as well.
Having a sync context that never runs more than one operation at a time is certainly not novel, and also not bad at all. Here you can see Stephen Toub describing how to make one two years ago. (In this case it's used simply as a tool to create a message pump, which actually sounds like it might be exactly what you want, but even if it's not, you could pull the sync context out of the solution and use it separately.)
It of course makes perfect conceptual sense to have a single threaded synchronization context. All of the sync contexts representing UI states are like this. The winforms, WPF, winphone, etc. sync contexts all ensure that only a single operation from that context is ever running at one time.
The one worrying bit is this:
In all cases, before executing the delegate, the context sets itself as the current context and restores the original context when the delegate returns.
I'd say that the context itself shouldn't be doing this. If the caller wants this sync context to be the current context they can set it. If they want to use it for something other than the current context you should allow them to do so. Sometimes you want to use a sync context without setting it as the current one to synchronize access to a certain resource; in such a case only operation specifically accessing that resource would need to use this context.
Regarding the use of locks. This question would be more appropriate for Code Review, but from the first glance I don't think your SerializingSynchronizationContext.Post is doing all right. Try calling it on a tight loop. Because of the Task.Run((Action)ProcessQueue), you'll quickly end up with more and more ThreadPool threads being blocked on lock (_lock) while waiting to acquire it inside ProcessQueue().
[EDITED] To address the comment, here is your current implementation:
public override void Post(SendOrPostCallback d, object state)
{
_queue.Enqueue(new CallbackInfo(d, state));
bool lockTaken = false;
try
{
Monitor.TryEnter(_lock, ref lockTaken);
if (lockTaken)
{
ProcessQueue();
}
else
{
Task.Run((Action)ProcessQueue);
}
}
finally
{
if (lockTaken)
{
Monitor.Exit(_lock);
}
}
}
// ...
private void ProcessQueue()
{
if (!_queue.IsEmpty)
{
lock (_lock)
{
var outer = SynchronizationContext.Current;
try
{
SynchronizationContext.SetSynchronizationContext(this);
CallbackInfo callback;
while (_queue.TryDequeue(out callback))
{
try
{
callback.D(callback.State);
}
catch (Exception e)
{
Console.WriteLine("Exception in posted callback on {0}: {1}",
GetType().FullName, e);
}
}
}
finally
{
SynchronizationContext.SetSynchronizationContext(outer);
}
}
}
}
In Post, why enqueue a callback with _queue.Enqueue and then preoccupy a new thread from the pool with Task.Run((Action)ProcessQueue), in the situation when ProcessQueue() is already pumping the _queue in a loop on another pool thread and dispatching callbacks? In this case, Task.Run looks like wasting a pool thread to me.
Related
I've just started using Nito.AsyncEx package and AsyncLock instead of a normal lock() { ... } section where I have async calls within the locked section (since you can't use lock() in such cases for good reasons I've just read about). This is within a job that I'm running from Hangfire. Let's call this the 'worker' thread.
In another thread, from an ASP.NET controller, I'd like to check if there's a thread that's currently executing within the locked section. If there's no thread in the locked section then I'll schedule a background job via Hangfire. If there's already a thread in the locked section then I don't want to schedule another one. (Yes, this might sound a little weird, but that's another story).
Is there a way to check this using the Nito.AsyncEx objects, or should I just set a flag at the start of the locked section and unset it at the end?
e.g. I'd like to this:
public async Task DoAJobInTheBackground(string queueName, int someParam)
{
// do other stuff...
// Ensure I'm the only job in this section
using (await _asyncLock.LockAsync())
{
await _aService.CallSomethingAsync());
}
// do other stuff...
}
and from a service called by a controller use my imaginary method IsSomeoneInThereNow():
public void ScheduleAJobUnlessOneIsRunning(string queueName, int someParam)
{
if (!_asyncLock.IsSomeoneInThereNow())
{
_backgroundJobClient.Enqueue<MyJob>(x =>
x.DoAJobInTheBackground(queueName, someParam));
}
}
but so far I can only see how to do this with a separate variable (imagining _isAnybodyInHere is a thread-safe bool or I used Interlocked instead):
public async Task DoAJobInTheBackground(string queueName, int someParam)
{
// do other stuff...
// Ensure I'm the only job in this section
using (await _asyncLock.LockAsync())
{
try
{
_isAnybodyInHere = true;
await _aService.CallSomethingAsync());
}
finally
{
_isAnybodyInHere = false;
}
}
// do other stuff...
}
and from a service called by a controller:
public void ScheduleAJobUnlessOneIsRunning(string queueName, int someParam)
{
if (!_isAnybodyInHere)
{
_backgroundJobClient.Enqueue<MyJob>(x =>
x.DoAJobInTheBackground(queueName, someParam));
}
}
Really it feels like there should be a better way. The AsyncLock doc says:
You can call Lock or LockAsync with an already-cancelled CancellationToken
to attempt to acquire the AsyncLock immediately
without actually entering the wait queue.
but I don't understand how to do that, at least using the synchronous Lock method.
I don't understand how to do that
You can create a new CancellationToken and pass true to create one that is already canceled:
using (_asyncLock.Lock(new CancellationToken(canceled: true)))
{
...
}
The call to Lock will throw if the lock is already held.
That said, I don't think this is a good solution to your problem. There's always the possibility that the background job is just about to finish, the controller checks the lock and determines it's held, and then the background job releases the lock. In that case, the controller will not trigger a background job.
You must never(!) make any assumptions about any other thread or process!
What you must instead do, in this particular example, is to "schedule another job," unless you have already done so. (To avoid "fork bombs.") Then, the job, once it actually begins executing, must decide: "Should I be doing this?" If not, the job quietly exits.
Or – perhaps the actual question here is: "Has somebody else already ≤scheduled_this_job≥?"
What prevents me from constantly using the async/wait pattern in my .net code is that once you create an async method the async keyword tends to spread thru my code forcing me to make all methods async. Is there a pattern to stop this efficiently?
What prevents me from constantly using the async/wait pattern in my .net code is that once you create an async method the async keyword tends to spread thru my code forcing me to make all methods async. Is there a pattern to stop this efficiently?
Let me ask you this, then: why are you using async/await? You need to take a step back and decide if you want the benefits of asynchronous code. If you do want those benefits, then your code must be asynchronous. As soon as you block a thread, you lose all the benefits of async. So really, why use asynchronous code at all, if you're just going to block a thread anyway?
That said, there are some scenarios where a partially-async stack makes sense. For example, if your code base is temporarily in a transition state. In this case, you can use one of the hacks described in my article on brownfield async:
Blocking directly (may cause deadlocks).
Blocking on a thread pool thread (executes code on a different thread and different context, may cause unexpected parallelism).
Blocking on the current thread with a thread pool context (executes code in a different context, may cause unexpected parallelism).
Blocking on a thread with a single-threaded context (executes code on a different thread and different context).
Blocking on a nested message loop (may cause unexpected reentrancy).
All of these are hacks, and all have different drawbacks. There is no hack that works in every scenario.
As commented earlier you should go async all the way...
In some cases we use a litle helper class to cut off the async pattern. This class alows you to wait for the result without ending up with a deadlock by replacing the SynchronizationContext.
public class NoSynchronizationContextScope : IDisposable
{
private readonly SynchronizationContext synchronizationContext;
public NoSynchronizationContextScope()
{
synchronizationContext = SynchronizationContext.Current;
SynchronizationContext.SetSynchronizationContext(null);
}
public void Dispose() => SynchronizationContext.SetSynchronizationContext(synchronizationContext);
}
void fn()
{
using (new NoSynchronizationContextScope())
{
fnAsync().Wait();
// or
// var result = fnAsync().Result();
// in case you have to wait for a result
}
}
In the very particular case of a method that you want to start running but not wait for the result of (i.e. triggering an event handler) you can use async void. Just make sure you catch all the errors at that point:
public async void DoSomething() {
try {
var o = await DoThing1();
await DoThing2(o);
} catch (Exception e) {
// Log the error somewhere (to a file, database, etc.)
}
}
You can use async method and invoke not async method in task so outer method will behave like async and inner method is still sync.
public async Task OuterFuncAsync()
{
await Task.Run(() => InnerFunc());
}
I have created a semaphore instance on top of this class
public static SemaphoreSlim _zReportSemaphore = new SemaphoreSlim(1, 500);
And somewhere in my code i need to retrieve and send some data.
while (_isRunning)
{
try
{
xBsonDocument = null;
//I think its very clear in this line...
MongoDBDAO.xGetInstance().GetZReportData(ref xBsonDocument);
foreach (BsonDocument item in xBsonDocument)
{
try
{
ThreadObject xThreadObject = new ThreadObject();
xThreadObject.m_strTerminalId = item.GetValue("_id")["TERMINAL_ID"].ToString();
xThreadObject.m_strZNo = item.GetValue("_id")["Z_NO"].ToString();
m_xBuildAndSendZReportThread =
new Thread(new ParameterizedThreadStart(vBuildAndSendZReport));
m_xBuildAndSendZReportThread.Start(xThreadObject);
}
catch (Exception xException)
{
xException.TraceError();
continue;
}
Thread.Sleep(m_litleStepQTime);
}
}
catch (Exception xException)
{
Thread.Sleep(m_bigStepQTime);
Trace.vInsertError(xException);
continue;
}
Thread.Sleep(m_iSleepTime);
}
This thread targeting to send files to ftp
private void vBuildAndSendZReport(object prm_objParameters)
{
_zReportSemaphore.Wait();
RetriveDataFromMongoAndSend();
_zReportSemaphore.Release();
}
In this structure; if i don't use a semaphore it has working great but sometimes thread count overloading the CPU or Memory usage and machine has been crushing.
1- How can i provide control over data usage (ballancing, isolating threads etc.) with this slim semaphore?
2- Can I use SemaphoreSlim for this type of job in production? What can be the advantages and disadvantages of using such a workflow organization like this?
Does it improve performance? in my special case
3- Is there another alternative that will provide system resource management and will wrap up the technical exception management
Update:
I asked this question during a job I did a long time ago. After solving the problem, I realized that I did not return.
In the above example, the report sending job was happening in the file sharing environment. Other solutions are possible, such as using a CDN.
The question was: Why should I use a thread if it can't keep me informed about what it's doing, if it doesn't tell me if it has had successful results? Why should I use SemaphoreSlim for example!?
yes, of course it can be done with async programming. but I didn't want to include this library in related environment. It had to be. I'm sure this situation is needed in many codes.
my solution was this: I eliminated the possibility of the exception in the code that was throwing the exception. so i synced the conflict with the thread outside the app. I made something like a threadpool. It worked flawlessly as a consumer. I did this by setting up a custom timing mechanism.
Regardless, I still agree. A thread should be set up to carry information about the job it is doing. I'm not talking about writing a Mutex object in between. Thread itself can carry this information.
By the way, I gave points to those who answered. Because they made the right comments according the question.
This is the first hit on Google for "Semaphore and SemaphoreSlim usage Best Practices", so I would like to add 1 remark:
At least, this code
semaphore.Wait();
DoSomeThing();
semaphore.Release();
should be at the minimum
semaphore.Wait();
try
{
DoSomeThing();
}
finally
{
semaphore.Release();
}
Or else you might end up in NEVER releasing the semaphore again if an exceptions occurs in DoSomeThing...
And in async programming, consider using
await semaphore.WaitAsync();
Is there any event when semaphore ends its all threads
No. It's not even clear what that might mean. For example, what do you want to happen if, due to thread-scheduling issues, you have just one running thread in the semaphore at the moment, and that thread completes, releasing the semaphore, before one or more other threads even get to try to acquire the semaphore?
The semaphore has no way to detect this condition as different from every thread being done.
If you want to know when some collection of asynchronous operations has completed, you'll need to wait on that specifically. You have a number of options in .NET, including:
Call Thread.Join() on all of the thread objects you've started.
Use Task to run your asynchronous tasks instead of Thread, and use Task.WhenAll() (or less preferably, Task.WaitAll()) to wait for them to complete.
Use CountdownEvent. Call AddCount() for each task you start, have each task call Signal() when it's done, and then wait on the CountdownEvent.
By the way, the code you posted is suspect in other ways:
Why are you specifying a maximum count for the SemaphoreSlim, and why is this maximum not the same as your initial count? Do you actually expect to call Release() more often than you call Wait()?
Code that calls Thread.Sleep() is often incorrect. It's not clear why you are doing that, but it's likely there are better ways to solve whatever issue you're trying to address with those calls.
Without a good Minimal, Complete, and Verifiable example, I can't say for sure that those things are wrong. But there's a low likelihood of them being right. :)
I have a class which has an inner state which can be changed.
These state changes are never simple, and often consist of several asynchronous operations which occur across multiple threads, such as opening a connection and sending some data
By using a lock and a boolean to indicate whether the state is currently changing I can ensure that only one operation can ever have access to the state at any given time
lock (thisLock) {
while (stateChanging)
Monitor.Wait(thisLock);
stateChanging= true;
//now free to go away and do other things while maintaining exclusive access to the inner state
}
This works fine, but it means there is needless blocking occurring in threads waiting to get exclusive access the state
So what I envision is a lock based on callbacks, where a state changing operation does something like this -
sharedLock.ObtainLock(delegate() {
//we now have exclusive access to the state
//do some work
socket = new Socket();
socket.BeginConnect(hostname, connectcallback);
});
void connectcallback(IAsyncResult result) {
socket.EndConnect(result);
isConnected = true;
sharedLock.ReleaseLock();
}
Is such a concept common? Does it have a name? Am I approaching things incorrectly?
I ended up creating an asynchronous semaphore and it works really really well without feeling hacky in any way.
Usually you would use Mutex or Semaphores for this purpose. For example, if a semaphore has just one token and one operation has taken that token, no other operation can be executed until the first operation is finished and the token has been put back to the semaphore.
Within your second code example you just called ObtainLock and ReleaseLock, but then the sharedLock does not know which operation called Obtain/Release. This is why ObtainLock usually returns a token that can either be diposed or released when the operation finished.
IDisposable myLock;
myLock = sharedLock.ObtainLock(delegate() {
socket = new Socket();
socket.BeginConnect(hostname, connectcallback);
});
void connectcallback(IAsyncResult result) {
socket.EndConnect(result);
isConnected = true;
myLock.Dispose();
}
The class that implements you sharedLock manages these tokens and according to the state of each token it knows if it's busy or not. Actually nothing more than a reference counter.
As another alternative you might use ManualResetEvent or AutoResetEvents as tokens that you return on ObtainLock. When your operation is finished, just call event.Set()
I have a class that implements the Begin/End Invocation pattern where I initially used ThreadPool.QueueUserWorkItem() to thread my work. The work done on the thread doesn't loop but does takes a bit of time to process so the work itself is not easily stopped.
I now have the side effect where someone using my class is calling the Begin (with callback) a ton of times to do a lot of processing so ThreadPool.QueueUserWorkItem is creating a ton of threads to do the processing. That in itself isn't bad but there are instances where they want to abandon the processing and start a new process but they are forced to wait for their first request to finish.
Since ThreadPool.QueueUseWorkItem() doesn't allow me to cancel the threads I am trying to come up with a better way to queue up the work and maybe use an explicit FlushQueue() method in my class to allow the caller to abandon work in my queue.
Anyone have any suggestion on a threading pattern that fits my needs?
Edit: I'm currently targeting the 2.0 framework. I'm currently thinking that a Consumer/Producer queue might work. Does anyone have thoughts on the idea of flushing the queue?
Edit 2 Problem Clarification:
Since I'm using the Begin/End pattern in my class every time the caller uses the Begin with callback I create a whole new thread on the thread pool. This call does a very small amount of processing and is not where I want to cancel. It's the uncompleted jobs in the queue I wish to stop.
The fact that the ThreadPool will create 250 threads per processor by default means if you ask the ThreadPool to queue a large amount of items with QueueUserWorkItem() you end up creating a huge amount of concurrent threads that you have no way of stopping.
The caller is able to push the CPU to 100% with not only the work but the creation of the work because of the way I queued the threads.
I was thinking by using the Producer/Consumer pattern I could queue these threads into my own queue that would allow me to moderate how many threads I create to avoid the CPU spike creating all the concurrent threads. And that I might be able to allow the caller of my class to flush all the jobs in the queue when they are abandoning the requests.
I am currently trying to implement this myself but figured SO was a good place to have someone say look at this code or you won't be able to flush because of this or flushing isn't the right term you mean this.
EDIT My answer does not apply since OP is using 2.0. Leaving up and switching to CW for anyone who reads this question and using 4.0
If you are using C# 4.0, or can take a depedency on one of the earlier version of the parallel frameworks, you can use their built-in cancellation support. It's not as easy as cancelling a thread but the framework is much more reliable (cancelling a thread is very attractive but also very dangerous).
Reed did an excellent article on this you should take a look at
http://reedcopsey.com/2010/02/17/parallelism-in-net-part-10-cancellation-in-plinq-and-the-parallel-class/
A method I've used in the past, though it's certainly not a best practice is to dedicate a class instance to each thread, and have an abort flag on the class. Then create a ThrowIfAborting method on the class that is called periodically from the thread (particularly if the thread's running a loop, just call it every iteration). If the flag has been set, ThrowIfAborting will simply throw an exception, which is caught in the main method for the thread. Just make sure to clean up your resources as you're aborting.
You could extend the Begin/End pattern to become the Begin/Cancel/End pattern. The Cancel method could set a cancel flag that the worker thread polls periodically. When the worker thread detects a cancel request, it can stop its work, clean-up resources as needed, and report that the operation was canceled as part of the End arguments.
I've solved what I believe to be your exact problem by using a wrapper class around 1+ BackgroundWorker instances.
Unfortunately, I'm not able to post my entire class, but here's the basic concept along with it's limitations.
Usage:
You simply create an instance and call RunOrReplace(...) when you want to cancel your old worker and start a new one. If the old worker was busy, it is asked to cancel and then another worker is used to immediately execute your request.
public class BackgroundWorkerReplaceable : IDisposable
{
BackgroupWorker activeWorker = null;
object activeWorkerSyncRoot = new object();
List<BackgroupWorker> workerPool = new List<BackgroupWorker>();
DoWorkEventHandler doWork;
RunWorkerCompletedEventHandler runWorkerCompleted;
public bool IsBusy
{
get { return activeWorker != null ? activeWorker.IsBusy; : false }
}
public BackgroundWorkerReplaceable(DoWorkEventHandler doWork, RunWorkerCompletedEventHandler runWorkerCompleted)
{
this.doWork = doWork;
this.runWorkerCompleted = runWorkerCompleted;
ResetActiveWorker();
}
public void RunOrReplace(Object param, ...) // Overloads could include ProgressChangedEventHandler and other stuff
{
try
{
lock(activeWorkerSyncRoot)
{
if(activeWorker.IsBusy)
{
ResetActiveWorker();
}
// This works because if IsBusy was false above, there is no way for it to become true without another thread obtaining a lock
if(!activeWorker.IsBusy)
{
// Optionally handle ProgressChangedEventHandler and other features (under the lock!)
// Work on this new param
activeWorker.RunWorkerAsync(param);
}
else
{ // This should never happen since we create new workers when there's none available!
throw new LogicException(...); // assert or similar
}
}
}
catch(...) // InvalidOperationException and Exception
{ // In my experience, it's safe to just show the user an error and ignore these, but that's going to depend on what you use this for and where you want the exception handling to be
}
}
public void Cancel()
{
ResetActiveWorker();
}
public void Dispose()
{ // You should implement a proper Dispose/Finalizer pattern
if(activeWorker != null)
{
activeWorker.CancelAsync();
}
foreach(BackgroundWorker worker in workerPool)
{
worker.CancelAsync();
worker.Dispose();
// perhaps use a for loop instead so you can set worker to null? This might help the GC, but it's probably not needed
}
}
void ResetActiveWorker()
{
lock(activeWorkerSyncRoot)
{
if(activeWorker == null)
{
activeWorker = GetAvailableWorker();
}
else if(activeWorker.IsBusy)
{ // Current worker is busy - issue a cancel and set another active worker
activeWorker.CancelAsync(); // Make sure WorkerSupportsCancellation must be set to true [Link9372]
// Optionally handle ProgressEventHandler -=
activeWorker = GetAvailableWorker(); // Ensure that the activeWorker is available
}
//else - do nothing, activeWorker is already ready for work!
}
}
BackgroupdWorker GetAvailableWorker()
{
// Loop through workerPool and return a worker if IsBusy is false
// if the loop exits without returning...
if(activeWorker != null)
{
workerPool.Add(activeWorker); // Save the old worker for possible future use
}
return GenerateNewWorker();
}
BackgroundWorker GenerateNewWorker()
{
BackgroundWorker worker = new BackgroundWorker();
worker.WorkerSupportsCancellation = true; // [Link9372]
//worker.WorkerReportsProgress
worker.DoWork += doWork;
worker.RunWorkerCompleted += runWorkerCompleted;
// Other stuff
return worker;
}
} // class
Pro/Con:
This has the benefit of having a very low delay in starting your new execution, since new threads don't have to wait for old ones to finish.
This comes at the cost of a theoretical never-ending growth of BackgroundWorker objects that never get GC'd. However, in practice the code below attempts to recycle old workers so you shouldn't normally encounter a large pool of ideal threads. If you are worried about this because of how you plan to use this class, you could implement a Timer which fires a CleanUpExcessWorkers(...) method, or have ResetActiveWorker() do this cleanup (at the cost of a longer RunOrReplace(...) delay).
The main cost from using this is precisely why it's beneficial - it doesn't wait for the previous thread to exit, so for example, if DoWork is performing a database call and you execute RunOrReplace(...) 10 times in rapid succession, the database call might not be immediately canceled when the thread is - so you'll have 10 queries running, making all of them slow! This generally tends to work fine with Oracle, causing only minor delays, but I do not have experiences with other databases (to speed up the cleanup, I have the canceled worker tell Oracle to cancel the command). Proper use of the EventArgs described below mostly solves this.
Another minor cost is that whatever code this BackgroundWorker is performing must be compatible with this concept - it must be able to safely recover from being canceled. The DoWorkEventArgs and RunWorkerCompletedEventArgs have a Cancel/Cancelled property which you should use. For example, if you do Database calls in the DoWork method (mainly what I use this class for), you need to make sure you periodically check these properties and take perform the appropriate clean-up.