I'll have a database object that can be accessed from multiple threads as well as from the main thread. I don't want them to access the underlying database object concurrently, so I'll write a set of thread safe public methods that can be accessed from multiple threads.
My first idea was to use a lock around my connection such as lock(oleDbConnection), but the problem is that I would have to lock it for the main thread since is one more thread that can access it. Which would mean rewriting lots of code.
But, since these threads and the main thread wont access the database very often, how about just using some of my control's (maybe the main form's) Invoke method every time I call any of the database methods from another thread. This way, as far as I understand, these methods would be never called concurrently, and I wouldn't need to worry about the main thread. I guess the only problem would be degrading performance a little bit, but as I said, the database is not accessed that often; the reason why I use threads is not so that they can access the database concurrently but so that they can perform other operations concurrently.
So does this sound like a good idea? Am I missing something? Sounds a bit too easy so I'm suspicious.
It sounds like it would work AFAIK, but it also sounds like a really bad idea.
The problem is that when writing lock you are saying "I want this code to be a critical section", whereas when writing Invoke you are saying "I want this to be executed on the UI thread". These two things are certainly not equivalent, which can lead to lots of problems. For example:
Invoke is normally used to access UI controls. What if a developer sees Invoke and nothing UI-related, and goes "gee, that's an unneeded Invoke; let's get rid of it"?
What if more than one UI thread ends up existing?
What if the database operation takes a long time (or times out)? Your UI would stop responding.
I would definitely go for the lock. You typically want the UI thread responsive when performing operations that may take time, which includes any sort of DB access; you don't know whether it's alive or not for instance.
Also, the typical way to handle connections is to create, use and dispose the connection for each request, rather than reusing the same connection. This might perhaps solve some of your concurrency problems.
Why don't you try to use Connection Pool. Every thread can do its work with a different DB connection and send the result to main thread with Invoke. Connection Pooling is a very common approach used in Servers.
See Using Connection Pooling with SQL Server
Related
I want to protect access to a resource in the following manner:
All threads can read concurrently, except during update (if the update is not atomic).
Only one thread can be assigned the task of updating, until next time an
update is required.
This may seem like a simple question of using a proper lock, or possibly making all operations atomic, but that is not it, I think.
If I just have a write-lock for updating (i.e. ReaderWriterLockSlim), or use non-locking code, nothing prevents more than one thread from running the update procedure (or queuing up to do so). If I use locking to block threads before checking if the resource need updating, they can't execute concurrently but are effectively serialized.
I could have specific threads performing all of the checking and updating of the resource, and utilize something like a ManualResetEvent to put other reading threads on hold until updating is finished. (Or if the updating is implemented as an atomic operation, just settle for having specific update threads.)
However, I'm uncertain about best practice, and I would like to ask if you think that the requirements may be met with less effort, or if I'm way off in any of my assumptions.
I think you are looking for a ReaderWriterLockSlim. Use the exclusive lock mode for writing.
I have an automatic betting BOT.
I use a Windows Service and timers to set off a job every 30 seconds in its own thread that takes bets from the DB, loops through and places them.
However in certain occurrences when the job is too long (over 30 seconds) I can get the same bet being placed twice using the same BetPK (unique ID) as the job for placing it runs at the same time as a previously started thread.
I am using C#, NET 4, VS 2012.
At the moment I set a "locked" flag in a table when the job to place bets runs and then unset it on finishing. So if another job runs and the job is locked it will return ASAP. However this is relying on the DB and network traffic.
What would be the best way in C# to prevent a job started by a timer thread from clashing with a previously started thread. I am thinking I could set a flag IN the service controller that spawns the threads so if a job is running another one won't spawn.
However I would like to learn the correct way to handle multi threaed clashes like this. I just lost a couple of hundred pounds today due to 2 LAY bets being placed at exactly the same time. As only one record existed for the Bet, the last bet placed had the Betfair ID updated so I had no clue about the duplicate until I checked Betfairs own page.
I do already do checks to see if the bet has already been placed before trying to place it but in cases where the "placebet" method is running on the same Bet record at exactly the same time then this is no good.
Any help much appreciated.
Thanks
No, the best solution is to keep the locks in the database. The app should be as stateless as possible. You already have a great solution.
Locking inside of your app is error prone and the errors are catastrophic (deadlock, the app stops to work until manually restarted). Locking using the database is much easier, and errors are recoverable.
Just get the locking with the database right. Ask a new question where you post details on what you're doing. I recommend that you XLOCK any betting jobs that you're working on. That way they can only be executed once. Use the power of database locks and transactions to make this work. This is by far easier than app-level threading.
You could always try implementing a db like Redis (redis.io) that offers built in POP functions (http://redis.io/commands/lpop). Redis has a C# client and is super useful for any kind of app where speed is crucial as it keeps the entire db in memory. It's also single threaded which makes it easy to implement distributors for multi-consumer type applications.
I'd also recommend checking out http://kkovacs.eu/cassandra-vs-mongodb-vs-couchdb-vs-redis as it lays out the pros and cons for Redis and other dbs. Might help you make future db decisions.
Old question, I know, but I wanted to throw this out there for anybody that stumbles across it.
C# (and presumably VB.NET) offers a couple of nice options for handling thread synchronization. You can use the lock keyword to block execution until a given lock is available, or Monitor.TryEnter() if you want to specify a timeout (possibly immediately) for taking the lock.
For either of these approaches, you need an object to use for locking. Pretty much any object will do; if you aren't synchronizing access to some object itself (collection, database connection, whatever), you can even just instantiate a throwaway object. For a polling timer, the latter is typical.
First, make sure you have an object to use for synchronization:
public class DatabasePollingClass {
object PollingTimerLock = new object();
...
Now, if you want the polling threads to block indefinitely waiting for their turn, use the lock keyword:
public class DatabasePollingClass {
object PollingTimerLock = new object();
...
protected void PollingTimerCallback() {
lock (PollingTimerLock) {
//Useful stuff here
}
}
}
Only a single thread will be allowed within the lock (PollingTimerLock) block of code at a time. All other threads will wait indefinitely, then resume executing as soon as they can acquire the lock for themselves.
However, you probably don't want that behavior. If you'd rather have the subsequent threads abort immediately (or after a short wait) if another polling thread is still running, you can use Monitor.TryEnter() when taking the lock. This does require slightly more caution, however:
public class DatabasePollingClass {
object PollingTimerLock = new object();
...
protected void PollingTimerCallback() {
if (Monitor.TryEnter(PollingTimerLock)) { //Acquires lock on PollingTimerLock object
try {
//Useful stuff here
} finally {
//Releases lock.
//You MUST do this in a finally block! (See below.)
Monitor.Exit(PollingTimerLock);
}
} else {
Console.WriteLine("Warning: Polling timer overlap. Skipping.");
}
}
}
The additional caution stems from the fact that, unlike the lock keyword, Monitor.TryEnter() requires you to manually release the lock when you're finished with it. In order to guarantee that this happens, you need to wrap your whole critical section in a try block, and release the lock in the finally block. This is to ensure that the lock will be released, even if the polling method fails or returns early. If the method returned without releasing the lock, your program would effectively be hung, as no further threads would be able to acquire the lock.
Another option, which doesn't use locking mechanisms, would be to configure your Timer without a repeat period, i.e. a one-shot Timer. At the end of your polling method, you would dispose the old Timer, and set a new one (you would also need to do this within a finally block to guarantee that the Timer gets reset by the end of the method). This approach would be useful if you want to poll the database at a certain interval since the end of the previous polling. It's a subtle distinction, but it also solves the problem of concurrent polling attempts.
Note that this is a really simple thread concurrency example. As long as all of your locking is happening on threads separate from your UI thread (the message pump itself can become a point of contention), and you're only ever locking a single object, you shouldn't have to worry too much about deadlocks. Those can be really unpleasant to debug; the symptom is usually "application stops responding, and now you get to guess which threads are waiting on what".
I'm currently developing a project with XNA that is pulling information (ID, name, file location, etc) about each of my objects (each object will be displayed on screen) from a local SQL database.
I'd like to run my database queries on a separate thread so the rendered screen doesn't freeze if the database hangs or some other unforeseen event occurs. I'm using XNA 4.0 and the application will only be running on windows. Is this possible, and if so, how?
There are a number of options available. Generally speaking you need the query to run in a separate thread. You can use
Thread pool
QueueUserWorkItem
Tasks
Background worker
Async calls to the database
Parallel invoke
Manually created threads here and here
I would start with thread pooling and see how that works, dedicated manual threads are not that robust in terms of memory management and reuse.
Not to do it at all. Seriously. There are good reasons for using threads, but your reasons are bogus:
the rendered screen doesn't freeze if the database hangs or some other unforeseen event occur
Databases dont hang and unforseen events are unforseen events. How you can cope with the database not answering for 3 minutes, for example? Show a screen with objects that are unknown?
How do you mean "best"? There are a lot of ways to use threads and they all have strengths and weaknesses.
Declaring a new thread explicitly and starting it gives you the most direct control over the execution state of that thread:
var myDbThread = new Thread(()=>myDbRepo.GetRecordById<MyEntity>(idString));
myDbThread.Start();
Now, as long as you have a reference to myDbThread, you can abort it, pause it, join on it, etc. BUT, with control comes responsibility; you have to manage the threads you create yourself.
For most parallel tasks, using the ThreadPool is recommended. However, you lose some of the control:
Action myDbLambda = () => myEntityProperty = myDbRepo.GetRecordById<MyEntity>(idString);
var asyncResult = myDbLambda.BeginInvoke();
Once asyncResult.IsComplete returns true, myEntityProperty has the value. You can also architect it as a Func, and use a callback to set the value (this is recommended). The Asynchronous Model is built in to the BeginInvoke()/EndInvoke() method pair, and many exceptions like timeouts are expected by the ThreadPool, which will simply restart the timed-out thread. However, you can't "give up" and terminate a ThreadPool thread, "joining" on a ThreadPool thread is a little trickier, and if you're launching a lot of threads, the ThreadPool will start them in 250ms intervals which may not be the best use of processor.
There are many ways to use the ThreadPool; before delegates became even more important to .NET programming in v3.5, ThreadPool.QueueUserWorkItem was the main method. Now, as I said, delegates have BeginInvoke and EndInvoke methods allowing you to kick off background processes with the asynchronous model built in behind the scenes. In WinForms/WPF, you can also create BackgroundWorker components which are event-driven, allowing you to monitor progress and completion in a GUI element.
One thing to be aware of; it is virtually never a good idea to use background threads in ASP.NET. Unless you really know what you're doing, best-case you won't get the results of the behavior you sent to the worker thread, and worst-case you can crash your site trying.
I have multiple threads starting at the roughly the same time --- all executing the same code path. Each thread needs to write records to a table in a database. If the table doesn't exist it should be created. Obviously two or more threads could see the table as missing, and try to create it.
What is the preferred approach to ensure that this particular block of code is executed only once by only one thread.
While I'm writing in C# on .NET 2.0, I assume that the approach would be framework/language neutral.
Something like this should work...
private object lockObject = new object();
private void CreateTableIfNotPresent()
{
lock(lockObject)
{
// check for table presence and create it if necessary,
// all inside this block
}
}
Have your threads call call the CreateTableIfNotPresent function. The lock block will ensure that no thread will be able to execute the code inside of the block concurrently, so no threads will be able to view the table as not present while another is creating it.
This is a classical application for either a Mutex or a Semaphore
A mutex ensures that a specific piece of code (or several pieces of code) can only be run by a single thread at a time. You could be clever and use a different mutex for each table, or simply constrain the whole initialisation block to one thread at a time.
A semaphore (or set of semaphores) could perform exactly the same function.
Most lock implementations will use a mutex internally, so look at what lock code is already available in the language or libraries you are using.
#ebpower has it right that in certain applications, you would actually be more efficient to catch an exception caused by an attempt to create the same table multiple times, though this may not be the case in your example.
However there are many other ways of proceeding. For example, you could use a single-threaded ExecutorService (sorry, I could only find a Java reference) that has responsibility for creating any tables that your worker threads discover are missing. If it gets two requests for the same table, it simply ignores the later ones.
A variant on a Memoizer (remembering table references, creating them first if necessary) would also work under the circumstances. The book Java Concurrency In Practice walks through the implementation of a nice Memoizer class, but this would be pretty simple to port to any other language with effective concurrency building blocks.
This is what Semaphores are for.
You may not even need to bother with locks since your database shouldn't let you create multiple tables with the same name. Why not just catch the appropriate exceptions and if two threads try to create the same table, one wins and continues on, while the other recovers and continues on.
I'd use a thread sync object such as ManualResetEvent though it sounds to me like you're willing a race condition which may mean you have a design problem
Some posts have suggested Mutexes - this is an overkill unless your threads are running on different processes.
Others have suggested using locks - this is fine but locking can lead to over-pessimistic locks on data which can negate the benefit of using threads in the first place.
A more fundamental question is why are you doing it this way at all? What benefit does threading bring to the problem domain? Does concurrency solve your problem?
You may want to try static constructors to get a reference of the table.
According to the MSDN (.net 2.0), A static constructor is used to initialize any static data, or to perform a particular action that needs performed once only.
Also, CLR automatically guarantees that a static constructor executes only once per AppDomain and is thread-safe.
For more info, check Chapter 8 of CLR via C# by Jeffrey Richter.
I understand the main function of the lock key word from MSDN
lock Statement (C# Reference)
The lock keyword marks a statement
block as a critical section by
obtaining the mutual-exclusion lock
for a given object, executing a
statement, and then releasing the
lock.
When should the lock be used?
For instance it makes sense with multi-threaded applications because it protects the data. But is it necessary when the application does not spin off any other threads?
Is there performance issues with using lock?
I have just inherited an application that is using lock everywhere, and it is single threaded and I want to know should I leave them in, are they even necessary?
Please note this is more of a general knowledge question, the application speed is fine, I want to know if that is a good design pattern to follow in the future or should this be avoided unless absolutely needed.
When should the lock be used?
A lock should be used to protect shared resources in multithreaded code. Not for anything else.
But is it necessary when the application does not spin off any other threads?
Absolutely not. It's just a time waster. However do be sure that you're not implicitly using system threads. For example if you use asynchronous I/O you may receive callbacks from a random thread, not your original thread.
Is there performance issues with using lock?
Yes. They're not very big in a single-threaded application, but why make calls you don't need?
...if that is a good design pattern to follow in the future[?]
Locking everything willy-nilly is a terrible design pattern. If your code is cluttered with random locking and then you do decide to use a background thread for some work, you're likely to run into deadlocks. Sharing a resource between multiple threads requires careful design, and the more you can isolate the tricky part, the better.
All the answers here seem right: locks' usefulness is to block threads from acessing locked code concurrently. However, there are many subtleties in this field, one of which is that locked blocks of code are automatically marked as critical regions by the Common Language Runtime.
The effect of code being marked as critical is that, if the entire region cannot be entirely executed, the runtime may consider that your entire Application Domain is potentially jeopardized and, therefore, unload it from memory. To quote MSDN:
For example, consider a task that attempts to allocate memory while holding a lock. If the memory allocation fails, aborting the current task is not sufficient to ensure stability of the AppDomain, because there can be other tasks in the domain waiting for the same lock. If the current task is terminated, other tasks could be deadlocked.
Therefore, even though your application is single-threaded, this may be a hazard for you. Consider that one method in a locked block throws an exception that is eventually not handled within the block. Even if the exception is dealt as it bubbles up through the call stack, your critical region of code didn't finish normally. And who knows how the CLR will react?
For more info, read this article on the perils of Thread.Abort().
Bear in mind that there might be reasons why your application is not as single-threaded as you think. Async I/O in .NET may well call-back on a pool thread, for example, as do some of the various timer classes (not the Windows Forms Timer, though).
Generally speaking if your application is single threaded, you're not going to get much use out of the lock statement. Not knowing your application exactly, I don't know if they're useful or not - but I suspect not. Further, if you're application is using lock everywhere I don't know that I would feel all that confident about it working in a multi-threaded environment anyways - did the original developer actually know how to develop multi-threaded code, or did they just add lock statements everywhere in the vague hope that that would do the trick?
lock should be used around the code that modifies shared state, state that is modified by other threads concurrently, and those other treads must take the same lock.
A lock is actually a memory access serializer, the threads (that take the lock) will wait on the lock to enter until the current thread exits the lock, so memory access is serialized.
To answer you question lock is not needed in a single threaded application, and it does have performance side effects. because locks in C# are based on kernel sync objects and every lock you take creates a transition to kernel mode from user mode.
If you're interested in multithreading performance a good place to start is MSDN threading guidelines
You can have performance issues with locking variables, but normally, you'd construct your code to minimize the lengths of time that are spent inside a 'locked' block of code.
As far as removing the locks. It'll depend on what exactly the code is doing. Even though it's single threaded, if your object is implemented as a Singleton, it's possible that you'll have multiple clients using an instance of it (in memory, on a server) at the same time..
Yes, there will be some performance penalty when using lock but it is generally neglible enough to not matter.
Using locks (or any other mutual-exclusion statement or construct) is generally only needed in multi-threaded scenarios where multiple threads (either of your own making or from your caller) have the opportunity to interact with the object and change the underlying state or data maintained. For example, if you have a collection that can be accessed by multiple threads you don't want one thread changing the contents of that collection by removing an item while another thread is trying to read it.
Lock(token) is only used to mark one or more blocks of code that should not run simultaneously in multiple threads. If your application is single-threaded, it's protecting against a condition that can't exist.
And locking does invoke a performance hit, adding instructions to check for simultaneous access before code is executed. It should only be used where necessary.
See the question about 'Mutex' in C#. And then look at these two questions regarding use of the 'lock(Object)' statement specifically.
There is no point in having locks in the app if there is only one thread and yes, it is a performance hit although it does take a fair number of calls for that hit to stack up into something significant.