I have a DataTable that can contains a large number of DataRows; this DataTable can be accessed from several threads. Some threads can also change values into some rows. Actually there are some search functions that locks the DataTable, search into it using linq and return the expected value, something similar of:
lock(tableContent)
{
var t = (from row in tableContent.AsEnumerable()
where row[fieldName] != DBNull.Value && row.Field<T>(fieldName).Equals(someValue) select row);
if (t.Any())
{ ... }
}
The question is: If I clone (locking the original) the DataTable and search into the cloned object, will the locked period be faster than searching directly into the original?
I think that the Cloning operation will take O(n) to copy each row, so the time will be the same as the search, but I don't know if there is some optimizations (memory copy, ...) that reduces the Cloning time or something similar.
Clone is O(n), but that doesn't tell the full story. A clone can be a shallow clone (just copy the references in the table) or a deep clone (copy the objects themselves). A deep clone can be a very expensive operation. Searching time can vary, too, from a quick search that checks just a single integer field, to a complex search that compares multiple values and is pretty expensive. In addition, if your data is sorted on the field that you're searching, then search is O(log n), which will be considerably faster than O(n).
If you need to take into account the possibility that somebody will add, modify, or delete rows, then you either have to lock or clone. If you're doing a single search, then cloning doesn't really make sense because you'd have to lock the table in order to clone it. And cloning will most likely take longer than searching, unless your searches are unusually expensive.
You say that modifications are rare and searches are frequent. In that case, I would suggest that you use a reader-writer lock, which will support an unlimited number of readers, or one writer. In .NET, you probably want the ReaderWriterLockSlim. Using that, your code would look like this:
private ReaderWriterLockSlim tableLock = new ReaderWriterLockSlim();
public bool Search(string s)
{
tableLock.EnterReadLock();
try
{
// do the search here
return result;
}
finally
{
tableLock.ExitReadLock();
}
}
Any number of readers can be searching the table concurrently. Provided, of course, that they don't modify it. If you want to modify the table, you have to acquire the write lock:
public void Modify(string s)
{
tableLock.EnterWriteLock();
try
{
// do the modification here
return;
}
finally
{
tableLock.ExitWriteLock();
}
}
When a thread tries to enter the write lock, it has to wait for all existing readers to exit. Readers that come in after the write lock was requested have to wait for the existing readers to exit, and for the writer to acquire and then release the lock.
The reader/writer lock works really well for me in situations similar to the one you described: frequent reads and infrequent writes. It's worth looking into, if nothing else because it's so easy to test.
The reader/writer lock still works well if searches and updates are approximately equal because it still allows multiple readers when possible. Come to think of it, it even works well if writes are much more frequent, again because it will allow multiple reads when possible. I almost always use ReaderWriterLockSlim when I have a data structure that can be searched and updated.
There are other solutions, but they involve custom data structures that can be much more difficult to implement and maintain. I'd suggest you give the reader/writer lock a try. If after that, profiling shows that threads are still waiting on the lock and slowing your application's response time, you can look into alternatives.
I'm a little concerned, though, that you're doing more than just searching. Your sample selects a bunch of rows and then you do if (t.Any()) { ... }. Just what are you doing in that { ... }? If that takes a long time, you might be better off making your code clone just the rows that you select. You can then release the lock and party on the result set to your heart's content without affecting other threads that need to access the data structure.
Related
I am in need of a data type that is able to insert entries and then be able to quickly determine if an entry has already been inserted. A Dictionary seems to suit this need (see example). However, I have no use for the dictionary's values. Should I still use a dictionary or is there another better suited data type?
public class Foo
{
private Dictionary<string, bool> Entities;
...
public void AddEntity(string bar)
{
if (!Entities.ContainsKey(bar))
{
// bool value true here has no use and is just a placeholder
Entities.Add(bar, true);
}
}
public string[] GetEntities()
{
return Entities.Keys.ToArray();
}
}
You can use HashSet<T>.
The HashSet<T> class provides high-performance set operations. A set
is a collection that contains no duplicate elements, and whose
elements are in no particular order.
Habib's answer is excellent, but for multi-threaded environments if you use a HashSet<T> then by consequence you have to use locks to protect access to it. I find myself more prone to creating deadlocks with lock statements. Also, locks yield a worse speedup per Amdahl's law because adding a lock statement reduces the percentage of your code that is actually parallel.
For those reasons, a ConcurrentDictionary<T,object> fits the bill in multi-threaded environments. If you end up using one, then wrap it like you did in your question. Just new up objects to toss in as values as needed, since the values won't be important. You can verify that there are no lock statements in its source code.
If you didn't need mutability of the collection then this would be moot. But your question implies that you do need it, since you have an AddEntity method.
Additional info 2017-05-19 - actually, ConcurrentDictionary does use locks internally, although not lock statements per se--it uses Monitor.Enter (check out the TryAddInternal method). However, it seems to lock on individual buckets within the dictionary, which means there will be less contention than putting the entire thing in a lock statement.
So all in all, ConcurrentDictionary is often better for multithreaded environments.
It's actually quite difficult (impossible?) to make a concurrent hash set using only the Interlocked methods. I tried on my own and kept running into the problem of needing to alter two things at the same time--something that only locking can do in general. One workaround I found was to use singly-linked lists for the hash buckets and intentionally create cycles in a list when one thread needed to operate on a node without interference from other threads; this would cause other threads to get caught spinning around in the same spot until that thread was done with its node and undid the cycle. Sure, it technically didn't use locks, but it did not scale well.
I've created a game emulation program using c# async socks. I need to remove/add & do iterations on a collection (a list that holds clients) concurrently. I am currently using "lock", however, it's a a huge performance drop. I also do not want to use "local lists/copies" to keep the list up-to-date. I've heard about "ConcurrentBags", however, I am not sure how thread safe they are for iterations (for instance if a thread removes an element from the list while another thread is doing an iteration on it!?).
What do you suggest?
Edit: here is a situation
this is when a packet is sent to all the users in a room
lock (parent.gameClientList)
{
for (int i = 0; i <= parent.gameClientList.Count() - 1; i++) if (parent.gameClientList[i].zoneId == zoneId) parent.gameClientList[i].SendXt(packetElements); //if room matches - SendXt sends a packet
}
When a new client connects
Client connectedClient = new Client(socket, this);
lock (gameClientList)
{
gameClientList.Add(connectedClient);
}
Same case when a client disconnects.
I am asking for a better alternative (performance-wise) because the locks slow down everything.
It sounds like the problem is that you're doing all the work within your foreach loop, and it's locking out the add/remove methods for too long. The way around this is to quickly make a copy of the collection while it's locked, and then you can close the lock and iterate on the copy.
Thing[] copy;
lock(myLock) {
copy = _collection.ToArray();
}
foreach(var thing in copy) {...}
The drawback is that by the time you get around to operating on some object of that copy, it may have been removed from the original collection and so maybe you don't want to operate on it anymore. That's another thing you'll just have to figure out the requirements. If that's a problem, a simple option would be to lock each iteration of the loop, which of course will slow things down but at least it won't lock for the entire duration the loop is running:
foreac(var thing in copy) {
lock(myLock) {
if (_collection.Contains(thing)) //check that it's still in the original colleciton
DoWork(thing); //If you can move this outside the lock it'd make your app snappier, but it could also cause problems if you're doing something "dangerous" in DoWork.
}
}
If this is what you meant by "local copies", then you can disregard this option, but I figured I'd offer it in case you meant something else.
Every time you do something concurrently you are going to have loss due to task management (i.e. locks). I suggest you look at what is the bottleneck in your process. You seem to have a shared memory model, as opposed to a message passing model. If you know you need to modify the entire collection at once, there may not be a good solution. But if you are making changes in a particular order you can leverage that order to prevent delays. Locks is an implementation of pessimistic concurrency. You could switch to an optimistic concurrency model. In one the cost is waiting in the other the cost is retrying. Again the actual solution depends on your use case.
On problem with ConcurrentBag is that it is unordered so you cannot pull items out by index the same way you are doing it currently. However, you can iterate it via foreach to get the same effect. This iteration will be thread-safe. It will not go bizerk if an item is added or removed while the iteration is happening.
There is another problem with ConcurrentBag though. It actually copies the contents to a new List internally to make the enumerator work correctly. So even if you wanted to just pick off a single item via the enumerator it would still be a O(n) operation because of the way enumerator works. You can verify this by disassembling it.
However, based on context clues from your update I assume that this collection is going to be small. It appears that there is only one entry per "game client" which means it is probably going to store a small number of items right? If that is correct then the performance of the GetEnumerator method will be mostly insignificant.
You should also consider ConcurrentDictionary as well. I noticed that you are trying to match items from the collection based on zoneId. If you store the items in the ConcurrentDictionary keyed by zoneId then you would not need to iterate the collection at all. Of course, this assumes that there is only one entry per zoneId which may not be the case.
You mentioned that you did not want to use "local lists/copies", but you never said why. I think you should reconsider this for the following reasons.
Iterations could be lock-free.
Adding and removing items appears to be infrequent based context clues from your code.
There are a couple of patterns you can use to make the list copying strategy work really well. I talk about them in my answers here and here.
I have an array, that represents an inventory, with nearly 50 elements (items: some costum objects) and I need a readerwritelock for it (okay, i think a simple lock would be enough too). It should support both reference changing and value changing.
As reading and writing to different position of the array is threadsafe (Proof) I want to ensure that multiple read/write operations on the same array position is also threadsafe.
I surely could create 50 readerwriterlocks, but I don't want that ;)
Is there a way to archive this? (I know ConcurrentList/Dictionary/etc. but I want an array...)
Thanks
If you are replacing the references in the array, then this is already safe, since reference swaps are inherently atomic. So you can use:
var val = arr[index];
// now work with val
and
var newVal = ...
arr[index] = newVal;
perfectly safely, at least in terms of avoiding torn references. So one pragmatic option is to make the object immutable, and just employ the above. If you need to change the value, take a local copy, make a new version based from that, and then swap them. If lost updates are a problem, then Interlocked.CompareExchange and a re-apply loop can be used very successfully (i.e. you keep reapplying your change until you "win"). This avoids the need for any locking.
If, however, you are mutating the individual objects, then the game changes. You could make the object internally thread-safe, but this is usually not pretty. You could have a single lock for all the objects. But if you want granular locking then you will need multiple locks.
My advice: make the object immutable and just use the atomic reference-swap trick.
First off, you may not need any locks. Reading and writing with an array of a type where the CPU would handle each read and write atomically, is in and of itself thread-safe (but you might want to put in a memory barrier to avoid stale reads).
That said, just like x = 34 for an integer is threadsafe but x++ is not, if you've writes that depend upon the current value (and which are hence a read and a write), then that is not threadsafe.
If you do need locks, but don't want as many as 50, you could stripe. First set up your striped locks (I'll use simple locks rather than ReaderWriterSlim for smaller example code, the same principle applies):
var lockArray = new object[8];
for(var i =0; i != lockArray.Length; ++i)
lockArray[i] = new object();
Then when you go to use it:
lock(lockArray[idx % 8])
{
//operate on item idx of your array here
}
It's a balance between the simplicity and size of one lock for everything, vs the memory use of one lock for each element.
The big difficulty comes in if an operation on one element depends on that of another, if you need to resize the array, or any other case where you need to have more than one lock. A lot of deadlock situations can be avoided by always acquiring the locks in the same order (so no other thread needing more than one lock will try to get one you already have while holding one you need), but you need to be very careful in these cases.
You also want to make sure that if you are dealing with say, index 3 and index 11, you avoid locking on object 3 twice (I can't think of a way this particular recursive locking would go wrong, but why not just avoid it rather than have to prove it's one of the cases where recursive locking is safe?)
What is better:
to have large code area in lock statement
or
to have small locks in large area?..
exchanges in this sample are not changable.
lock (padLock)
{
foreach (string ex in exchanges)
{
sub.Add(x.ID, new Subscription(ch, queue.QueueName, true));
.........
}
or
foreach (string ex in exchanges)
{
lock (padLock)
{
sub.Add(x.ID, new Subscription(ch, queue.QueueName, true));
}
.....
The wider lock - the less you get from multithreading, and vice versa
So, use of locks completely depends on logic. Lock only things and places which changing and have to run only by one thread in a time
If you lock for using the collection sub - use smaller lock, but if you run multiple simultaneous foreach loops in parallel
Good practise would be to only lock that area which you want to be executed by only one thread at a given time
If that area is whole foreach loop then first approach is fine
but if that area is just one line as you have shown I second approach then go for the second approach
In this particular case, the best option is the first one, because otherwise you're just wasting time locking/unlocking since you have to execute the entire loop anyway. So there's not much opportunity for parallelism in a loop that executes individually atomic operations anyway.
For more general advice on critical section sizes check this article: http://software.intel.com/en-us/articles/managing-lock-contention-large-and-small-critical-sections/
I think there are two different questions:
1. Which would be correct?
2. Which would give better performance?
The correctness question is complicated. It depends on your data structures, and how you intend the lock to protect them. If the "sub" object is not thread-safe, you definitely need the big lock.
The performance question is simpler and less important (but for some reason, I think you're focused on it more).
Many small locks may be slower, because they just do more work. But if you manage to run a large portion of the loop code without lock, you get some concurrency, so it would be better.
You can't effectively judge which is "right" with the given code snippets. The first example says it is not OK for people to see sub with only part of the content from exchanges. The second example says it is OK for people to see sub with only part of the content from exchanges.
I've an application that makes use of parallelization for processing data.
The main program is in C#, while one of the routine for analyzing data is on an external C++ dll. This library scans data and calls a callback everytime a certain signal is found within the data. Data should be collected, sorted and then stored into HD.
Here is my first simple implementation of the method invoked by the callback and of the method for sorting and storing data:
// collection where saving found signals
List<MySignal> mySignalList = new List<MySignal>();
// method invoked by the callback
private void Collect(int type, long time)
{
lock(locker) { mySignalList.Add(new MySignal(type, time)); }
}
// store signals to disk
private void Store()
{
// sort the signals
mySignalList.Sort();
// file is a object that manages the writing of data to a FileStream
file.Write(mySignalList.ToArray());
}
Data is made up of a bidimensional array (short[][] data) of size 10000 x n, with n variable. I use parallelization in this way:
Parallel.For(0, 10000, (int i) =>
{
// wrapper for the external c++ dll
ProcessData(data[i]);
}
Now for each of the 10000 arrays I estimate that 0 to 4 callbacks could be fired. I'm facing a bottleneck and given that my CPU resources are not over-utilized, I suppose that the lock (together with thousand of callbacks) is the problem (am I right or there could be something else?). I've tried the ConcurrentBag collection but performances are still worse (in line with other user findings).
I thought that a possible solution for use lock-free code would be to have multiple collections. Then it would be necessary a strategy to make each thread of the parallel process working on a single collection. Collections could be for instance inside a dictionary with thread ID as key, but I do not know any .NET facility for this (I should know the threads ID for initialize the dictionary before launching the parallelization). Could be this idea feasible and, in case yes, does exist some .NET tool for this? Or alternatively, any other idea to speed up the process?
[EDIT]
I've followed the Reed Copsey's suggestion and I used the following solution (according to the profiler of VS2010, before the burden for locking and adding to the list was taking 15% of the resources, while now only 1%):
// master collection where saving found signals
List<MySignal> mySignalList = new List<MySignal>();
// thread-local storage of data (each thread is working on its List<MySignal>)
ThreadLocal<List<MySignal>> threadLocal;
// analyze data
private void AnalizeData()
{
using(threadLocal = new ThreadLocal<List<MySignal>>(() =>
{ return new List<MySignal>(); }))
{
Parallel.For<int>(0, 10000,
() =>
{ return 0;},
(i, loopState, localState) =>
{
// wrapper for the external c++ dll
ProcessData(data[i]);
return 0;
},
(localState) =>
{
lock(this)
{
// add thread-local lists to the master collection
mySignalList.AddRange(local.Value);
local.Value.Clear();
}
});
}
}
// method invoked by the callback
private void Collect(int type, long time)
{
local.Value.Add(new MySignal(type, time));
}
thought that a possible solution for use lock-free code would be to have multiple collections. Then it would be necessary a strategy to make each thread of the parallel process working on a single collection. Collections could be for instance inside a dictionary with thread ID as key, but I do not know any .NET facility for this (I should know the threads ID for initialize the dictionary before launching the parallelization). Could be this idea feasible and, in case yes, does exist some .NET tool for this? Or alternatively, any other idea to speed up the process?
You might want to look at using ThreadLocal<T> to hold your collections. This automatically allocates a separate collection per thread.
That being said, there are overloads of Parallel.For which work with local state, and have a collection pass at the end. This, potentially, would allow you to spawn your ProcessData wrapper, where each loop body was working on its own collection, and then recombine at the end. This would, potentially, eliminate the need for locking (since each thread is working on it's own data set) until the recombination phase, which happens once per thread (instead of once per task,ie: 10000 times). This could reduce the number of locks you're taking from ~25000 (0-4*10000) down to a few (system and algorithm dependent, but on a quad core system, probably around 10 in my experience).
For details, see my blog post on aggregating data with Parallel.For/ForEach. It demonstrates the overloads and explains how they work in more detail.
You don't say how much of a "bottleneck" you're encountering. But let's look at the locks.
On my machine (quad core, 2.4 GHz), a lock costs about 70 nanoseconds if it's not contended. I don't know how long it takes to add an item to a list, but I can't imagine that it takes more than a few microseconds. But let's it takes 100 microseconds (I would be very surprised to find that it's even 10 microseconds) to add an item to the list, taking into account lock contention. So if you're adding 40,000 items to the list, that's 4,000,000 microseconds, or 4 seconds. And I would expect one core to be pegged if this were the case.
I haven't used ConcurrentBag, but I've found the performance of BlockingCollection to be very good.
I suspect, though, that your bottleneck is somewhere else. Have you done any profiling?
The basic collections in C# aren't thread safe.
The problem you're having is due to the fact that you're locking the entire collection just to call an add() method.
You could create a thread-safe collection that only locks single elements inside the collection, instead of the whole collection.
Lets look at a linked list for example.
Implement an add(item (or list)) method that does the following:
Lock collection.
A = get last item.
set last item reference to the new item (or last item in new list).
lock last item (A).
unclock collection.
add new items/list to the end of A.
unlock locked item.
This will lock the whole collection for just 3 simple tasks when adding.
Then when iterating over the list, just do a trylock() on each object. if it's locked, wait for the lock to be free (that way you're sure that the add() finished).
In C# you can do an empty lock() block on the object as a trylock().
So now you can add safely and still iterate over the list at the same time.
Similar solutions can be implemented for the other commands if needed.
Any built-in solution for a collection is going to involve some locking. There may be ways to avoid it, perhaps by segregating the actual data constructs being read/written, but you're going to have to lock SOMEWHERE.
Also, understand that Parallel.For() will use the thread pool. While simple to implement, you lose fine-grained control over creation/destruction of threads, and the thread pool involves some serious overhead when starting up a big parallel task.
From a conceptual standpoint, I would try two things in tandem to speed up this algorithm:
Create threads yourself, using the Thread class. This frees you from the scheduling slowdowns of the thread pool; a thread starts processing (or waiting for CPU time) when you tell it to start, instead of the thread pool feeding requests for threads into its internal workings at its own pace. You should be aware of the number of threads you have going at once; the rule of thumb is that the benefits of multithreading are overcome by the overhead when you have more than twice the number of active threads as "execution units" available to execute threads. However, you should be able to architect a system that takes this into account relatively simply.
Segregate the collection of results, by creating a dictionary of collections of results. Each results collection is keyed to some token carried by the thread doing the processing and passed to the callback. The dictionary can have multiple elements READ at one time without locking, and as each thread is WRITING to a different collection within the Dictionary there shouldn't be a need to lock those lists (and even if you did lock them you wouldn't be blocking other threads). The result is that the only collection that has to be locked such that it would block threads is the main dictionary, when a new collection for a new thread is added to it. That shouldn't have to happen often if you're smart about recycling tokens.