In continuation for my latest ponders about locks in C# and .NET,
Consider the following scenario:
I have a class which contains a specific collection (for this example, i've used a Dictionary<string, int>) which is updated from a data source every few minutes using a specific method which it's body you can see below:
DataTable dataTable = dbClient.ExecuteDataSet(i_Query).GetFirstTable();
lock (r_MappingLock)
{
i_MapObj.Clear();
foreach (DataRow currRow in dataTable.Rows)
{
i_MapObj.Add(Convert.ToString(currRow[i_Column1]), Convert.ToInt32[i_Column2]));
}
}
r_MappingLock is an object dedicated to lock the critical section which refreshes the dictionary's contents.
i_MapObj is the dictionary object
i_Column1 and i_Column2 are the datatable's column names which contain the desired data for the mapping.
Now, I also have a class method which receives a string and returns the correct mapped int based on the mentioned dictionary.
I want this method to wait until the refresh method completes it's execution, so at first glance one would consider the following implementation:
lock (r_MappingLock)
{
int? retVal = null;
if (i_MapObj.ContainsKey(i_Key))
{
retVal = i_MapObj[i_Key];
}
return retVal;
}
This will prevent unexpected behaviour and return value while the dictionary is being updated, but another issue arises:
Since every thread which executes the above method tries to claim the lock, it means that if multiple threads try to execute this method at the same time, each will have to wait until the previous thread finished executing the method and try to claim the lock, and this is obviously an undesirable behaviour since the above method is only for reading purposes.
I was thinking of adding a boolean member to the class which will be set to true or false wether the dictionary is being updated or not and checking it within the "read only" method, but this arise other race-condition based issues...
Any ideas how to solve this gracefully?
Thanks again,
Mikey
Have a look at the built in ReaderWriterLock.
I would just switch to using a ConcurrentDictionary to avoid this situation altogether - manually locking is error-prone. Also as I can gather from "C#: The Curious ConcurrentDictionary", ConcurrentDictionary is already read-optimized.
Albin pointed out correctly at ReaderWriterLock. I will add an even nicer one: ReaderWriterGate by Jeffrey Richter. Enjoy!
You might consider creating a new dictionary when updating, instead of locking. This way, you will always have consistent results, but reads during updates would return previous data:
private volatile Dictionary<string, int> i_MapObj = new Dictionary<string, int>();
private void Update()
{
DataTable dataTable = dbClient.ExecuteDataSet(i_Query).GetFirstTable();
var newData = new Dictionary<string, int>();
foreach (DataRow currRow in dataTable.Rows)
{
newData.Add(Convert.ToString(currRow[i_Column1]), Convert.ToInt32[i_Column2]));
}
// Start using new data - reference assignments are atomic
i_MapObj = newData;
}
private int? GetValue(string key)
{
int value;
if (i_MapObj.TryGetValue(key, out value))
return value;
return null;
}
In C# 4.0 there is ReaderWriterLockSlim class that is a lot faster!
Almost as fast as a lock().
Keep the policy to disallow recursion (LockRecursionPolicy::NoRecursion) to keep performances so high.
Look at this page for more info.
Related
Why does the following code at random times write null values to the DataRow object?
ConcurrentDictionary<long, DataRow> dRowDict = new ConcurrentDictionary<long, DataRow>();
foreach (long klucz in data.Keys)
{
DataRow row = table.NewRow();
dRowDict.TryAdd(klucz, row);
}
Parallel.ForEach(data.Keys, klucz =>
{
Dane prz = data[klucz] as Dane;
DataRow dr = dRowDict[klucz];
foreach (PropertyDescriptor prop in llProp)
{
dr[prop.Name] = prop.GetValue(prz) ?? DBNull.Value;
}
});
foreach (DataRow dRow in dRowDict.Values)
{
table.Rows.Add(dRow);
if (dRow["FILED"] == DBNull.Value)
{
MessageBox.Show("ERROR...NULL VALUE"); //why this happen?
}
}
A plain for loop does not cause this problem - why, where is my bug?
My question is - can I modify the properties of any object inside a parallel loop?
Parallel.ForEach(myData.AsEnumerable()..., value =>
{
Object x = new Object(); // or x = dict[key];
x.A = ...;
x.B = ...; //Can I do this and is it safe?
}
Modifying the properties of an object inside a parallel loop is only allowed if the object's class is thread-safe. In other word if the class was specifically designed to allow access to its members from multiple threads concurrently. Most classes are not designed this way, and instead they assume that will be accessed by one thread at a time.
From my experience thread-"unsafety" is considered the default, so if you are searching in the documentation of a specific class for a Thread Safety section, and you can't find it, you should assume that the class is not thread safe.
Accessing a non-thread-safe object from multiple threads concurrently results to "undefined behavior". This is a nice and formal synonym of "all kind of nastiness". It includes but it's not limited to random exceptions, corrupted state, lost updates, torn values, compromised security etc. These phenomena are non-deterministic in nature and often non-reproducible. So it is a strongly inadvisable practice.
The ADO.NET classes are thread-safe only for multithreaded read operations. Meaning that it is safe for multiple threads to read their properties concurrently. But when a thread is modifying an ADO.NET object, no other thread should access this object, either for reading or modifying it.
Please suggest to me a pattern to do the following multi-threading task:
I am going to cache table rows, and need to find them by two ways:
by Id (int)
by Key (string)
I would like to have single row storage, and use two Dictionaries to find rows efficiently.
In the cache, I must read each row from the db and store it in global storage, and add it by key and by id to both dictionaries. I need all this stuff must work in a multithreading environment.
Can anyone suggest an elegant way to do this?
Update. My fault. I missed the obvious (for myself) restriction in trying to avoid locks because in the case of more "common" usage, when row is readed from some different source (not db) lock could lead to deadlock...
This is pretty much a classic case of dealing with atomic operations. Adding an item to cache involves, using your approach, at least three operations that need to be executed atomically: retrieve data from db, store it in dictionaryByKey, store it in dictionaryByName.
ConcurrentDictionary won't help you here because that object can only safeguard itself against concurrent requests - since it has no knowledge of the fact that there are other operations that need to happen atomically, it can't help you avoid consistency problems.
The basic solution is simple: use a rwlock to safeguard reads and writes to cache. ReaderWriterLock(Slim) should work just fine especially since I assume that the majority of cache hits will hopefully be reads.
Assuming MyCache is your cache class, fetching an item would look something like this:
public class MyCache{
private ReaderWriterLock rwlock;
..................
public object Get(int id)//same for the other one based on name
{
rwlock.AcquireReaderLock(Timeout.Infinite);
try{
if(cacheID.Contains(id)){return cacheID[id];}
//item MIGHT not be in cache (not certain since we're still under read lock)
//1. fetch from db BEFORE upgrade to write - avoid blocking all other readers
var item = GetItemFromStorage(id);//you get the idea
LockCookie lk = rwlock.UpgradeToWriterLock(Timeout.Infinite);
try{
if(cacheID.Contains(id)){return cacheID[id];}//check again!!!
//2. insert in cacheID
cacheID[id]=item;
//3. insert in cacheName
cacheName[item->key]=item;
//return value
return item;
}finally{rwlock.DowngradeFromWriterLock(ref lk);}
}
finally{rwlock.ExitReadLock();}
}
private object dictLock = new object();
private Dictionary<int, int> dict1 = new Dictionary<int, int>();
private Dictionary<string, int> dict2 = new Dictionary<string, int>();
public void Add(int rownr, int id, string key)
{
lock(dictLock)
{
dict1.Add(id, rownr);
dict2.Add(key, rownr);
}
}
public int GetRow(int id)
{
lock(dictLock)
{
return dict1[id];
}
}
public int GetRow(string key)
{
lock(dictLock)
{
return dict2[key];
}
}
Recently I was running into the following exception when using a generic dictionary
An InvalidOperationException has occurred. A collection was modified
I realized that this error was primarily because of thread safety issues on the static dictionary I was using.
A little background: I currently have an application which has 3 different methods that are related to this issue.
Method A iterates through the dictionary using foreach and returns a value.
Method B adds data to the dictionary.
Method C changes the value of the key in the dictionary.
Sometimes while iterating through the dictionary, data is also being added, which is the cause of this issue. I keep getting this exception in the foreach part of my code where I iterate over the contents of the dictionary. In order to resolve this issue, I replaced the generic dictionary with the ConcurrentDictionary and here are the details of what I did.
Aim : My main objective is to completely remove the exception
For method B (which adds a new key to the dictionary) I replaced .Add with TryAdd
For method C (which updates the value of the dictionary) I did not make any changes. A rough sketch of the code is as follows :
static public int ChangeContent(int para)
{
foreach (KeyValuePair<string, CustObject> pair in static_container)
{
if (pair.Value.propA != para ) //Pending cancel
{
pair.Value.data_id = prim_id; //I am updating the content
return 0;
}
}
return -2;
}
For method A - I am simply iterating over the dictionary and this is where the running code stops (in debug mode) and Visual Studio informs me that this is where the error occured.The code I am using is similar to the following
static public CustObject RetrieveOrderDetails(int para)
{
foreach (KeyValuePair<string, CustObject> pair in static_container)
{
if (pair.Value.cust_id.Equals(symbol))
{
if (pair.Value.OrderStatus != para)
{
return pair.Value; //Found
}
}
}
return null; //Not found
}
Are these changes going to resolve the exception that I am getting.
Edit:
It states on this page that the method GetEnumerator allows you to traverse through the elements in parallel with writes (although it may be outdated). Isnt that the same as using foreach ?
For modification of elements, one option is to manually iterate the dictionary using a for loop, e.g.:
Dictionary<string, string> test = new Dictionary<string, string>();
int dictionaryLength = test.Count();
for (int i = 0; i < dictionaryLength; i++)
{
test[test.ElementAt(i).Key] = "Some new content";
}
Be weary though, that if you're also adding to the Dictionary, you must increment dictionaryLength (or decrement it if you move elements) appropriately.
Depending on what exactly you're doing, and if order matters, you may wish to use a SortedDictionary instead.
You could extend this by updating dictionaryLength explicitly by recalling test.Count() at each iteration, and also use an additional list containing a list of keys you've already modified and so on and so forth if there's a danger of missing any, it really depends what you're doing as much as anything and what your needs are.
You can further get a list of keys using test.Keys.ToList(), that option would work as follows:
Dictionary<string, string> test = new Dictionary<string, string>();
List<string> keys = test.Keys.ToList();
foreach (string key in keys)
{
test[key] = "Some new content";
}
IEnumerable<string> newKeys = test.Keys.ToList().Except(keys);
if(newKeys.Count() > 0)
// Do it again or whatever.
Note that I've also shown an example of how to find out whether any new keys were added between you getting the initial list of keys, and completing iteration such that you could then loop round and handle the new keys.
Hopefully one of these options will suit (or you may even want to mix and match- for loop on the keys for example updating that as you go instead of the length) - as I say, it's as much about what precisely you're trying to do as much as anything.
Before doing foreach() try out copying container to a new instance
var unboundContainer = static_container.ToList();
foreach (KeyValuePair<string, CustObject> pair in unboundContainer)
Also I think updating Value property is not right from thread safety perspectives, refactor your code to use TryUpdate() instead.
I am trying to re-write some code using Dictionary to use ConcurrentDictionary. I have reviewed some examples but I am still having trouble implementing the AddOrUpdate function. This is the original code:
dynamic a = HttpContext;
Dictionary<int, string> userDic = this.HttpContext.Application["UserSessionList"] as Dictionary<int, String>;
if (userDic != null)
{
if (useDic.ContainsKey(authUser.UserId))
{
userDic.Remove(authUser.UserId);
}
}
else
{
userDic = new Dictionary<int,string>();
}
userDic.Add(authUser.UserId, a.Session.SessionID.ToString());
this.HttpContext.Application["UserDic"] = userDic;
I don't know what to add for the update portion:
userDic.AddOrUpdate(authUser.UserId,
a.Session.SessionID.ToString(),
/*** what to add here? ***/);
Any pointers would be appreciated.
You need to pass a Func which returns the value to be stored in the dictionary in case of an update. I guess in your case (since you don't distinguish between add and update) this would be:
var sessionId = a.Session.SessionID.ToString();
userDic.AddOrUpdate(
authUser.UserId,
sessionId,
(key, oldValue) => sessionId);
I.e. the Func always returns the sessionId, so that both Add and Update set the same value.
BTW: there is a sample on the MSDN page.
I hope, that I did not miss anything in your question, but why not just like this? It is easier, atomic and thread-safe (see below).
userDic[authUser.UserId] = sessionId;
Store a key/value pair into the dictionary unconditionally, overwriting any value for that key if the key already exists: Use the indexer’s setter
(See: http://blogs.msdn.com/b/pfxteam/archive/2010/01/08/9945809.aspx)
The indexer is atomic, too. If you pass a function instead, it might not be:
All of these operations are atomic and are thread-safe with regards to all other operations on the ConcurrentDictionary. The only caveat to the atomicity of each operation is for those which accept a delegate, namely AddOrUpdate and GetOrAdd. [...] these delegates are invoked outside of the locks
See: http://blogs.msdn.com/b/pfxteam/archive/2010/01/08/9945809.aspx
I ended up implementing an extension method:
static class ExtensionMethods
{
// Either Add or overwrite
public static void AddOrUpdate<K, V>(this ConcurrentDictionary<K, V> dictionary, K key, V value)
{
dictionary.AddOrUpdate(key, value, (oldkey, oldvalue) => value);
}
}
For those who are interested in, I am currently implementing a case which is a great example for using the "oldValue" aka existing value instead of forcing a new one (personally I don't like the term "oldValue" as it is not that old when it was created just a few processor ticks ago from within a parallel thread).
dictionaryCacheQueues.AddOrUpdate(
uid,
new ConcurrentQueue<T>(),
(existingUid, existingValue) => existingValue
);
I have an ICollection that I know will only ever have one member. Currently, I loop through it, knowing the loop will only ever run once, to grab the value. Is there a cleaner way to do this?
I could alter the persistentState object to return single values, but that would complicate the rest of the interface. It's grabbing data from XML, and for the most part ICollections are appropriate.
// worldMapLinks ensured to be a singleton
ICollection<IDictionary<string, string>> worldMapLinks = persistentState.GetAllOfType("worldMapLink");
string levelName = ""; //worldMapLinks.GetEnumerator().Current['filePath'];
// this loop will only run once
foreach (IDictionary<string, string> dict in worldMapLinks) // hacky hack hack hack
{
levelName = dict["filePath"];
}
// proceed with levelName
loadLevel(levelName);
Here is another example of the same issue:
// meta will be a singleton
ICollection<IDictionary<string, string>> meta = persistentState.GetAllOfType("meta");
foreach (IDictionary<string, string> dict in meta) // this loop should only run once. HACKS.
{
currentLevelName = dict["name"];
currentLevelCaption = dict["teaserCaption"];
}
Yet another example:
private Vector2 startPositionOfKV(ICollection<IDictionary<string, string>> dicts)
{
Vector2 result = new Vector2();
foreach (IDictionary<string, string> dict in dicts) // this loop will only ever run once
{
result.X = Single.Parse(dict["x"]);
result.Y = Single.Parse(dict["y"]);
}
return result;
}
Why not use the Single or FirstOrDefault extension methods?
var levelName = worldMapLinks.Single().Value;
Single has the advantage of enforcing your assumption that there is only 1 value in the enumeration. If this is not true an exception will be raised forcing you to reconsider your logic. FirstOrDefault will return a default value if there is not at least 1 element in the enumeration.
If you can use LINQ-to-objects in your class, use the Single() extension method on the collection if you know there will be exactly one member. Otherwise, if there could be zero or one, use SingleOrDefault()
Why do you have a collection with only one member? It seems that the real answer should be to better design your system rather than rely on any method to retrieve one element from a collection. You say it makes it more complicated, but how? Isn't this solution itself a complication? Is it possible to change the interface to return one element where applicable and a collection elsewhere? Seems like a code smell to me.