I have a list of dictionaries
var ProductItemsDictionary = new List<Dictionary<string, string>>();
Is it possible to use linq to search the list and find a dictionary based on a key inside that dictionary and retrieve the value?
Sure it is, but is it worth? See for instance the dotctor's answer. It will do the job, but is inefficient - 2 key lookups (one for checking and one for retrieving the value), compiler generated class and heap allocation (because of the specificKey variable capture), etc. Less code? More readable? What about the non linq equivalent:
static void Foo(List<Dictionary<string, string>> ProductItemsDictionary, string key)
{
string value;
foreach (var dictionary in ProductItemsDictionary)
if (dictionary.TryGetValue(key, out value)) { /* Use value */ }
// Not found
}
Zero allocations, minimum key lookups, good readability (IMO) - what else do we need? :-)
Please suggest to me a pattern to do the following multi-threading task:
I am going to cache table rows, and need to find them by two ways:
by Id (int)
by Key (string)
I would like to have single row storage, and use two Dictionaries to find rows efficiently.
In the cache, I must read each row from the db and store it in global storage, and add it by key and by id to both dictionaries. I need all this stuff must work in a multithreading environment.
Can anyone suggest an elegant way to do this?
Update. My fault. I missed the obvious (for myself) restriction in trying to avoid locks because in the case of more "common" usage, when row is readed from some different source (not db) lock could lead to deadlock...
This is pretty much a classic case of dealing with atomic operations. Adding an item to cache involves, using your approach, at least three operations that need to be executed atomically: retrieve data from db, store it in dictionaryByKey, store it in dictionaryByName.
ConcurrentDictionary won't help you here because that object can only safeguard itself against concurrent requests - since it has no knowledge of the fact that there are other operations that need to happen atomically, it can't help you avoid consistency problems.
The basic solution is simple: use a rwlock to safeguard reads and writes to cache. ReaderWriterLock(Slim) should work just fine especially since I assume that the majority of cache hits will hopefully be reads.
Assuming MyCache is your cache class, fetching an item would look something like this:
public class MyCache{
private ReaderWriterLock rwlock;
..................
public object Get(int id)//same for the other one based on name
{
rwlock.AcquireReaderLock(Timeout.Infinite);
try{
if(cacheID.Contains(id)){return cacheID[id];}
//item MIGHT not be in cache (not certain since we're still under read lock)
//1. fetch from db BEFORE upgrade to write - avoid blocking all other readers
var item = GetItemFromStorage(id);//you get the idea
LockCookie lk = rwlock.UpgradeToWriterLock(Timeout.Infinite);
try{
if(cacheID.Contains(id)){return cacheID[id];}//check again!!!
//2. insert in cacheID
cacheID[id]=item;
//3. insert in cacheName
cacheName[item->key]=item;
//return value
return item;
}finally{rwlock.DowngradeFromWriterLock(ref lk);}
}
finally{rwlock.ExitReadLock();}
}
private object dictLock = new object();
private Dictionary<int, int> dict1 = new Dictionary<int, int>();
private Dictionary<string, int> dict2 = new Dictionary<string, int>();
public void Add(int rownr, int id, string key)
{
lock(dictLock)
{
dict1.Add(id, rownr);
dict2.Add(key, rownr);
}
}
public int GetRow(int id)
{
lock(dictLock)
{
return dict1[id];
}
}
public int GetRow(string key)
{
lock(dictLock)
{
return dict2[key];
}
}
After an extensive search here I still need an expert advice:
Requirements
Requirements are pretty standard: map int consts to strings from resource files so localization will work fine and we can easily create a list of value pairs for dropdown lists in current user language as well as doing backward lookup on int value. Const integers are defined as static in different classes.
So, here is how it's implemented currently and something tells me it can be better and deserves critics:
Dictionary class:
public class TLiteralDic : Dictionary<int, string>
{
//Lookup method
public string getLiteral(int key)
{
if (ContainsKey(key)) {
return this[key];
}
return string.Empty;
}
}
Somewhere close to UI layer an extension method to fill dropdown controls is defined like this:
public static void fill(this TLiteralDic dic, DropDownList ddlst)
{
ddlst.Items.Clear();
foreach (KeyValuePair<int, string> v in dic) {
ddlst.Items.Add(new ListItem(v.Value, v.Key.ToString()));
}
}
Adding pairs (static const int to string from a resource file):
public static class TLiterals
{
private static TLiteralDic _fileStatus;
public static TLiteralDic FileStatus
{
get
{
if (_fileStatus == null) {
_fileStatus = new TLiteralDic() {
{TFiles.Status.Cancelled, Lists.FileStatus_Cancelled},
{TFiles.Status.Closed, Lists.FileStatus_Closed},
{TFiles.Status.OnHold, Lists.FileStatus_OnHold},
{TFiles.Status.Open, Lists.FileStatus_Open},
{TFiles.Status.Pending, Lists.FileStatus_Pending},
{TFiles.Status.Portal, Lists.FileStatus_Portal}
};
}
return _fileStatus;
}
}
//... hundred of lists like that, short and long (up to 15 entries)
}
Using in code
Lookup:
textStatus.Text = TLiterals.FileStatus.getLiteral(row.FileStatus);
List fill:
TLiterals.FileStatus.fill(ddlstFileStatus);
Considerations
The idea is to have only one place where the mapping is defined and be able to create a list out of it or do a lookup by int value. Ideally with good performance, minimal memory footprint and coding hassle.
Already considered the following alternatives:
using switch (was used in the past for lookup - a lot of redundant code)
using reflection
closures
JSON, XML
using T4 class generation in VS2010
No simple or obviously better solution was found so far.
Issues in the current implementation
a lot of repetitive code around the actual list of pairs, ideally must be all hidden and reused
need to define static private property; the idea was to use "lazy" initialization for lists only when they are retrieved the first time.
keeping literals in memory after the first use maybe too high price for such simple operation
Advantages with the current implementation
list defined in one place (vs switch lookup and manual list population in 2 different places using the same value pairs)
reused code for lookup and list filling.
simplicity to maintain and compile time checks.
Any ideas for a better "beautiful code"? :)
Ideally, I'd like to see something like this when defining the list but it should not initialize until really needed/used:
public static TLiteralDic FileStatus = new TLiteralDic () {
{TFiles.Status.Cancelled, Lists.FileStatus_Cancelled},
{TFiles.Status.Closed, Lists.FileStatus_Closed},
{TFiles.Status.OnHold, Lists.FileStatus_OnHold},
{TFiles.Status.Open, Lists.FileStatus_Open},
{TFiles.Status.Pending, Lists.FileStatus_Pending},
{TFiles.Status.Portal, Lists.FileStatus_Portal}
}
it should not initialize until really needed/used:
you could use lazy instantiation to do this :
Lazy<Dictionary<string, string>> resources = new Lazy<Dictionary<string, string>>(() => new Dictionary<string, string>(StringComparer.OrdinalIgnoreCase));
Sorting the dictionary would also help you speed up the search, using BinarySearch for instance.
The whole story; I have some KeyValuePairs that I need to store in a session and my primary goal is to keep it small. Therefore I don't have the option of using many different collection. While the key is a different enum value of of a different enum type the value is always just a enum value of the same enum type. I have chosen a HashTable for this approach which content look like this (just many more):
// The Key-Value-Pairs
{ EnumTypA.ValueA1, MyEnum.ValueA },
{ EnumTypB.ValueB1, MyEnum.ValueB },
{ EnumTypC.ValueC1, MyEnum.ValueA },
{ EnumTypA.ValueA2, MyEnum.ValueC },
{ EnumTypB.ValueB1, MyEnum.ValueC }
At most I am running contains on that HashTable but for sure I also need to fetch the value at some point and I need to loop through all elements. That all works fine but now I have a new requirement to keep the order I have added them to the HashTable -> BANG
A HashTable is a map and that is not possible!
Now I thought about using a SortedList<object, MyEnum> or to go with more Data but slightly faster lookups and use a SortedSet<object> in addition to the HashTable.
Content below has been edited
The SortedList is implemented as
SortedList<Enum, MyEnum> mySortedList = new SortedList<Enum, MyEnum>();
the SortedSet is implemented as
SortedSet<Enum> mySortedSet = new SortedSet<Enum>();
The described Key - Value - Pairs are added to the sorted list with
void AddPair(Enum key, MyEnum value)
{
mySortedList.Add(key, value);
}
And for the SortedSett like this
void AddPair(Enum key)
{
mySortedSet.Add(key);
}
Both are failing with the exception:
Object must be the same type as the
enum
My question is: What goes wrong and how can I archive my goal?
Used Solution
I've decided to life with the downside
of redundant data against slower
lookups and decided to implement a
List<Enum> which will retain the
insert order parallel to my already
existing HashTable.
In my case I just have about 50-150
Elements so I decided to benchmark the
Hashtable against the
List<KeyValuePair<object,object>>
Therefore I have create me the
following helper to implement
ContainsKey() to the
List<KeyValuePair<object,object>>
static bool ContainsKey(this List<KeyValuePair<object, object>> list, object key)
{
foreach (KeyValuePair<object, object> p in list)
{
if (p.Key.Equals(key))
return true;
}
return false;
}
I inserted the same 100 Entries and
checked randomly for one of ten
different entries in a 300000 loop.
And... the difference was tiny so I
decided to go with the
List<KeyValuePair<object,object>>
I think you should store your data in an instance of List<KeyValuePair<Enum, MyEnum>> or Dictionary<Enum, MyEnum>.
SortedSet and SortedList are generic, but your keys are EnumTypeA/EnumTypeB, you need to specify the generic T with their base class(System.Enum) like:
SortedList<Enum, MyEnum> sorted = new SortedList<Enum, MyEnum>();
EDIT
Why you got this exception
SortedList and SortedSet use a comparer inside to check if two keys are equal. Comparer<Enum>.Default will be used as the comparer if you didn't specify the comparer in the constructor. Unfortunately Comparer<Enum>.Default isn't implemented as you expected. It throws the exception if the two enums are not the same type.
How to resolve the problem
If you don't want to use a List<KeyValuePair<Enum, MyEnum>> and insist using SortedLIst, you need to specify a comparer to the constructor like this:
class EnumComparer : IComparer<Enum>
{
public int Compare(Enum x, Enum y)
{
return x.GetHashCode() - y.GetHashCode();
}
}
var sorted = new SortedList<Enum, MyEnum>(new EnumComparer());
Btw, I think you need to obtain the "inserting order"? If so, List<KeyValuePair<K,V>> is a better choice, because SortedSet will prevent duplicated items.
I'm using a Dictionary<string, int> where the int is a count of the key.
Now, I need to access the last-inserted Key inside the Dictionary, but I do not know the name of it. The obvious attempt:
int LastCount = mydict[mydict.keys[mydict.keys.Count]];
does not work, because Dictionary.Keys does not implement a []-indexer.
I just wonder if there is any similar class? I thought about using a Stack, but that only stores a string. I could now create my own struct and then use a Stack<MyStruct>, but I wonder if there is another alternative, essentially a Dictionary that implements an []-indexer on the Keys?
As #Falanwe points out in a comment, doing something like this is incorrect:
int LastCount = mydict.Keys.ElementAt(mydict.Count -1);
You should not depend on the order of keys in a Dictionary. If you need ordering, you should use an OrderedDictionary, as suggested in this answer. The other answers on this page are interesting as well.
You can use an OrderedDictionary.
Represents a collection of key/value
pairs that are accessible by the key
or index.
A Dictionary is a Hash Table, so you have no idea the order of insertion!
If you want to know the last inserted key I would suggest extending the Dictionary to include a LastKeyInserted value.
E.g.:
public MyDictionary<K, T> : IDictionary<K, T>
{
private IDictionary<K, T> _InnerDictionary;
public K LastInsertedKey { get; set; }
public MyDictionary()
{
_InnerDictionary = new Dictionary<K, T>();
}
#region Implementation of IDictionary
public void Add(KeyValuePair<K, T> item)
{
_InnerDictionary.Add(item);
LastInsertedKey = item.Key;
}
public void Add(K key, T value)
{
_InnerDictionary.Add(key, value);
LastInsertedKey = key;
}
.... rest of IDictionary methods
#endregion
}
You will run into problems however when you use .Remove() so to overcome this you will have to keep an ordered list of the keys inserted.
Why don't you just extend the dictionary class to add in a last key inserted property. Something like the following maybe?
public class ExtendedDictionary : Dictionary<string, int>
{
private int lastKeyInserted = -1;
public int LastKeyInserted
{
get { return lastKeyInserted; }
set { lastKeyInserted = value; }
}
public void AddNew(string s, int i)
{
lastKeyInserted = i;
base.Add(s, i);
}
}
You could always do this:
string[] temp = new string[mydict.count];
mydict.Keys.CopyTo(temp, 0)
int LastCount = mydict[temp[mydict.count - 1]]
But I wouldn't recommend it. There's no guarantee that the last inserted key will be at the end of the array. The ordering for Keys on MSDN is unspecified, and subject to change. In my very brief test, it does seem to be in order of insertion, but you'd be better off building in proper bookkeeping like a stack--as you suggest (though I don't see the need of a struct based on your other statements)--or single variable cache if you just need to know the latest key.
I think you can do something like this, the syntax might be wrong, havent used C# in a while
To get the last item
Dictionary<string, int>.KeyCollection keys = mydict.keys;
string lastKey = keys.Last();
or use Max instead of Last to get the max value, I dont know which one fits your code better.
I agree with the second part of Patrick's answer. Even if in some tests it seems to keep insertion order, the documentation (and normal behavior for dictionaries and hashes) explicitly states the ordering is unspecified.
You're just asking for trouble depending on the ordering of the keys. Add your own bookkeeping (as Patrick said, just a single variable for the last added key) to be sure. Also, don't be tempted by all the methods such as Last and Max on the dictionary as those are probably in relation to the key comparator (I'm not sure about that).
In case you decide to use dangerous code that is subject to breakage, this extension function will fetch a key from a Dictionary<K,V> according to its internal indexing (which for Mono and .NET currently appears to be in the same order as you get by enumerating the Keys property).
It is much preferable to use Linq: dict.Keys.ElementAt(i), but that function will iterate O(N); the following is O(1) but with a reflection performance penalty.
using System;
using System.Collections.Generic;
using System.Reflection;
public static class Extensions
{
public static TKey KeyByIndex<TKey,TValue>(this Dictionary<TKey, TValue> dict, int idx)
{
Type type = typeof(Dictionary<TKey, TValue>);
FieldInfo info = type.GetField("entries", BindingFlags.NonPublic | BindingFlags.Instance);
if (info != null)
{
// .NET
Object element = ((Array)info.GetValue(dict)).GetValue(idx);
return (TKey)element.GetType().GetField("key", BindingFlags.Public | BindingFlags.Instance).GetValue(element);
}
// Mono:
info = type.GetField("keySlots", BindingFlags.NonPublic | BindingFlags.Instance);
return (TKey)((Array)info.GetValue(dict)).GetValue(idx);
}
};
One alternative would be a KeyedCollection if the key is embedded in the value.
Just create a basic implementation in a sealed class to use.
So to replace Dictionary<string, int> (which isn't a very good example as there isn't a clear key for a int).
private sealed class IntDictionary : KeyedCollection<string, int>
{
protected override string GetKeyForItem(int item)
{
// The example works better when the value contains the key. It falls down a bit for a dictionary of ints.
return item.ToString();
}
}
KeyedCollection<string, int> intCollection = new ClassThatContainsSealedImplementation.IntDictionary();
intCollection.Add(7);
int valueByIndex = intCollection[0];
The way you worded the question leads me to believe that the int in the Dictionary contains the item's "position" on the Dictionary. Judging from the assertion that the keys aren't stored in the order that they're added, if this is correct, that would mean that keys.Count (or .Count - 1, if you're using zero-based) should still always be the number of the last-entered key?
If that's correct, is there any reason you can't instead use Dictionary<int, string> so that you can use mydict[ mydict.Keys.Count ]?
I don't know if this would work because I'm pretty sure that the keys aren't stored in the order they are added, but you could cast the KeysCollection to a List and then get the last key in the list... but it would be worth having a look.
The only other thing I can think of is to store the keys in a lookup list and add the keys to the list before you add them to the dictionary... it's not pretty tho.
To expand on Daniels post and his comments regarding the key, since the key is embedded within the value anyway, you could resort to using a KeyValuePair<TKey, TValue> as the value. The main reasoning for this is that, in general, the Key isn't necessarily directly derivable from the value.
Then it'd look like this:
public sealed class CustomDictionary<TKey, TValue>
: KeyedCollection<TKey, KeyValuePair<TKey, TValue>>
{
protected override TKey GetKeyForItem(KeyValuePair<TKey, TValue> item)
{
return item.Key;
}
}
To use this as in the previous example, you'd do:
CustomDictionary<string, int> custDict = new CustomDictionary<string, int>();
custDict.Add(new KeyValuePair<string, int>("key", 7));
int valueByIndex = custDict[0].Value;
int valueByKey = custDict["key"].Value;
string keyByIndex = custDict[0].Key;
A dictionary may not be very intuitive for using index for reference but, you can have similar operations with an array of KeyValuePair:
ex.
KeyValuePair<string, string>[] filters;
You can also use SortedList and its Generic counterpart. These two classes and in Andrew Peters answer mentioned OrderedDictionary are dictionary classes in which items can be accessed by index (position) as well as by key. How to use these classes you can find: SortedList Class , SortedList Generic Class .
Visual Studio's UserVoice gives a link to generic OrderedDictionary implementation by dotmore.
But if you only need to get key/value pairs by index and don't need to get values by keys, you may use one simple trick. Declare some generic class (I called it ListArray) as follows:
class ListArray<T> : List<T[]> { }
You may also declare it with constructors:
class ListArray<T> : List<T[]>
{
public ListArray() : base() { }
public ListArray(int capacity) : base(capacity) { }
}
For example, you read some key/value pairs from a file and just want to store them in the order they were read so to get them later by index:
ListArray<string> settingsRead = new ListArray<string>();
using (var sr = new StreamReader(myFile))
{
string line;
while ((line = sr.ReadLine()) != null)
{
string[] keyValueStrings = line.Split(separator);
for (int i = 0; i < keyValueStrings.Length; i++)
keyValueStrings[i] = keyValueStrings[i].Trim();
settingsRead.Add(keyValueStrings);
}
}
// Later you get your key/value strings simply by index
string[] myKeyValueStrings = settingsRead[index];
As you may have noticed, you can have not necessarily just pairs of key/value in your ListArray. The item arrays may be of any length, like in jagged array.