Develop Reference

C# Populate dictionary directly from SqlDataReader

In a program that I've been working on, there are three steps to get the data into a Dictionary that's been created:
execute the SQL command
pull those results into a DataTable, then
pull the DataTable into the Dictionary
Code:
var myDr = myLookup.ExecuteReader();
dt.Load(myDr);
customerLookup = dt.AsEnumerable()
.ToDictionary(key => key.Field<string>("code"),
value => value.Field<string>("customerText"));
My question is, is it possible to “cut out the middleman,” so to speak, and pull the data from the SqlDataReater directly into the Dictionaries? Or is it necessary to pull it into a DataTable first? If what I'm looking to do is possible, can someone please post code for me to try?
Thanks very much!

You can just loop through the rows returned by the reader:
var customerLookup = new Dictionary<string, string>();
using (var reader = myLookup.ExecuteReader())
{
while (reader.Read())
{
customerLookup[(string)reader["code"]] = (string)reader["customerText"];
}
}
You should be aware that if there are any duplicate codes, subsequent code values will overwrite previous ones in the dictionary. You can use customerLookup.Add() instead if you'd rather an exception be thrown in such a case.

Not only can you, but you definitely should. The code, as you show it, shows a complete lack of knowledge of how .NET works.
Some of the code I am suggesting may be considered "overkill" for the question at hand, but it does demonstrate some best practices.
Dictionary<string, string> customerLookup = new Dictionary<string, string>();
using (var reader = myLookup.ExecuteReader())
{
int ordinalCode = reader.GetOrdinal("code");
int ordinalCustomerText = reader.GetOrdinal("customerText");
while (reader.Read())
{
//this code assumes the values returned by the reader cannot be null
customerLookup.Add(reader.GetString(ordinalCode), reader.GetString(ordinalCustomerText))
}
}

Yes, it's possible. You should use SqlDataReader.Read method.

Why do I get a Unable to cast object of type 'WhereSelectEnumerableIterator for the following code?

I get an 'Unable to cast object of type 'WhereSelectEnumerableIterator'' when I try to convert one Dictionary type to another from within a Select. I don't see any obvious reasons. Please help.
void Main()
{
Dictionary<string,Dictionary<string,string>> test = new Dictionary<string,Dictionary<string,string>>();
Dictionary<string,string> inner = new Dictionary<string,string>();
for(int n = 0; n< 10; n++)
{
var data = n.ToString();
inner[data] = data;
}
for(int n2 = 0; n2 < 10; n2++)
{
test[Guid.NewGuid().ToString()] = inner;
}
// test conversion
var output = (Dictionary<Guid,Dictionary<string,string>>) test.Select(x => new KeyValuePair<Guid, Dictionary<string,string>>(new Guid(x.Key), x.Value));
}

You have an IEnumerable<Guid,Dictionary<string,string>>. You apparently want a Dictionary<Guid,Dictionary<string,string>>. When you try to assign one to the other, you get a compile time error telling you that your IEnumerable isn't a Dictionary, it's just an IEnumerable. You provided a cast, which is a way of saying, "Sorry compiler, but you're wrong, I know better than you; this IEnumerable is in fact a dictionary, under the hood." Sadly, this is not the case here. You didn't actually have a Dictionary, you had a WhereSelectEnumerableIterator, which isn't a Dictionary.
There are any number of ways you have of creating a dictionary, one of which is to use ToDictionary:
var output = test.ToDictionary(x => new Guid(x.Key), x => x.Value);
On an unrelated note, you've created just one inner dictionary, and set it as the value of every single key you create. You might be under the impression that you've copied this dictionary 10 times. You have not. There only ever is one Dictionary<string, string> here; you just have 10 different GUIDs all pointing to that one dictionary. You'll need to create a bunch of different dictionaries if you want these keys to actually point to different dictionaries.

Remove a single value from a NameValueCollection

My data source could have duplicate keys with values.
typeA : 1
typeB : 2
typeA : 11
I chose to use NameValueCollection as it enables entering duplicate keys.
I want to remove specific key\value pair from the collection, but NameValueCollection.Remove(key) removes all values associated with the specified key.
Is there a way to remove single key\value pair from a NameValueCollection,
OR
Is there a better collection in C# that fits my data
[EDIT 1]
First, thanks for all the answers :)
I think I should have mentioned that my data source is XML.
I used System.Xml.Linq.XDocument to query for type and also it was handy to remove a particular value.
Now, my question is, for large size data, is using XDocument a good choice considering the performance?
If not what are other alternatives (maybe back to NameValueCollection and using one of the techniques mentioned to remove data)

The idea of storing multiple values with the same key is somehow strange. But I think you can retrieve all values using GetValues then remove the one you don't need and put them back using Set and then subsequent Add methods. You can make a separate extension method method for this.

NameValueCollection doesn't really allow to have multiple entries with the same key. It merely concatenates the new values of existing keys into a comma separated list of values (see NameValueCollection.Add.
So there really is just a single value per key. You could conceivably get the value split them on ',' and remove the offending value.
Edit: #ElDog is correct, there is a GetValues method which does this for you so no need to split.
A better option I think would be to use Dictionary<string, IList<int>> or Dictionary<string, ISet<int>> to store the values as discrete erm, values

You may convert it to Hashtable
var x = new NameValueCollection();
x.Add("a", "1");
x.Add("b", "2");
x.Add("a", "1");
var y = x.AllKeys.ToDictionary(k => k, k=>x[k]);

make your own method, it works for me --
public static void Remove<TKey,TValue>(
this List<KeyValuePair<TKey,TValue>> list,
TKey key,
TValue value) {
return list.Remove(new KeyValuePair<TKey,TValue>(key,value));
}
then call it on list as --
list.Remove(key,value); //Pass the key value...

Perhaps not the best way, but....
public class SingleType
{
public string Name;
public int Value;
}
List<SingleType> typeList = new List<SingleType>();
typeList.Add (new SingleType { Name = "TypeA", Value = 1 });
typeList.Add (new SingleType { Name = "TypeA", Value = 3 });
typeList.Remove (typeList.Where (t => t.Name == "TypeA" && t.Value == 1).Single());

You can use the Dictionary collection instead:
Dictionary<string, int> dictionary = new Dictionary<string, int>();
dictionary.Add("typeA", 1);
dictionary.Add("typeB", 1);
When you try to insert type: 11 it will throw exception as Key already exists. So you can enter a new key to insert this data.
Refer this Tutorial for further help.

how to add an associative index to an array. c#

i have an array of custom objects. i'd like to be able to reference this array by a particular data member, for instance myArrary["Item1"]
"Item1" is actually the value stored in the Name property of this custom type and I can write a predicate to mark the appropriate array item. However I am unclear as to how to let the array know i'd like to use this predicate to find the array item.
I'd like to just use a dictionary or hashtable or NameValuePair for this array, and get around this whole problem but it's generated and it must remain as CustomObj[]. i'm also trying to avoid loading a dictionary from this array as it's going to happen many times and there could be many objects in it.
For clarification
myArray[5] = new CustomObj() // easy!
myArray["ItemName"] = new CustomObj(); // how to do this?
Can the above be done? I'm really just looking for something similar to how DataRow.Columns["MyColumnName"] works
Thanks for the advice.

What you really want is an OrderedDictionary. The version that .NET provides in System.Collections.Specialized is not generic - however there is a generic version on CodeProject that you could use. Internally, this is really just a hashtable married to a list ... but it is exposed in a uniform manner.
If you really want to avoid using a dictionary - you're going to have to live with O(n) lookup performance for an item by key. In that case, stick with an array or list and just use the LINQ Where() method to lookup a value. You can use either First() or Single() depending on whether duplicate entries are expected.
var myArrayOfCustom = ...
var item = myArrayOfCustom.Where( x => x.Name = "yourSearchValue" ).First();
It's easy enough to wrap this functionality into a class so that external consumers are not burdened by this knowledge, and can use simple indexers to access the data. You could then add features like memoization if you expect the same values are going to be accessed frequently. In this way you could amortize the cost of building the underlying lookup dictionary over multiple accesses.

If you do not want to use "Dictionary", then you should create class "myArrary" with data mass storage functionality and add indexers of type "int" for index access and of type "string" for associative access.
public CustomObj this [string index]
{
get
{
return data[searchIdxByName(index)];
}
set
{
data[searchIdxByName(index)] = value;
}
}
First link in google for indexers is: http://www.csharphelp.com/2006/04/c-indexers/

you could use a dictionary for this, although it might not be the best solution in the world this is the first i came up with.
Dictionary<string, int> d = new Dictionary<string, int>();
d.Add("cat", 2);
d.Add("dog", 1);
d.Add("llama", 0);
d.Add("iguana", -1);
the ints could be objects, what you like :)
http://dotnetperls.com/dictionary-keys

Perhaps OrderedDictionary is what you're looking for.

you can use HashTable ;
System.Collections.Hashtable o_Hash_Table = new Hashtable();
o_Hash_Table.Add("Key", "Value");

There is a class in the System.Collections namespace called Dictionary<K,V> that you should use.
var d = new Dictionary<string, MyObj>();
MyObj o = d["a string variable"];
Another way would be to code two methods/a property:
public MyObj this[string index]
{
get
{
foreach (var o in My_Enumerable)
{
if (o.Name == index)
{
return o;
}
}
}
set
{
foreach (var o in My_Enumerable)
{
if (o.Name == index)
{
var i = My_Enumerable.IndexOf(0);
My_Enumerable.Remove(0);
My_Enumerable.Add(value);
}
}
}
}
I hope it helps!

It depends on the collection, some collections allow accessing by name and some don't. Accessing with strings is only meaningful when the collection has data stored, the column collection identifies columns by their name, thus allowing you to select a column by its name. In a normal array this would not work because items are only identified by their index number.

My best recommendation, if you can't change it to use a dictionary, is to either use a Linq expression:
var item1 = myArray.Where(x => x.Name == "Item1").FirstOrDefault();
or, make an extension method that uses a linq expression:
public static class CustomObjExtensions
{
public static CustomObj Get(this CustomObj[] Array, string Name)
{
Array.Where(x => x.Name == Name).FirstOrDefault();
}
}
then in your app:
var item2 = myArray.Get("Item2");
Note however that performance wouldn't be as good as using a dictionary, since behind the scenes .NET will just loop through the list until it finds a match, so if your list isn't going to change frequently, then you could just make a Dictionary instead.

I have two ideas:
1) I'm not sure you're aware but you can copy dictionary objects to an array like so:
Dictionary dict = new Dictionary();
dict.Add("tesT",40);
int[] myints = new int[dict.Count];
dict.Values.CopyTo(myints, 0);
This might allow you to use a Dictionary for everything while still keeping the output as an array.
2) You could also actually create a DataTable programmatically if that's the exact functionality you want:
DataTable dt = new DataTable();
DataColumn dc1 = new DataColumn("ID", typeof(int));
DataColumn dc2 = new DataColumn("Name", typeof(string));
dt.Columns.Add(dc1);
dt.Columns.Add(dc2);
DataRow row = dt.NewRow();
row["ID"] = 100;
row["Name"] = "Test";
dt.Rows.Add(row);
You could also create this outside of the method so you don't have to make the table over again every time.

Fill an array (or arraylist) from SqlDataReader

Is there a way to fill an array via a SqlDataReader (or any other C# ADO.NET object) without looping through all the items? I have a query that is returning a single column, and I want to put that into a string array (or ArrayList, or List, etc).

It is possible. In .NET 2.0+, SqlDataReader inherits from DbDataReader, which implements IEnumerable (non-generic one). This means that you can use LINQ:
List<string> list = (from IDataRecord r in dataReader
select (string)r["FieldName"]
).ToList();
That said, the loop is still there, it's just hidden in Enumerable.Select, rather than being explicit in your code.

No, since SqlDataReader is a forward-only read-only stream of rows from a SQL Server database, the stream of rows will be looped through whether explicitly in your code or hidden in a framework implementation (such as DataTable's Load method).
It sounds like using a generic list and then returning the list as an array would be a good option. For example,
List<int> list = new List<int>();
using (SqlDataReader reader = cmd.ExecuteReader())
{
while (reader.Read())
{
list.Add(reader.GetInt32(0));
}
}
return list.ToArray();
In response to your comment, calling ToArray() may be overhead, it depends. Do you need an array of objects to work with or would a generic collection (such as List<T> or ReadOnlyCollection<T>) be more useful?

Apparently, ever since .NET 1.1 SqlDataReader had the following method:
int size;
object[] data = new object[]{};
size = reader.GetValues(data);
This populates data with the values of the current reader row, assigning into size the number of objects that were put into the array.

Since any IDataReader implementation (SqlDataReader included) will be a forward-only reader by definition, no there is no way to do this without looping. Even if there were a framework library method to do this it would have to loop through the reader, just like you would.

The orignial OP asked for Array, ArrayList or List. You can return Array as well. Just call the .ToArray() method and assign it to a previously declared array. Arrays are very fast when it comes to enumerating each element. Much faster than a List if the list has more than 1000 elements.
You can return to Array, List, or Dictionary.
ids_array = (from IDataRecord r in idReader
select (string)r["ID"]).ToArray<string>();
Additionally, if you are using a lookup of keys for example, you might consider creating a HashSet object with has excellent lookup performance if you are simply checking one list against another to determine if an elements key exists in the HashSet object.
example:
HashSet<string> hs = new HashSet<string>(
(from IDataRecord r in idReader select (string)r["ID"]).AsEnumerable<string>() );

You have to loop, but there are projects that can make it simpler. Also, try not to use ArrayList, use List instead.
You can checkout FluentAdo for one: http://fluentado.codeplex.com
public IList<UserAccount> List()
{
var list = new FluentCommand<UserAccount>("SELECT ID, UserName, Password FROM UserAccount")
.SetMap(reader => new UserAccount
{
ID = reader.GetInt("ID"),
Password = reader.GetString("Password"),
UserName = reader.GetString("UserName"),
})
.AsList();
return list;
}

If you read your SqlDataAdapter into a DataTable:
DataTable dt as DataTable;
dt.fill(data);
Then you can use some of the toys in System.Data.DataSetExtensions as referenced in Joel Muller's answer to this question.
In uses a bit of Linq, so you will net .Net 3.5 or higher.

var array = reader.GetValue("field_name") as long[];