Is there a way to fill an array via a SqlDataReader (or any other C# ADO.NET object) without looping through all the items? I have a query that is returning a single column, and I want to put that into a string array (or ArrayList, or List, etc).
It is possible. In .NET 2.0+, SqlDataReader inherits from DbDataReader, which implements IEnumerable (non-generic one). This means that you can use LINQ:
List<string> list = (from IDataRecord r in dataReader
select (string)r["FieldName"]
).ToList();
That said, the loop is still there, it's just hidden in Enumerable.Select, rather than being explicit in your code.
No, since SqlDataReader is a forward-only read-only stream of rows from a SQL Server database, the stream of rows will be looped through whether explicitly in your code or hidden in a framework implementation (such as DataTable's Load method).
It sounds like using a generic list and then returning the list as an array would be a good option. For example,
List<int> list = new List<int>();
using (SqlDataReader reader = cmd.ExecuteReader())
{
while (reader.Read())
{
list.Add(reader.GetInt32(0));
}
}
return list.ToArray();
In response to your comment, calling ToArray() may be overhead, it depends. Do you need an array of objects to work with or would a generic collection (such as List<T> or ReadOnlyCollection<T>) be more useful?
Apparently, ever since .NET 1.1 SqlDataReader had the following method:
int size;
object[] data = new object[]{};
size = reader.GetValues(data);
This populates data with the values of the current reader row, assigning into size the number of objects that were put into the array.
Since any IDataReader implementation (SqlDataReader included) will be a forward-only reader by definition, no there is no way to do this without looping. Even if there were a framework library method to do this it would have to loop through the reader, just like you would.
The orignial OP asked for Array, ArrayList or List. You can return Array as well. Just call the .ToArray() method and assign it to a previously declared array. Arrays are very fast when it comes to enumerating each element. Much faster than a List if the list has more than 1000 elements.
You can return to Array, List, or Dictionary.
ids_array = (from IDataRecord r in idReader
select (string)r["ID"]).ToArray<string>();
Additionally, if you are using a lookup of keys for example, you might consider creating a HashSet object with has excellent lookup performance if you are simply checking one list against another to determine if an elements key exists in the HashSet object.
example:
HashSet<string> hs = new HashSet<string>(
(from IDataRecord r in idReader select (string)r["ID"]).AsEnumerable<string>() );
You have to loop, but there are projects that can make it simpler. Also, try not to use ArrayList, use List instead.
You can checkout FluentAdo for one: http://fluentado.codeplex.com
public IList<UserAccount> List()
{
var list = new FluentCommand<UserAccount>("SELECT ID, UserName, Password FROM UserAccount")
.SetMap(reader => new UserAccount
{
ID = reader.GetInt("ID"),
Password = reader.GetString("Password"),
UserName = reader.GetString("UserName"),
})
.AsList();
return list;
}
If you read your SqlDataAdapter into a DataTable:
DataTable dt as DataTable;
dt.fill(data);
Then you can use some of the toys in System.Data.DataSetExtensions as referenced in Joel Muller's answer to this question.
In uses a bit of Linq, so you will net .Net 3.5 or higher.
var array = reader.GetValue("field_name") as long[];
Related
I have a file with 2 columns and multiple rows. 1st column is ID, 2nd column is Name. I would like to display a Dropdown where I will show only all the names from this file.
I will only iterate through the collection. So what is the better approach? Is creating the objects more readable for other developers? Or maybe creating new objects is too slow and it's not worth.
while (!reader.EndOfStream)
{
var row = reader.ReadLine();
var values = row.Split(' ');
list.Add(new Object { Id = int.Parse(values[0]), Name = values[1] });
}
or
while (!reader.EndOfStream)
{
var row = reader.ReadLine();
var values = row.Split(' ');
dict.Add(int.Parse(values[0]), values[1]);
}
Do I lose the speed in the case if I will create new objects?
You create new objects, so to speak, also while adding to the Dictionary<T>, you create new Key-Value pair of the desired type.
As you already mentioned in your question, the decision is made on primary
expected access pattern
performance considerations, which are the function also of access pattern per se.
If you need read-only array to iterate over, use List<T> (even better if the size of the data is known upfront use Type[] data, and just read it where you need it).
If you need key-wise access to your data, use Dictionary<T>.
If you want to only iterate objects, then use List. No need to use Dictionary class at all.
I have a datatable which I have filterd using Linq:
result = myDataTable.AsEnumerable().Where(row => row.Field<string>("id").Contains(values));
I have also tried using CopyToDataTable methods like
result.CopyToDataTable()
and
result.CopyToDataTable<DataRow>()
but they didn't work.
How can I convert result to new DataTable?
I have search many Stack Overflow questions and many other tutorials but I can't find what I want.
UPDATE
I have concatenated HashSet to comma separated values, I think I should use array of String or HashSet?
I suggest you create object of DataTable and import row in it by calling ImportRow() function , that will resolve issue.
DataTable.ImportRow Method
Example code.
DataTable tblClone = datTab.Clone();
foreach (DataRow datRow in datTab.Rows)
{
tblClone.ImportRow(datRow);
}
for you it will be like
var result = myDataTable.AsEnumerable().Where(row => row.Field<string>("id").Contains(values));
DataTable tblClone = myDataTable.Clone();
foreach(DataRow dr in result)
tblClone.ImportRow(dr);
.CopyToDataTable<DataRow>() returns a DataTable, it will not modify the variable unless you re-assign it.
result = myDataTable.AsEnumerable().Where(row => row.Field<string>("id").Contains(values));
Then you actually need a DataTable object.
DataTable resultDT = result.CopyToDataTable<DataRow>();
Edit: As Tim pointed out, if no rows are returned by your query, an exception will be thrown "The source contains no DataRows"
You could do something like so;
DataTable resultDT = result.Any() ? result.CopyToDataTable<DataRow>() : myDataTable.Clone();
But that will run the query twice (also as Tim pointed out).
Therefore you could convert that to a list object using (.ToList()), check the count and do your processing then. That has performance implications in such that you create a new instance of the object (List object).
Doing a try/catch with attempt to convert it to DataTable also isn't a good idea. See Pranays answer for another great way to achieve the final result.
while (Running)
{
while (rdr.Read())
{
List<PackageDetailFile> pkgFiles = rdr.Read().ToString().ToList();
}
}
Hello, the read() returns an inner joined table which has 7 8 columns, is it possible to store them inside List ?
if we can hows the correct casting of rdr.read(), ( to string to list ... ? )
Depending on the structure of your PackageDetailFile class and the underlying data you could do something like this
var packageDetailFiles = new List<PackageDetailFile>();
while (Running)
{
while (rdr.Read())
{
var detailFile = new PackageDetailFile();
// the following will depend on your data structure and class properties
detailFile.Property1 = rdr.GetString(0);
detailFile.Property2 = rdr.GetString(1);
detailFile.Property3 = rdr.GetString(2);
packageDetailFiles.Add(detailFile);
}
}
Note: You can use assemblies like AutoMapper to map from a data reader to a POCO reducing the need for the tedious MyObject.MyProperty = dataReader[collectionIndex] making your code far more readable and test friendly.
try use GetValues method of rdr object. I suppose rdr is a SqlDataReader or another DataReader.
This method return an object array with all columns of the current row.
After it you can store the object array in a list of objects. If you want to store in another type of list you must cast every element to the correct type before storing it.
i have an array of custom objects. i'd like to be able to reference this array by a particular data member, for instance myArrary["Item1"]
"Item1" is actually the value stored in the Name property of this custom type and I can write a predicate to mark the appropriate array item. However I am unclear as to how to let the array know i'd like to use this predicate to find the array item.
I'd like to just use a dictionary or hashtable or NameValuePair for this array, and get around this whole problem but it's generated and it must remain as CustomObj[]. i'm also trying to avoid loading a dictionary from this array as it's going to happen many times and there could be many objects in it.
For clarification
myArray[5] = new CustomObj() // easy!
myArray["ItemName"] = new CustomObj(); // how to do this?
Can the above be done? I'm really just looking for something similar to how DataRow.Columns["MyColumnName"] works
Thanks for the advice.
What you really want is an OrderedDictionary. The version that .NET provides in System.Collections.Specialized is not generic - however there is a generic version on CodeProject that you could use. Internally, this is really just a hashtable married to a list ... but it is exposed in a uniform manner.
If you really want to avoid using a dictionary - you're going to have to live with O(n) lookup performance for an item by key. In that case, stick with an array or list and just use the LINQ Where() method to lookup a value. You can use either First() or Single() depending on whether duplicate entries are expected.
var myArrayOfCustom = ...
var item = myArrayOfCustom.Where( x => x.Name = "yourSearchValue" ).First();
It's easy enough to wrap this functionality into a class so that external consumers are not burdened by this knowledge, and can use simple indexers to access the data. You could then add features like memoization if you expect the same values are going to be accessed frequently. In this way you could amortize the cost of building the underlying lookup dictionary over multiple accesses.
If you do not want to use "Dictionary", then you should create class "myArrary" with data mass storage functionality and add indexers of type "int" for index access and of type "string" for associative access.
public CustomObj this [string index]
{
get
{
return data[searchIdxByName(index)];
}
set
{
data[searchIdxByName(index)] = value;
}
}
First link in google for indexers is: http://www.csharphelp.com/2006/04/c-indexers/
you could use a dictionary for this, although it might not be the best solution in the world this is the first i came up with.
Dictionary<string, int> d = new Dictionary<string, int>();
d.Add("cat", 2);
d.Add("dog", 1);
d.Add("llama", 0);
d.Add("iguana", -1);
the ints could be objects, what you like :)
http://dotnetperls.com/dictionary-keys
Perhaps OrderedDictionary is what you're looking for.
you can use HashTable ;
System.Collections.Hashtable o_Hash_Table = new Hashtable();
o_Hash_Table.Add("Key", "Value");
There is a class in the System.Collections namespace called Dictionary<K,V> that you should use.
var d = new Dictionary<string, MyObj>();
MyObj o = d["a string variable"];
Another way would be to code two methods/a property:
public MyObj this[string index]
{
get
{
foreach (var o in My_Enumerable)
{
if (o.Name == index)
{
return o;
}
}
}
set
{
foreach (var o in My_Enumerable)
{
if (o.Name == index)
{
var i = My_Enumerable.IndexOf(0);
My_Enumerable.Remove(0);
My_Enumerable.Add(value);
}
}
}
}
I hope it helps!
It depends on the collection, some collections allow accessing by name and some don't. Accessing with strings is only meaningful when the collection has data stored, the column collection identifies columns by their name, thus allowing you to select a column by its name. In a normal array this would not work because items are only identified by their index number.
My best recommendation, if you can't change it to use a dictionary, is to either use a Linq expression:
var item1 = myArray.Where(x => x.Name == "Item1").FirstOrDefault();
or, make an extension method that uses a linq expression:
public static class CustomObjExtensions
{
public static CustomObj Get(this CustomObj[] Array, string Name)
{
Array.Where(x => x.Name == Name).FirstOrDefault();
}
}
then in your app:
var item2 = myArray.Get("Item2");
Note however that performance wouldn't be as good as using a dictionary, since behind the scenes .NET will just loop through the list until it finds a match, so if your list isn't going to change frequently, then you could just make a Dictionary instead.
I have two ideas:
1) I'm not sure you're aware but you can copy dictionary objects to an array like so:
Dictionary dict = new Dictionary();
dict.Add("tesT",40);
int[] myints = new int[dict.Count];
dict.Values.CopyTo(myints, 0);
This might allow you to use a Dictionary for everything while still keeping the output as an array.
2) You could also actually create a DataTable programmatically if that's the exact functionality you want:
DataTable dt = new DataTable();
DataColumn dc1 = new DataColumn("ID", typeof(int));
DataColumn dc2 = new DataColumn("Name", typeof(string));
dt.Columns.Add(dc1);
dt.Columns.Add(dc2);
DataRow row = dt.NewRow();
row["ID"] = 100;
row["Name"] = "Test";
dt.Rows.Add(row);
You could also create this outside of the method so you don't have to make the table over again every time.
Is there an easy way to convert all the columns of the current row of a SqlDataReader to a dictionary?
using (SqlDataReader opReader = command.ExecuteReader())
{
// Convert the current row to a dictionary
}
Thanks
You can use LINQ:
return Enumerable.Range(0, reader.FieldCount)
.ToDictionary(reader.GetName, reader.GetValue);
Easier than this?:
// Need to read the row in, usually in a while ( opReader.Read ) {} loop...
opReader.Read();
// Convert current row into a dictionary
Dictionary<string, object> dict = new Dictionary<string, object>();
for( int lp = 0 ; lp < opReader.FieldCount ; lp++ ) {
dict.Add(opReader.GetName(lp), opReader.GetValue(lp));
}
I'm still not sure why you would need this particular transformation from one type of collection to another.
I came across this question on 3/9/2016 and ended up using the answer provided by SLaks. However, I needed to slightly modify it to:
dataRowDictionary = Enumerable.Range(0, reader.FieldCount).ToDictionary(i => reader.GetName(i), i=> reader.GetValue(i).ToString());
I found guidance from this StackOverflow question: convert dataReader to Dictionary
It's already an IDataRecord.
That should give you just about the same access (by key) as a dictionary. Since rows don't typically have more than a few handfuls of columns, the performance of the lookups shouldn't be that different. The only important difference is the type of the "payload", and even there your dictionary would have to use object for the value type, so I give the edge to IDataRecord.
GetValues method accepts & puts in, all the values in a 1D array.
Does that help?