I'm building a console application on visual studio that involves some SQL. Basically, I have a database table with a large amount of columns, and I need set them as variables and have a way of addressing them efficiently. Right now I'm thinking an array which I can then loop through. Here's sort of what my code looks like right now:
SqlCommand getColumns = new SqlCommand("SELECT * FROM tableName",
connection1);
myReader = getColumns.ExecuteReader();
while (myReader.Read())
{
string[] array = new string { myReader["ColumnName"].ToString(),
myReader["ColumnName2"].ToString, etc...};
for (int i = 0; i <= array.length; i++) {
some action...array[i];
}
}
I admit I'm somewhat new to c# and this kind of development, so let me know if there's a more efficient way to do this or if I need to post more code. Thanks!
You do not need custom type for this. .NET already gives you DataSet and DataTable. You need DataTable since it's just one result set, and results look just as same as you see in SSMS results window.
var dt = new DataTable();
dt.Load(myReader);
If you have a lot of columns, accessing them by index instead of name would be more efficient:
string[] array = new string[myReader.FieldCount];
for(int i=0; i<myReader.FieldCount; i++)
{
array[i] = myReader[i].ToString();
}
Also, if you know the number of columns in advance (which is usually the case), you could move array declaration out of while(myReader.Read()), resulting in less memory allocation due to reusing same array.
Another suggestion is that you might not want to convert everything to string... Declare array of objects and handle each data type as you wish. Integer probably should be assined to some entity's integer property instead of saving in string array.
Personally I would declare some entity and declare a List of those. Then populate the list from database:
public class MyEntity
{
public string Name { get; set; }
public int Age { get; set; }
//Other properties
}
//...
SqlCommand getColumns = new SqlCommand("SELECT * FROM tableName",
connection1);
var myDataFromTable = new List<MyEntity>();
myReader = getColumns.ExecuteReader();
while (myReader.Read())
{
myDataFromTable.Add(new MyEntity {
Name = myReader[0] as string,
Age = (int)myReader[1]
//...
});
}
//Process your list of entities here
Another approach may be not to save everything into memory (as you do now), but process data on the fly - everything depends on your requirements and the size of data set. But anyway I recommend parsing data to some entity and work with that.
As per my understanding - On way of doing it.
We can create a class containing properties similar to that of columns in table. In the while loop you can create object of that class, then set the properties with the respected columns and add that object to Arraylist (Arraylist containing the class objects ).
Reference Link : https://msdn.microsoft.com/en-us/library/system.collections.arraylist(v=vs.110).aspx
Hope its helps.
Related
I have some problem with my code. I want to replace the ForEach loop with the help of LINQ here, is there any way or solution to solve my problem? My code is given bellow.
static public string table2Json(DataSet ds, int table_no)
{
try
{
object[][] tb = new object[ds.Tables[table_no].Rows.Count][];
int r = 0;
foreach (DataRow dr in ds.Tables[table_no].Rows)
{
tb[r] = new object[ds.Tables[table_no].Columns.Count];
int col = 0;
foreach (DataColumn column in ds.Tables[table_no].Columns)
{
tb[r][col] = dr[col];
if ((tb[r][col]).Equals(System.DBNull.Value))
{
tb[r][col] = "";
}
col++;
}
r++;
}
string table = JsonConvert.SerializeObject(tb, Formatting.Indented);
return table;
}
catch (Exception ex)
{
tools.log(ex.Message);
throw ex;
}
}
This question really asks 3 different things:
how to serialize a DataTable
how to change the DataTable serialization format and finally
how to replace nulls with empty strings, even though an empty string isn't a NULL.
JSON.NET already handles DataSet and DataTable instance serialization with a DataTableConverter whose source can be found here. You could just write :
var str = JsonConvert.SerializeObject(data);
Given this DataTable :
var dataTable=new DataTable();
dataTable.Columns.Add("Name",typeof(string));
dataTable.Columns.Add("SurName",typeof(string));
dataTable.Rows.Add("Moo",null);
dataTable.Rows.Add("AAA","BBB");
You get :
[{"Name":"Moo","SurName":null},{"Name":"AAA","SurName":"BBB"}]
DataTables aren't 2D arrays and the column names and types matter. Generating a separate row object with named fields is far better than generating an object[] array. It also allows makes it far easier for clients to handle the JSON string without knowing its schema in advance. With an object[] for each row, the clients will have to know what's stored in each location in advance.
If you want to use a different serialization format, you could customize the DataTableConverter. Another option though, is to use DataRow.ItemArray to get the values as an object[] and LINQ to get the rows, eg :
object[][] values=dataTable.Rows.Cast<DataRow>()
.Select(row=>row.ItemArray)
.ToArray();
Serializing this produces :
[["Moo",null],["AAA","BBB"]]
And there's no way to tell which item is the name and which is the surname any more.
Replacing DBNulls with strings in this last form needs an extra Select() to replace DBNull.Value with "" :
object[][] values=dataTable.Rows.Cast<DataRow>()
.Select(row=>row.ItemArray
.Select(x=>x==DBNull.Value?"":x)
.ToArray())
.ToArray();
Serializing this produces :
[["Moo",""],["AAA","BBB"]]
That's what was asked, but now we have no way to tell whether the Surname is an empty string, or just doesn't exist.
This may sound strange, but Arabic names may be one long name without surname. Makes things interesting for airlines or travel agents that try to issue tickets (ask me how I know).
We can get rid of ToArray() if we use var :
var values=dataTable.Rows.Cast<DataRow>()
.Select(row=>row.ItemArray
.Select(x=>x==DBNull.Value?"":x));
JSON serialization will work the same.
LINQ is not a nice fit for this sort of thing because you are using explicit indexes r and col into multiple "array structures" (and there is no easy/tidy way to achieve multiple, parallel enumeration).
Other issues
tb is repeatedly newed, filled with data and then replaced in the next iteration, so you end up capturing only the last row of input to the JSON string - that's a logical bug and won't work as I think you intend.
The inner foreach loop declares but does not use the iteration variable column - that's not going to break anything but it is redundant.
You will get more mileage out of using JSON.Net properly (or coding the foreach loops as for loops instead if you want to navigate the structures yourself).
Beginner programmer here so please keep (explanation of) answers as simple as possible.
For an assignment we got a text file that contains a large amount of lines.
Each line is a different 'Order' with an ordernumber, location, frequency (amount of times per week), and a few other arguments.
Example:
18; New York; 3PWK; ***; ***; etc
I've made it so that each line is read and stored in a string with
string[] orders = System.IO.File.ReadAllLines(#<filepath here>);
And I've made a separate class called "Order" which has a get+set for all the properties like ordernumber, etc.
Now I'm stuck on how to actually get the values in there. I know how string splitting works but not how I can make unique objects in a loop.
I'd need something that created a new Order for every line and then assign Order.ordernumber, Order.location etc.
Any help would be MUCH appreciated!
An easy approach will be to make a class to define the orders like this:
public class Order{
public string OrderNumber{get;set;}
public string OrderId{get;set;}
public string OrderSomeThingElse{get;set;}
}
Then initialize a List:
var orderList = new List<Order>();
Then loop through and populate it:
foreach( var order in orders ){
var splitString = order.Split(';');
orderList.Add( new Order{
OrderNumber = splitString[0],
OrderId = splitString[1],
OrderSomeThingElse = splitString[2]
});
}
If you want an easy, but not that elegant approach, this is it.
In addition to all the good answers you've already received. I recommend you to use File.ReadLines() instead File.ReadAllLines() Because you are reading large file.
The ReadLines and ReadAllLines methods differ as follows: When you use ReadLines, you can start enumerating the collection of strings before the whole collection is returned; when you use ReadAllLines, you must wait for the whole array of strings be returned before you can access the array. Therefore, when you are working with very large files, ReadLines can be more efficient. MSDN
Unless I misunderstand... do you mean something like this?
var ordersCollection = new List<Order>();
foreach (var order in orders)
{
var o = new Order();
o.PropertyName = ""; // Assign your property values like this
ordersCollection.Add(o);
}
// ordersCollection is now full of your orders.
while (Running)
{
while (rdr.Read())
{
List<PackageDetailFile> pkgFiles = rdr.Read().ToString().ToList();
}
}
Hello, the read() returns an inner joined table which has 7 8 columns, is it possible to store them inside List ?
if we can hows the correct casting of rdr.read(), ( to string to list ... ? )
Depending on the structure of your PackageDetailFile class and the underlying data you could do something like this
var packageDetailFiles = new List<PackageDetailFile>();
while (Running)
{
while (rdr.Read())
{
var detailFile = new PackageDetailFile();
// the following will depend on your data structure and class properties
detailFile.Property1 = rdr.GetString(0);
detailFile.Property2 = rdr.GetString(1);
detailFile.Property3 = rdr.GetString(2);
packageDetailFiles.Add(detailFile);
}
}
Note: You can use assemblies like AutoMapper to map from a data reader to a POCO reducing the need for the tedious MyObject.MyProperty = dataReader[collectionIndex] making your code far more readable and test friendly.
try use GetValues method of rdr object. I suppose rdr is a SqlDataReader or another DataReader.
This method return an object array with all columns of the current row.
After it you can store the object array in a list of objects. If you want to store in another type of list you must cast every element to the correct type before storing it.
I have an arraylist which has data structures in it.
I am having problems trying to figure out how to get those values back and show them in a table..
Thanks
this is my structure..
public BackupSpecEntry(string Path, string InclExcl, byte InclExclFlags, bool IndexContents,
int ServerBackupSpecId, int Freq, int Retention)
{
path = Path;
inclExcl = InclExcl;
inclExclFlags = InclExclFlags;
indexContents = IndexContents;
serverBackupSpecId = ServerBackupSpecId;
freq = Freq;
retention = Retention;
}
With an ArrayList you need to cast them,
ArrayList list = new ArrayLIst();
: your code
BackupSpecEntry entry = (BackupSpecEntry)list[0];
However, with generics with C# you can create a template list:
List<BackupSpecEntry> list = new List<BackupSpecEntry>();
: your fill list code
BackupSpecEntry entry = list[0];
An ArrayList isn't strongly typed so whenever you pull out an item, you have to cast it back to your custom object type. Then you should be able to access it's properties.
I assume by Table and by the Asp.net attribute that you mean a DataGrid, GridView, or DetailsView. Is this correct. Assuming that it is: read up on Data binding and custom objects. There is a lot of information about this subject on the internet if you look for it.
I've often had to load multiple items to a particular record in the database. For example: a web page displays items to include for a single report, all of which are records in the database (Report is a record in the Report table, Items are records in Item table). A user is selecting items to include in a single report via a web app, and let's say they select 3 items and submit. The process will add these 3 items to this report by adding records to a table called ReportItems (ReportId,ItemId).
Currently, I would do something like this in in the code:
public void AddItemsToReport(string connStr, int Id, List<int> itemList)
{
Database db = DatabaseFactory.CreateDatabase(connStr);
string sqlCommand = "AddItemsToReport"
DbCommand dbCommand = db.GetStoredProcCommand(sqlCommand);
string items = "";
foreach (int i in itemList)
items += string.Format("{0}~", i);
if (items.Length > 0)
items = items.Substring(0, items.Length - 1);
// Add parameters
db.AddInParameter(dbCommand, "ReportId", DbType.Int32, Id);
db.AddInParameter(dbCommand, "Items", DbType.String, perms);
db.ExecuteNonQuery(dbCommand);
}
and this in the Stored procedure:
INSERT INTO ReportItem (ReportId,ItemId)
SELECT #ReportId,
Id
FROM fn_GetIntTableFromList(#Items,'~')
Where the function returns a one column table of integers.
My question is this: is there a better way to handle something like this? Note, I'm not asking about database normalizing or anything like that, my question relates specifically with the code.
If going to SQL Server 2008 is an option for you, there's a new feature called "Table-valued parameters" to solve this exact problem.
Check out more details on TVP here and here or just ask Google for "SQL Server 2008 table-valued parameters" - you'll find plenty of info and samples.
Highly recommended - if you can move to SQL Server 2008...
Your string join logic can probably be simplified:
string items =
string.Join("~", itemList.Select(item=>item.ToString()).ToArray());
That will save you some string concatenation, which is expensive in .Net.
I don't think anything is wrong with the way you are saving the items. You are limiting trips to the db, which is a good thing. If your data structure was more complex than a list of ints, I would suggest XML.
Note: I was asked in the comments if this would save us any string concatenation (it does indeeed). I think it is an excellent question and would like to follow up on that.
If you peel open string.Join with Reflector you will see that Microsoft is using a couple of unsafe (in the .Net sense of the word) techniques, including using a char pointer and a structure called UnSafeCharBuffer. What they are doing, when you really boil it down, is using pointers to walk across an empty string and build up the join. Remember that the main reason string concatenation is so expensive in .Net is that a new string object is placed on the heap for every concatenation, because string is immutable. Those memory operations are expensive. String.Join(..) is essentially allocating the memory once, then operating upon it with a pointer. Very fast.
One potential issue with your technique is that it doesn't handle very large lists - you may exceed the maximum string length for your database. I use a helper method that concatenates the integer values into an enumeration of strings, each of which is less than a specified maximum (the following implementation also optionally checks for and removes duplicates ids):
public static IEnumerable<string> ConcatenateValues(IEnumerable<int> values, string separator, int maxLength, bool skipDuplicates)
{
IDictionary<int, string> valueDictionary = null;
StringBuilder sb = new StringBuilder();
if (skipDuplicates)
{
valueDictionary = new Dictionary<int, string>();
}
foreach (int value in values)
{
if (skipDuplicates)
{
if (valueDictionary.ContainsKey(value)) continue;
valueDictionary.Add(value, "");
}
string s = value.ToString(CultureInfo.InvariantCulture);
if ((sb.Length + separator.Length + s.Length) > maxLength)
{
// Max length reached, yield the result and start again
if (sb.Length > 0) yield return sb.ToString();
sb.Length = 0;
}
if (sb.Length > 0) sb.Append(separator);
sb.Append(s);
}
// Yield whatever's left over
if (sb.Length > 0) yield return sb.ToString();
}
Then you use it something like:
using(SqlCommand command = ...)
{
command.Connection = ...;
command.Transaction = ...; // if in a transaction
SqlParameter parameter = command.Parameters.Add("#Items", ...);
foreach(string itemList in ConcatenateValues(values, "~", 8000, false))
{
parameter.Value = itemList;
command.ExecuteNonQuery();
}
}
You either do what you've already got, pass in a delimited string and then parse out to a table value, or the other choice is passing in a wodge of XML and kinda much the same:
http://weblogs.asp.net/jgalloway/archive/2007/02/16/passing-lists-to-sql-server-2005-with-xml-parameters.aspx
I haven't had a chance to look at SQL 2008 yet to see if they've added any new functionality to handle this type of thing.
Why not use a table-valued parameter?
https://learn.microsoft.com/en-us/dotnet/framework/data/adonet/sql/table-valued-parameters
See http://www.sommarskog.se/arrays-in-sql-2005.html for a detailed discussion of this issue and the different approaches that you could use.
Here's a very clear-cut explanation to Table Valued Parameters from sqlteam.com: Table Valued Parameters
Query a Single Field for Multiple Values in a Stored Procedure
http://www.norimek.com/blog/post/2008/04/Query-a-Single-Field-for-Multiple-Values-in-a-Stored-Procedure.aspx