I have a DataTable similar to
Col1 Col2 Col3 Col4 Col5 Date
21 22 23 24 25 7/25/2014 12:00:00 AM
31 32 33 34 35 7/25/2014 12:00:00 AM
11 12 13 14 15 7/25/2014 12:00:00 AM
and I loop through it as
StringBuilder output = new StringBuilder("Col1\tCol2\tCol3\tCol4\tCol5\tDate\n");
foreach(DataRow row in partTable.Select()) {
output.AppendLine(String.Join("\t", row.ItemArray.ToArray()));
}
and do some formatting using StringBuilder.Replace on output. I print the result to a MessageBox and it duplicates my rows. The first time I call this it prints 2 copies, the next it prints 3, etc. (After one call.) I have checked repeatedly that the table is correct and doesn't contain duplicates. Below is the full code for this function.
private void printTable() {
updateDataSet();
if (partTable.Rows.Count == 0) {
MessageBox.Show("Table is empty.", "Table");
return;
}
StringBuilder output = new StringBuilder("Col1\tCol2\tCol3\tCol4\tCol5\tDate\n");
foreach(DataRow row in partTable.Select()) {
output.AppendLine(String.Join("\t", row.ItemArray.ToArray()));
}
// Get rid of time and type
output.Replace("12:00:00 AM", "");
output.Replace("W\t", "");
MessageBox.Show(output.ToString(), "Table");
output.Clear();
}
Solution Implemented: Commenting out updateDataSet() removes the duplication. I guess I just need to try to read MSDN more carefully... Replaced Fill with Update, but it would not remove any rows I deleted. Used a combination of Clear and Fill to get an updated table without recreating the connection.
If you call two times the DataAdapter.Fill(DataTable) method you double the records present in your datatable. To avoid this behaviour you need to write (inside the updateDataSet() method)
OleDbDataAdapter da = new OleDbDataAdapter(.....)
DataTable dt = new DataTable();
da.MissingSchemaAction = MissingSchemaAction.AddWithKey;
da.Fill(dt);
// Just for testing
// Check these results with and without the MissingSchemaAction flag
Console.WriteLine(dt.Rows.Count);
da.Fill(dt);
Console.WriteLine(dt.Rows.Count);
Of course, the presence of MissingSchemaAction.AddWithKey results in poorer performances if you don't remove the cause of the second (or third call) to updateDataSet() Infact, in this scenario the loading method should check every row present to find duplicates.
Please put all your code for analysis.
Also, there is logic/design issue in the code which has been provided.
PrintTable method is expected to read and print data only. UpdateDataset is breaking intention of code.
Related
i need to make a data table like that:
Subjects old new diff
Sub_1 10 50 40
Sub_2 30 10 -20
total 40 60 20
this is a part of code
DataTable subjects = new DataTable();
subjects.Columns.Add("Subjects");
subjects.Columns.Add("old");
subjects.Columns.Add("new");
subjects.Columns.Add("diff");
subjects.Rows.Add("Sub_1", sub1.Old, sub1.New, (sub1.New - sub1.Old));
subjects.Rows.Add("Sub_2", sub2.Old, sub2.New, (sub2.New - sub2.Old));
subjects.Rows.Add("Total", .. total of above .. , .. total of above .., .. total of above ..);
so i need to ask how to calculate the total value of last column ( Total) , and is there is any other way to calculate the 4th coulmn ( 3rdcol - 2 2nd col )
First of all, you have declared your columns without DataType(thanks to #Steve for comment). So, please change Add() methods as:
subjects.Columns.Add("old", typeof(Int32));
subjects.Columns.Add("new", typeof(Int32));
Also, you can set the value for diff column like this:
subjects.Columns.Add("diff", typeof(Int32), "new - old");
And, then remove any other calculations in Rows.Add method:
subjects.Rows.Add("Sub_1", sub1.Old, sub1.New);
subjects.Rows.Add("Sub_2", sub2.Old, sub2.New);
And then you can use DataTable.Compute(string expression, string filter) method. It computes the given expression on the current rows that pass the filter criteria.
In your case expression will be Sum(columnName) and the filter will be empty string, because you don't need any filter.
subjects.Compute("Sum(old)", "")
So, change your code as:
subjects.Rows.Add("Total",
subjects.Compute("Sum(old)", ""),
subjects.Compute("Sum(new)", ""),
subjects.Compute("Sum(diff)", ""));
I have a datatable with a few columns .
I am trying to add the column values using the datacolumn.expression.
The columns used for adding is of type decimal. Also the calculated column is also decimal. But while processing the expression, (like datatable column1+ datatable column2) its just appending the data.
SlNo Name F1 F2 F3
1 A 1 2 3
2 B 3 4 5
I am expecting an output similar to this.
SlNo Name F1 F2 F3 Total
1 A 1 2 3 6
2 B 3 4 5 12
What I tried.
dtTempData.Columns.Add("Total", typeof(Decimal));
dtTempData.Columns["Total"].Expression = "[F1]+[F2]+[F3]";
Now the output I am getting is in the following way
123
345
its just appending the data.Thanks in advance of any help.
I don't know why this is happening.
dtTempData.Columns.Add("Total", typeof(Decimal));
dtTempData.Columns["Total"].DefaultValue = 0;
dtTempData.Columns["Total"].Expression = expression;
This is the way I created the columns, but while performing suming based on expression, it appends the data.
I am importing data from another datatable which is type of string. So I tried to convert the data by using the following manner.
string expression="Convert(F1, 'System.Decimal') + Convert(F2,
'System.Decimal') + Convert(F3, 'System.Decimal')"
Now this is working and the Total column is having the value after addition. Thanks all for your help.
In my project, I'm using a StringBuilder to create a table in RichEditBox. But in my column the data value is differ from each other. So it looks like shuffled data. But my excepted output is to be perfectly fix the length for the data in table. I can see the table as properly completed in my output .
Can anyone say how to fix the length to the string in StringBuilder in C#?
My code:
Connection();
try
{
string Today = "Student Test Mark Details";
string Line = "----------------------------------";
SqlCommand cmd = new SqlCommand();
cmd.CommandType = CommandType.Text;
cmd = new SqlCommand("select ExaminationName, ExaminationCenter, ExaminationDate, Subjects, MarkObtained, Total, Percentage, Grade from StudentMarksHistory where StudentCode='" + ICBEStudentCode.Text + "' and ExaminationName='" + TBExaminationName.Text + "' and ClassName='" + ICBEClassSection.Text + "' and Remark='Record Saved'", cs);
StringBuilder paragraph = new StringBuilder();
SqlDataReader dr = cmd.ExecuteReader();
paragraph.Append(Today).Append("\t\n");
paragraph.Append(Line).Append("\t\n\n\n\n\n");
paragraph.Append("ExamName").Append("\t");
paragraph.Append("ExamCenter").Append("\t");
paragraph.Append("ExamDate").Append("\t");
paragraph.Append("Subject").Append("\t\t\t");
paragraph.Append("Mark").Append("\t");
paragraph.Append("Total").Append("\t");
paragraph.Append("Percentage").Append("\t");
paragraph.Append("Grade").Append("\n\n");
while (dr.Read())
{
DateTime DateName = Convert.ToDateTime(dr["ExaminationDate"]);
string subject = dr["Subjects"].ToString().Trim();
//int Len = subject.Length;
//subject = subject.ToString().PadRight(50 - Len, ' ');
paragraph.Append(dr["ExaminationName"].ToString()).Append("\t");
paragraph.Append(dr["ExaminationCenter"].ToString()).Append("\t");
paragraph.Append(DateName.ToString("dd/MM/yyyy")).Append("\t");
paragraph.Append(subject);
paragraph.Append(' ', 15 - subject.Length);
//paragraph.Append(subject.PadRight(100));
paragraph.Append(dr["MarkObtained"].ToString()).Append("\t");
paragraph.Append(dr["Total"].ToString()).Append("\t");
paragraph.Append(dr["Percentage"].ToString()).Append("\t");
paragraph.Append(dr["Grade"].ToString()).Append("\n");
}
string Notes = "************[ Minimum Pass Mark is 35 ]*************";
paragraph.Append(" ").Append("\n\n\n\n");
paragraph.Append(Notes).Append("\r\n");
dr.Close();
cs.Close();
}
RTBMessage.Text = paragraph.ToString();
}
catch (Exception ex)
{
MessageBox.Show(ex.Message);
}
Results expceted:
Student Test Mark Details
----------------------------------
ExamName ExamCenter ExamDate Subject Mark Total
Class Test 1 Room No1 24/05/2013 STOREDPROCEDURE 97 404
Class Test 1 Room No1 25/05/2013 DOTNET 86 404
Class Test 1 Room No1 26/05/2013 TAMIL 80 404
Class Test 1 Room No1 23/05/2013 SOCIAL 80 404
Class Test 1 Room No1 27/05/2013 COMPUTER 61 404
************[ Minimum Pass Mark is 35 ]*************
Do not use TABs to align the column but use Composite Formatting through AppendFormat method of the StringBuilder
Just as an example for your first column.
I suppose that ExamName should be inserted left aligned in a column large 20 characters.
You could write
paragraph.AppendFormat("{0:20}","ExamName");
while for the Grade column right aligned in a 6 char space
paragraph.AppendFormat("{0:-6}\r\n", "Total");
Of course this kind of alignement is dependent on the kind of font used in the RichTextBox. If you use a proportional font there is no way to align this text in the way you like.
At the end I really suggest you to use a DataGridView instead
As the lenght of your data is very fluctuating, do not add tabs "\t", this will not work if some data is longer than tabsize
You can use the
String.PadRight(Int32, Char)
Method if you know your maximum expected data lenght.
Remember the usage: the first parameter describes the total lengt not the number of characters to fill
http://msdn.microsoft.com/en-us/library/66f6d830(v=vs.71).aspx
This is a followup to my first question "Porting “SQL” export to T-SQL".
I am working with a 3rd party program that I have no control over and I can not change. This program will export it's internal database in to a set of .sql each one with a format of:
INSERT INTO [ExampleDB] ( [IntField] , [VarcharField], [BinaryField])
VALUES
(1 , 'Some Text' , 0x123456),
(2 , 'B' , NULL),
--(SNIP, it does this for 1000 records)
(999, 'E' , null);
(1000 , 'F' , null);
INSERT INTO [ExampleDB] ( [IntField] , [VarcharField] , BinaryField)
VALUES
(1001 , 'asdg', null),
(1002 , 'asdf' , 0xdeadbeef),
(1003 , 'dfghdfhg' , null),
(1004 , 'sfdhsdhdshd' , null),
--(SNIP 1000 more lines)
This pattern continues till the .sql file has reached a file size set during the export, the export files are grouped by EXPORT_PATH\%Table_Name%\Export#.sql Where the # is a counter starting at 1.
Currently I have about 1.3GB data and I have it exporting in 1MB chunks (1407 files across 26 tables, All but 5 tables only have one file, the largest table has 207 files).
Right now I just have a simple C# program that reads each file in to ram then calls ExecuteNonQuery. The issue is I am averaging 60 sec/file which means it will take about 23 hrs for it to do the entire export.
I assume if I some how could format the files to be loaded with a BULK INSERT instead of a INSERT INTO it could go much faster. Is there any easy way to do this or do I have to write some kind of Find & Replace and keep my fingers crossed that it does not fail on some corner case and blow up my data.
Any other suggestions on how to speed up the insert into would also be appreciated.
UPDATE:
I ended up going with the parse and do a SqlBulkCopy method. It went from 1 file/min. to 1 file/sec.
Well, here is my "solution" for helping convert the data into a DataTable or otherwise (run it in LINQPad):
var i = "(null, 1 , 'Some''\n Text' , 0x123.456)";
var pat = #",?\s*(?:(?<n>null)|(?<w>[\w.]+)|'(?<s>.*)'(?!'))";
Regex.Matches(i, pat,
RegexOptions.IgnoreCase | RegexOptions.Singleline).Dump();
The match should be run once per value group (e.g. (a,b,etc)). Parsing of the results (e.g. conversion) is left to the caller and I have not tested it [much]. I would recommend creating the correctly-typed DataTable first -- although it may be possible to pass everything "as a string" to the database? -- and then use the information in the columns to help with the extraction process (possibly using type converters). For the captures: n is null, w is word (e.g. number), s is string.
Happy coding.
Apparently your data is always wrapped in parentheses and starts with a left parenthesis. You might want to use this rule to split(RemoveEmptyEntries) each of those lines and load it into a DataTable. Then you can use SqlBulkCopy to copy all at once into the database.
This approach would not necessarily be fail-safe, but it would be certainly faster.
Edit: Here's the way how you could get the schema for every table:
private static DataTable extractSchemaTable(IEnumerable<String> lines)
{
DataTable schema = null;
var insertLine = lines.SkipWhile(l => !l.StartsWith("INSERT INTO [")).Take(1).First();
var startIndex = insertLine.IndexOf("INSERT INTO [") + "INSERT INTO [".Length;
var endIndex = insertLine.IndexOf("]", startIndex);
var tableName = insertLine.Substring(startIndex, endIndex - startIndex);
using (var con = new SqlConnection("CONNECTION"))
{
using (var schemaCommand = new SqlCommand("SELECT * FROM " tableName, con))
{
con.Open();
using (var reader = schemaCommand.ExecuteReader(CommandBehavior.SchemaOnly))
{
schema = reader.GetSchemaTable();
}
}
}
return schema;
}
Then you simply need to iterate each line in the file, check if it starts with ( and split that line by Split(new[] { ',' }, StringSplitOptions.RemoveEmptyEntries). Then you could add the resulting array into the created schema-table.
Something like this:
var allLines = System.IO.File.ReadAllLines(path);
DataTable result = extractSchemaTable(allLines);
for (int i = 0; i < allLines.Length; i++)
{
String line = allLines[i];
if (line.StartsWith("("))
{
String data = line.Substring(1, line.Length - (line.Length - line.LastIndexOf(")")) - 1);
var fields = data.Split(new[] { ',' }, StringSplitOptions.RemoveEmptyEntries);
// you might need to parse it to correct DataColumn.DataType
result.Rows.Add(fields);
}
}
The following VB line, where _DSversionInfo is a DataSet, returns no rows:
_DSversionInfo.Tables("VersionInfo").Select("FileID=88")
but inspection shows that the table contains rows with FileID's of 92, 93, 94, 90, 88, 89, 215, 216. The table columns are all of type string.
Further investigation showed that using the ID of 88, 215 and 216 will only return rows if the number is quoted.
ie _DSversionInfo.Tables("VersionInfo").Select("FileID='88'")
All other rows work regardless of whether the number is quoted or not.
Anyone got an explanation of why this would happen for some numbers but not others? I understand that the numbers should be quoted just not why some work and others don't?
I discovered this in some VB.NET code but (despite my initial finger pointing) don't think it is VB.NET specific.
According to the MSDN documentation on building expressions, strings should always be quoted. Failing to do so produces some bizarro unpredictable behavior... You should quote your number strings to get predictable and proper behavior like the documentation says.
I've encounted what you're describing in the past, and kinda tried to figure it out - here, pop open your favorite .NET editor and try the following:
Create a DataTable, and into a string column 'Stuff' of that DataSet, insert rows in the following order: "6", "74", "710", and Select with the filter expression "Stuff = 710". You will get 1 row back. Now, change the first row into any number greater than 7 - suddenly, you get 0 rows back.
As long as the numbers are ordered in proper descending order using string ordering logic (i.e., 7 comes after 599) the unquoted query appears to work.
My guess is that this is a limitation of how DataSet filter expressions are parsed, and it wasn't meant to work this way...
The Code:
// Unquoted filter string bizzareness.
var table = new DataTable();
table.Columns.Add(new DataColumn("NumbersAsString", typeof(String)));
var row1 = table.NewRow(); row1["NumbersAsString"] = "9"; table.Rows.Add(row1); // Change to '66
var row2 = table.NewRow(); row2["NumbersAsString"] = "74"; table.Rows.Add(row2);
var row4 = table.NewRow(); row4["NumbersAsString"] = "90"; table.Rows.Add(row4);
var row3 = table.NewRow(); row3["NumbersAsString"] = "710"; table.Rows.Add(row3);
var results = table.Select("NumbersAsString = 710"); // Returns 0 rows.
var results2 = table.Select("NumbersAsString = 74"); // Throws exception "Min (1) must be less than or equal to max (-1) in a Range object." at System.Data.Select.GetBinaryFilteredRecords()
Conclusion: Based on the exception text in that last case, there appears to be some wierd casting going on inside filter expressions that is not guaranteed to be safe. Explicitely putting single quotes around the value for which you're querying avoids this problem by letting .NET know that this is a literal.
DataTable builds an index on the columns to make Select() queries fast. That index is sorted by value, then it uses a binary search to select the range of records that matches the query expression.
So the records will be sorted like this 215,216,88,89,90,92,93,94. A binary search is done treating them as integer (as per our filter expression) cannot locate certain records because, it is designed to only search properly sorted collections.
It indexes the data as string and Binary search searches as number. See the below explanation.
string[] strArr = new string[] { "115", "118", "66", "77", "80", "81", "82" };
int[] intArr = new int[] { 215, 216, 88, 89, 90, 92, 93, 94 };
int i88 = Array.BinarySearch(intArr, 88); //returns -ve index
int i89 = Array.BinarySearch(intArr, 89); //returns +ve index
This should be a bug in the framework.
this error usually comes due to invalid data table column type in which you are going to search
i got this error when i was using colConsultDate instead of Convert(colConsultDate, 'System.DateTime')
because colConsultDate was a data table column of type string which i must have to convert into System.DateTime therefor your search query should be like
string query = "Convert(colConsultDate, 'System.DateTime') >= #" + sdateDevFrom.ToString("MM/dd/yy") + "# AND Convert(colConsultDate, 'System.DateTime') <= #" + sdateDevTo.ToString("MM/dd/yy") + "#";
DataRow[] dr = yourDataTable.Select(query);
if (dr.Length > 0)
{
nextDataTabel = dr.CopyToDataTable();
}
#Val Akkapeddi just wanna add things to your answer.
if you do something like this it would be benefited specially when you have to use comparison operators. because you put quotes around 74 it will be treated as string. please see yourself by actually writing code. Comparison operators
(decimal is just for reference you can add your desired datatype instead.)
var results2 = table.Select("Convert(NumbersAsString , 'System.Decimal') = 74.0")