I have something like this:
using (SqlDataAdapter a = new SqlDataAdapter(query, c))
{
// 3
// Use DataAdapter to fill DataTable
DataTable t = new DataTable();
a.Fill(t);
string txt = "";
for (int i = 0; i < t.Rows.Count; i++)
{
txt += "\n" + t.Rows[i][0] + "\t" + t.Rows[i][1] + "\t" + t.Rows[i][2] + "\t" + t.Rows[i][3] + "\t" + t.Rows[i][4] + "\t" + t.Rows[i][5] + "\t" + t.Rows[i][6] + "\t" + t.Rows[i][7] + "\t" + t.Rows[i][8] + "\t" + t.Rows[i][9] + "\t" + t.Rows[i][10] + "\t" + t.Rows[i][11] + "\t" + t.Rows[i][12] + "\t" + t.Rows[i][13] + "\t" + t.Rows[i][14] + "\t" + t.Rows[i][15] + "\t" + t.Rows[i][16] + "\t" + t.Rows[i][17] + "\t" + t.Rows[i][18] + "\t" + t.Rows[i][19];
}
}
I need this tabs between columns to generate txt file later.
Problem is that t.Rows.Count = 600000 and this loop works 9 hours.
Any ideas how to make this faster?
Regards
kazik
Use a StringBuilder instead of concatenating your strings.
string txt = "";
StringBuilder sb = new StringBuilder();
for (int i = 0; i < t.Rows.Count; i++)
{
sb.Append(t.Rows[i][0]);
sb.Append("\t");
sb.Append(t.Rows[i][1]);
//and so on...
}
string txt = sb.ToString();
Adding a string to another string (by assigning it back again) takes effort relative to the length of the combined string - this gets very costly fast for large strings - it is a Schlemiel the Painter's algorithm. Using a StringBuilder avoids this since no new string have to be created every time you add a new string to its content.
I have to question though why you keep all of this data in memory just to write it out to a file - you should switch to a model that allows you to generate line by line of the text content, I would suggest using a data reader instead and method that yields text lines, e.g.:
public IEnumerable<string> GenerateTextOutput()
{
//...
//using SqlDataReader instead
while(reader.Read())
{
//assemble string for next row
string txt = "...";
yield return txt;
}
}
Then you can just write this output to disk:
File.WriteAllLines("foo.txt", GenerateTextOutput());
Use StringBuilder. You can find examples everywhere on the web.
Others mentioned StringBuilder, and they are correct. However, note that in .NET 4 StringBuilder became more efficient, avoiding reallocations and memory copying altogether. If you're using a previous version of .NET, you will get worse performance. It will still beat what you're doing.
I wonder what you're doing with txt after it's ready. You might want to avoid storing the string in memory altogether, and write it directly to a file (I don't believe you're showing your users 600,000 lines...)
StringBuilder has already been mentioned by others and should provide significant benefit.
Do you really need to keep the entire text in memory with 600K rows? You can speed this up a lot by directly writing to the file.
And for that many iterations, assigning Rows[i] to a variable might also provide some extra spped-up.
Related
As an introduction, I have created a document using MS Word and then saved as html document.
From C# I am building an unordered html list (using MS Word format) and then add it to the html document by replacing a specific tag.
I have below string variable unorderedHtmlList initially initialized to empty string. Then I am concatenating html string and replacing some tags enclosed by "[[" and "]]" characters. For some reason when I apply the Replace it is not replacing the items [[fieldName]] and [[fieldValue]] by the new values. See code below:
string unorderedHtmlList = string.Empty;
foreach (System.Data.DataRow row in myDataTable.Rows)
{
string name = row["fieldName"].ToString();
string value = row["fieldValue"].ToString();
unorderedHtmlList += "<p style='margin-left:36.0pt;text-align:justify;text-indent:-18.0pt;" +
"line-height:125%;mso-list:l1 level1 lfo3'><![if !supportLists]><span" +
"style='font-size:10.5pt;line-height:125%;font-family:\"Arial\",sans-serif;" +
"mso-fareast-font-family:Arial;color:#222222'><span" +
"style='mso-list:Ignore'>-<span style='font:7.0pt \"Times New Roman\"'> " +
"</span></span></span><![endif]><span style='font-size:10.5pt;" +
"line-height:125%;font-family:\"Arial\",sans-serif;color:#222222'>[[fieldName]]" +
"</span><span style='font-size:10.5pt;line-height:125%;font-family:" +
"\"Helvetica\",sans-serif;color:#222222'>[[fieldValue]]</span><span" +
"style='font-size:10.5pt;line-height:125%;font-family:\"Arial\",sans-serif;" +
"color:#222222'><o:p></o:p></span></p>".Replace("[[fieldName]]", name).Replace("[[fieldValue]]", value);
}
Any ideas why Replace is not working?
You are concatanating the string and the replace operation is executed only on the last part.
"color:#222222'><o:p></o:p></span></p>".Replace("[[fieldName]]", name).Replace("[[fieldValue]]", value);
Try this:
unorderedHtmlList += ("<p style='margin-left:36.0pt;text-align:justify;text-indent:-18.0pt;" +
"line-height:125%;mso-list:l1 level1 lfo3'><![if !supportLists]><span" +
"style='font-size:10.5pt;line-height:125%;font-family:\"Arial\",sans-serif;" +
"mso-fareast-font-family:Arial;color:#222222'><span" +
"style='mso-list:Ignore'>-<span style='font:7.0pt \"Times New Roman\"'> " +
"</span></span></span><![endif]><span style='font-size:10.5pt;" +
"line-height:125%;font-family:\"Arial\",sans-serif;color:#222222'>[[fieldName]]" +
"</span><span style='font-size:10.5pt;line-height:125%;font-family:" +
"\"Helvetica\",sans-serif;color:#222222'>[[fieldValue]]</span><span" +
"style='font-size:10.5pt;line-height:125%;font-family:\"Arial\",sans-serif;" +
"color:#222222'><o:p></o:p></span></p>").Replace("[[fieldName]]", name).Replace("[[fieldValue]]", value);
I need to save multidimensional array into file. Then be able to change things in that so it will also appear in that file and load from said file into array.
Let's say I have some array:
private string[,] array = new string[5, 5];
Then I need to do a function, that will fill the array with space if file does not exist and if it does, load it. Something like this:
private void Load()
{
if (File.Exists("save.txt"))
{
}
else
{
for (int i = 0; i < 5; i++)
{
for (int j = 0; j < 5; j++)
{
array[i,j] = " ";
}
}
}
}
I checked if the file exists, if not I will fill the array with spaces but then I need in some way write the array in the file in this format:
" ", " ", " "," ", " ",
" ", " ", " "," ", " ",
" ", " ", " "," ", " ",
" ", " ", " "," ", " ",
" ", " ", " "," ", " ",
or in some format, that will work.
Then I basically planned to split the string with ',' and save it to that array. And if file exists, read it and save the values to said array.
So basically I need to check if the file exist, if the answer is yes, load it to the multidimensional array, If the answer is no, fill the array with empty strings and load that in the file.
If somebody is willing to help me, I will be very grateful.
I have created an application. This app contains the five textboxes id, name, surname, age and score.
When a user clicks the "okay button", these values are stores in an sql database.
Additionally, I want to store all of these information in an QR code. And when I decode it, the information should be shown in the textboxes respectively.
These are the references I am using so far.
using AForge.Video.DirectShow;
using Zen.Barcode;
using ZXing.QrCode;
using ZXing;
I can encode an ID number into a picture box, like so:
CodeQrBarcodeDraw qrcode = BarcodeDrawFactory.CodeQr;
pictureBox1.Image = qrcode.Draw(textBox1.Text, 50);
But I want all of the values in the textboxes to be storee in this QR code.
How can i do that?
The essence of the solution is, that you have to combine all the values from the textboxes into one string. To seperate them after decoding the QR code, you have to add a special character between the data values, that does not exist insinde the user input. After decoding the QR code, you can seperate the values by splitting the string at each occurance of the special character.
This is the quick and dirty way of doing that. If you want the QR code to be conformant to any specific format (like vcard), you have to reserach what it takes to compose the data for this format.
I expect your users cannot enter more than one line into the textboxes, so the newline character can be used as seperator character.
Encode all the information into one QR code.
var qrText = textBox1.Text + "\n" +
textBox2.Text + "\n" +
textBox3.Text + "\n" +
textBox4.Text + "\n" +
textBox5.Text;
pictureBox1.Image = qrcode.Draw(qrText, 50);
You can decode the QR code and assigning the data to the different textboxes again.
var bitmap = new Bitmap(pictureBox1.Image);
var lumianceSsource = new BitmapLuminanceSource(bitmap);
var binBitmap = new BinaryBitmap(new HybridBinarizer(source));
var reader = new MultiFormatReader();
Result result = null;
try
{
result = reader.Decode(binBitmap);
}
catch (Exception err)
{
// Handle the exceptions, in a way that fits to your application.
}
var resultDataArray = result.Text.Split(new char[] {'\n'});
// Only if there were 5 linebreaks to split the result string, it was a valid QR code.
if (resultDataArray.length == 5)
{
textBox1.Text = resultDataArray[0];
textBox2.Text = resultDataArray[1];
textBox3.Text = resultDataArray[2];
textBox4.Text = resultDataArray[3];
textBox5.Text = resultDataArray[4];
}
You can get this done by implementing below code :
"{" + '"' + "name" + '"' + ":" + '"' + txtName.Text + '"' + "," + '"' + "lname" + '"' + ":" + '"' + txtLname.Text + '"' + "," + '"' + "Roll" + '"' + ":" + '"' + txtRoll.Text + '"' + '"' + "class" + '"' + ":" + '"' + txtClass.Text + '"' + "}"
Result will be:
{"name":"Diljit","lname":"Dosanjh","Roll","2071","class":"BCA"}
Such that your QR scanner will recognize the data belong to its specific filed.
Having trouble figuring out how to prevent the last key in my array to not have a comma. Since its being exported to a .Json file the last key shouldn't have a ",".
I know you can detect it by using .Last();, but I can't seem to make that work. Any recommendations?
//Data Path
string dataPath = #"..\..\FileIOExtraFiles\DataFieldsLayout.txt";
string[] dataList = File.ReadAllLines(dataPath);
//save Data data
using (StreamWriter outStream = new StreamWriter(outputFolder + #"\CharacterStringData3.json"))
{
outStream.WriteLine("{");
for (int i = 0; i < dataFile.Length; i++)
{
string s = dataFile[i];
char last = s.Last();
if (s == "")
{
outStream.WriteLine("\"" + dataList[i] + "\"" + " : " + "\" \",");
}
else
{
outStream.WriteLine("\"" + dataList[i] + "\"" + " : \"" + s + "\",");
}
}
outStream.WriteLine("}");
}
Output:
{
"data1":"item1",
"data2":item2",
"lastKey":item3",//trying to remove comma from last key in array.
}
As others have pointed out, it doesn't make sense that you are building json manually, but given that this is a question more about technique, here is one approach: you could change it to this:
var commaSuffix = (i == dataFile.Length - 1) ? "," : string.Empty;
outStream.WriteLine("\"" + dataList[i] + "\"" + " : \"" + s + "\"" + commaSuffix);
The suffix would be used on every iteration except the last.
Change this
outStream.WriteLine("\"" + dataList[i] + "\"" + " : " + "\" \",");
To this
outStream.WriteLine("\"" + dataList[i] + "\"" + " : " + "\" \""+(i==dataFile.Length?",":""));
Instead of using outStream.WriteLine() at every step, store it in a string. Then you can remove the last comma from that string and write the whole string at once:
//Get last index of comma
int lastCommaIndex = outputString.LastIndexOf(',');
//Create new StringBuilder with everything before the last comma
StringBuilder sb = new StringBuilder(outputString.Substring(0,lastCommaIndex));
//Add everything after the last comma, or just add a closing brace
//sb.Append("}"); //This instead of next line
sb.Append(outputString.Substring(lastCommaIndex+1));
//Add contents of StringBuilder to the Stream
outSteam.WriteLine(sb);
Help me please write every value to different cell, I try to write data in the CSV file but when I open this file in Excel the value showing in one line (cell). Did different various separators between the values "." and ",". But the results in Office 2010 and 2012 the same... And one more interesting think, in Office 2007 that work's with (*.xls)...
string s = "";
s += "Nick-Name, Category, Web-Pages, ICQ, Skype, Mail, Phone\r";
for (int i = 0; i < listOfUsers.Count; i+=3)
{
s += nickName[i] + "," + category[i] + "," + webList[i] + "," + ICQList[i] + "," + skypeList[i] + "," + mailList[i] + "," + phoneList[i] + ";\r";
}
System.IO.File.WriteAllText(#"" + folder + "Parse_people_list.csv", s, System.Text.Encoding.UTF8);
(source: pixs.ru)
(source: pixs.ru)
Try delimiting with tabs instead - Excel handles tab-delimited data better than comma-delimited since commas within the data may cause issues.
You can also use StringBuilder.AppendLine to improve performance and avoid adding line breaks manually:
StringBuilder s = new StringBuilder();
s.AppendLine("Nick-Name\tCategory\tWeb-Pages\tICQ\tSkype\tMail\tPhone");
for (int i = 0; i < listOfUsers.Count; i+=3)
{
s.AppendLine(
nickName[i] + "\t"
+ category[i] + "\t"
+ webList[i] + "\t"
+ ICQList[i] + "\t"
+ skypeList[i] + "\t"
+ mailList[i] + "\t"
+ phoneList[i]);;
}