Develop Reference

Declaring variables slow down program

I have a program which uses a RichTextBox (which is part of Tab Control) and in the TextChanged event I these variable declarations:
RichTextBox programTextBox = (RichTextBox)tabControl.TabPages[tabControl.SelectedIndex].Controls[0];
int selectStart = programTextBox.SelectionStart;
int programCurrentLine = programTextBox.GetLineFromCharIndex(programTextBox.SelectionStart);
int programCurrentLineIndex = programTextBox.GetFirstCharIndexFromLine(programCurrentLine);
int programCurrentLineLength = programTextBox.Lines[programCurrentLine].Length;
string programCurrentLineText = programTextBox.Lines[programCurrentLine].ToString();
All of them are important in that particular event and I use them multiple times for multiple purposes. However, recalculating every single time slows down my program.
For example, I've noticed that if I open a somewhat large file (with my RichTextBox) and then start pressing e.g. the 'a' button, there is some noticable lag. Deleting every single piece of code except the declaration of these variables does not help at all with the lag, but also deleting the code above, completely solves the problem.
I have two questions: 1) Why do these declarations slow down that much the TextChanged event and 2) What can I do? (Is there a faster way to calculate these variables?)

It's not the variable declarations - it's the code you're using to initialize them. I wouldn't be at all surprised to find that GetLineFromCharIndex and GetFirstCharIndexFromLine were expensive - and currently you're calling the Lines property twice.
You could probably improve matters at least slightly just by removing one of those Lines calls, simply by fetching the line first and then looking at its length:
// No need to call ToString() - Lines is a string[]
string programCurrentLineText = programTextBox.Lines[programCurrentLine];
int programCurrentLineLength = programCurrentLineText.Length;
It's a shame there isn't some way of saying "Get all the information about the position of the given index: its line, first char index from line, and the line itself" in one call :(

If it is not important to recalculate on every key press you might be better off implementing a boolean variable which is set on keypress then has a timer to reset it. For example adding a 1.2 second delay on the event. I am not sure the needs of your app so that time might be a bit much.

Why is my encoding showing twice?

byte[] lengthBytes = new byte[4];
serverStream.Read(lengthBytes, 0, 4);
MessageBox.Show("'>>" + System.Text.Encoding.UTF8.GetString(lengthBytes) + "<<'");
MessageBox.Show("Hello");
This is the code I used for debugging. I get 2 messageboxes now. If I used Debug.WriteLine it was also printed twice.
Msgbox 1: '>>/ (Note that this is still 4 characters long, the last 3 bytes are null.
Msgbox 2: '>>{"ac<<'
Msgbox 3: Hello
I'm trying to send 4 bytes with an integer, the length of the message. This is going fine ('/ ' is utf8 for 47). The problem is that the first 4 bytes of the message are also being read ('{"ac'). I totally dont know how this happens, I'm already debugging this for several hours and I just can't get my head around it. One of my friends suggested to make an account on StackOverflow so here I am :p
Thanks for all the help :)
EDIT: The real code for the people who asked
My code http://kutj.es/2ah-j9

You are making traditional programmer mistakes, everybody has to make them once to learn how to avoid it and do it right. This primarily went off the rails by writing debugging code that is buggy and made it lot harder to find your mistake:
Never write debugging code that uses MessageBox.Show(). It is a very, very evil function, it causes re-entrancy. And expensive word that means that it only freezes the user interface, it doesn't freeze your program. It continues to run, one of the things that can go wrong is that the code that you posted is executed again. Re-entered. You'll see two message boxes. And you'll have a completely corrupted program state because your code was never written to assume it could be re-entered. Which is why you complained that 4 bytes of data were swallowed.
The proper tool to use here is the feature that really freezes your program. A debugger breakpoint.
Never assume that binary data can be converted to text. Those 4 bytes you received contain binary zeros. There is no character for it. Worse, it acts as a string terminator to many operating system calls, the kind used by the debugger, Debug.WriteLine() etc. This is why you can't see the "<<"
The proper tool to use here is a debugger watch or tooltip, it lets you look into the array directly. If you absolutely have to generate a diagnostic string then use BitConverter.GetString().
Never assume that a stream's Read() method will always return the number of bytes you asked for. Using the return value in your code is a hard requirement. This is the real bug in your program, the only you are actually trying to fix.
The proper solution is to continue to call Read() until you counted down the number of bytes you expected to receive from the length you read earlier. You'll need a MemoryStream to store the chunks of byte[]s you get.

Perhaps this link regarding Encoding.GetString() will help you out a bit. The part to pay attention to being:
If the data to be converted is available only in sequential blocks
(such as data read from a stream) or if the amount of data is so large
that it needs to be divided into smaller blocks, you should use the
Decoder object returned by the GetDecoder method of a derived class.

The problem was that I started the getMessage void 2 times. This started the while 2 times (in different threads).
Elgonzo helped me finding the problem, he is a great guy :)

Why does appending to TextBox.Text during a loop take up more memory with each iteration?

Short Question
I have a loop that runs 180,000 times. At the end of each iteration it is supposed to append the results to a TextBox, which is updated real-time.
Using MyTextBox.Text += someValue is causing the application to eat huge amounts of memory, and it runs out of available memory after a few thousand records.
Is there a more efficient way of appending text to a TextBox.Text 180,000 times?
Edit I really don't care about the result of this specific case, however I want to know why this seems to be a memory hog, and if there is a more efficient way to append text to a TextBox.
Long (Original) Question
I have a small app which reads a list of ID numbers in a CSV file and generates a PDF report for each one. After each pdf file is generated, the ResultsTextBox.Text gets appended with the ID Number of the report that got processed and that it was successfully processed. The process runs on a background thread, so the ResultsTextBox gets updated real-time as items get processed
I am currently running the app against 180,000 ID numbers, however the memory the application is taking up is growing exponentially as time goes by. It starts by around 90K, but by about 3000 records it is taking up roughly 250MB and by 4000 records the application is taking up about 500 MB of memory.
If I comment out the update to the Results TextBox, the memory stays relatively stationary at roughly 90K, so I can assume that writing ResultsText.Text += someValue is what is causing it to eat memory.
My question is, why is this? What is a better way of appending data to a TextBox.Text that doesn't eat memory?
My code looks like this:
try
{
report.SetParameterValue("Id", id);
report.ExportToDisk(ExportFormatType.PortableDocFormat,
string.Format(#"{0}\{1}.pdf", new object[] { outputLocation, id}));
// ResultsText.Text += string.Format("Exported {0}\r\n", id);
}
catch (Exception ex)
{
ErrorsText.Text += string.Format("Failed to export {0}: {1}\r\n",
new object[] { id, ex.Message });
}
It should also be worth mentioning that the app is a one-time thing and it doesn't matter that it is going to take a few hours (or days :)) to generate all the reports. My main concern is that if it hits the system memory limit, it will stop running.
I'm fine with leaving the line updating the Results TextBox commented out to run this thing, but I would like to know if there is a more memory efficient way of appending data to a TextBox.Text for future projects.

I suspect the reason the memory usage is so large is because textboxes maintain a stack so that the user can undo/redo text. That feature doesn't seem to be required in your case, so try setting IsUndoEnabled to false.

Use TextBox.AppendText(someValue) instead of TextBox.Text += someValue. It's easy to miss since it's on TextBox, not TextBox.Text. Like StringBuilder, this will avoid creating copies of the entire text each time you add something.
It would be interesting to see how this compares to the IsUndoEnabled flag from keyboardP's answer.

Don't append directly to the text property. Use a StringBuilder for the appending, then when done, set the .text to the finished string from the stringbuilder

Instead of using a text box I would do the following:
Open up a text file and stream the errors to a log file just in case.
Use a list box control to represent the errors to avoid copying potentially massive strings.

Personally, I always use string.Concat* . I remember reading a question here on Stack Overflow years ago that had profiling statistics comparing the commonly-used methods, and (seem) to recall that string.Concat won out.
Nonetheless, the best I can find is this reference question and this specific String.Format vs. StringBuilder question, which mentions that String.Format uses a StringBuilder internally. This makes me wonder if your memory hog lies elsewhere.
**based on James' comment, I should mention that I never do heavy string formatting, as I focus on web-based development.*

Maybe reconsider the TextBox? A ListBox holding string Items will probably perform better.
But the main problem seem to be the requirements, Showing 180,000 items cannot be aimed at a (human) user, neither is changing it in "Real Time".
The preferable way would be to show a sample of the data or a progress indicator.
When you do want to dump it at the poor User, batch string updates. No user could descern more than 2 or 3 changes per second. So if you produce 100/second, make groups of 50.

Some responses have alluded to it, but nobody has outright stated it which is surprising.
Strings are immutable which means a String cannot be modified after it is created. Therefore, every time you concatenate to an existing String, a new String Object needs to be created. The memory associated with that String Object also obviously needs to be created, which can get expensive as your Strings become larger and larger. In college, I once made the amateur mistake of concatenating Strings in a Java program that did Huffman coding compression. When you're concatenating extremely large amounts of text, String concatenation can really hurt you when you could have simply used StringBuilder, as some in here have mentioned.

Use the StringBuilder as suggested.
Try to estimate the final string size then use that number when instantiating the StringBuilder. StringBuilder sb = new StringBuilder(estSize);
When updating the TextBox just use assignment eg: textbox.text = sb.ToString();
Watch for cross-thread operations as above. However use BeginInvoke. No need to block
the background thread while the UI updates.

A) Intro: already mentioned, use StringBuilder
B) Point: don't update too frequently, i.e.
DateTime dtLastUpdate = DateTime.MinValue;
while (condition)
{
DoSomeWork();
if (DateTime.Now - dtLastUpdate > TimeSpan.FromSeconds(2))
{
_form.Invoke(() => {textBox.Text = myStringBuilder.ToString()});
dtLastUpdate = DateTime.Now;
}
}
C) If that's one-time job, use x64 architecture to stay within 2Gb limit.

StringBuilder in ViewModel will avoid string rebindings mess and bind it to MyTextBox.Text. This scenario will increase performance many times over and decrease memory usage.

Something that has not been mentioned is that even if you're performing the operation in the background thread, the update of the UI element itself HAS to happen on the main thread itself (in WinForms anyway).
When updating your textbox, do you have any code that looks like
if(textbox.dispatcher.checkAccess()){
textbox.text += "whatever";
}else{
textbox.dispatcher.invoke(...);
}
If so, then your background op is definitely being bottlenecked by the UI Update.
I would suggest that your background op use StringBuilder as noted above, but instead of updating the textbox every cycle, try updating it at regular intervals to see if it increases performance for you.
EDIT NOTE:have not used WPF.

You say memory grows exponentially. No, it is a quadratic growth, i.e. a polynomial growth, which is not as dramatic as an exponential growth.
You are creating strings holding the following number of items:
1 + 2 + 3 + 4 + 5 ... + n = (n^2 + n) /2.
With n = 180,000 you get total memory allocation for 16,200,090,000 items, i.e. 16.2 billion items! This memory will not be allocated at once, but it is a lot of cleanup work for the GC (garbage collector)!
Also, bear in mind, that the previous string (which is growing) must be copied into the new string 179,999 times. The total number of copied bytes goes with n^2 as well!
As others have suggested, use a ListBox instead. Here you can append new strings without creating a huge string. A StringBuild does not help, since you want to display the intermediate results as well.

Writing huge amounts of text to a textbox

I am writing a log of lots and lots of formatted text to a textbox in a .net windows form app.
It is slow once the data gets over a few megs. Since I am appending the string has to be reallocated every time right? I only need to set the value to the text box once, but in my code I am doing line+=data tens of thousands of times.
Is there a faster way to do this? Maybe a different control? Is there a linked list string type I can use?

StringBuilder will not help if the text box is added to incrementally, like log output for example.
But, if the above is true and if your updates are frequent enough it may behoove you to cache some number of updates and then append them in one step (rather than appending constantly). That would save you many string reallocations... and then StringBuilder would be helpful.
Notes:
Create a class-scoped StringBuilder member (_sb)
Start a timer (or use a counter)
Append text updates to _sb
When timer ticks or certain counter reached reset and append to
text box
restart process from #1

No one has mentioned virtualization yet, which is really the only way to provide predictable performance for massive volumes of data. Even using a StringBuilder and converting it to a string every half a second will be very slow once the log gets large enough.
With data virtualization, you would only hold the necessary data in memory (i.e. what the user can see, and perhaps a little more on either side) whilst the rest would be stored on disk. Old data would "roll out" of memory as new data comes in to replace it.
In order to make the TextBox appear as though it has a lot of data in it, you would tell it that it does. As the user scrolls around, you would replace the data in the buffer with the relevant data from the underlying source (using random file access). So your UI would be monitoring a file, not listening for logging events.
Of course, this is all a lot more work than simply using a StringBuilder, but I thought it worth mentioning just in case.

Build your String together with a StringBuilder, then convert it to a String using toString(), and assign this to the textbox.

I have found that setting the textbox's WordWrap property to false greatly improves performance, as long as you're ok with having to scroll to the right to see all of your text. In my case, I wanted to paste a 20-50 MB file into a MultiLine textbox to do some processing on it. That took several minutes with WordWrap on, and just several seconds with WordWrap off.

C# Application Becomes Slow and Unresponsive as String in Multiline Textbox Grows

I have a C# application in which a LOT of information is being added to a Textbox for display to the user. Upon processing of the data, almost immediately, the application becomes very slow and unresponsive. This is how I am currently attempting to handle this:
var saLines = textBox1.Lines;
var saNewLines = saLines.Skip(50);
textBox1.Lines = saNewLines.ToArray();
This code is run from a timer every 100mS. Is there a better way to handle this? I am using Microsoft Visual C# 2008 Express Edition. Thanks.

The simple answer is TextBox.AppendText().
You get much better performance initially.
I tested writing a 500 char message every 20 ms for 2 mins (with BackgroundWorker) and the UI remained responsive and CPU minimal. At some point, of course, it will become unresponsive but it was good enough for my needs.

Try by having in memory a list with the content, and removing the first 50 elements by RemoveRange and then going with ToArray();
Like this :
lst.RemoveRange(0,50);
textBox1.Lines = lst.ToArray();
It should be a lot faster.

I'd say your main problem here is that you are using the TextBox as your primary storage for your text. Everytime you call TextBox.Lines, the string is split on Environment.NewLine.
Try turning it around:
Store the text in a new List<String>(maxLines)
In your AddLine method, check the length of your text buffer and use RemoveRange(0, excessCount)
Update your display TextBox by calling String.Join(Environment.NewLine, textBuffer.ToArray())
That last call is a bit expensive, but it should stop your slowdowns. To get it any faster you'd need to use a statically sized string array and move the references around yourself.

The most efficient way to trim an array is to create a new array of the desired size, then use Array.Copy to copy the desired portion of the old array.
I would recommend that you maintain a List<string> containing all of your lines.
You should use a StringBuilder to build a string containing the lines you're looking for, and set the textbox's Text proeprty to the StringBuilder's string. For added performance, set the StringBuilder's capacity to a reasonable guess of the final sie of the string. (Or to list.Skip(...).Take(...).Sum(s => s.Length))
If you're concerned about memory, you can trim the List<string> by calling RemoveRange.
As long as you don't put too much in the textbox at once, doing it this way should be extremely fast. All of the manipulation of the List<string> and the StringBuilder can be done in a background thread, and you can pass the completed string to the UI thread.
The TextBox.Lines property simply concatenates the array you give it using a StringBuilder, so there's no point in using it (and making a needless array).

Instead of splitting the text and then re-joining it, just get the sub-string from the 51st line:
int i = textBox1.GetFirstCharIndexFromLine(50);
if (i > 0) textBox1.Text = textBox1.Text.Substring(i);