Develop Reference

Time complexity for C# StringBuilder initialized with string [duplicate]

How does StringBuilder work?
What does it do internally? Does it use unsafe code?
And why is it so fast (compared to the + operator)?

When you use the + operator to build up a string:
string s = "01";
s += "02";
s += "03";
s += "04";
then on the first concatenation we make a new string of length four and copy "01" and "02" into it -- four characters are copied. On the second concatenation we make a new string of length six and copy "0102" and "03" into it -- six characters are copied. On the third concat, we make a string of length eight and copy "010203" and "04" into it -- eight characters are copied. So far a total of 4 + 6 + 8 = 18 characters have been copied for this eight-character string. Keep going.
...
s += "99";
On the 98th concat we make a string of length 198 and copy "010203...98" and "99" into it. That gives us a total of 4 + 6 + 8 + ... + 198 = a lot, in order to make this 198 character string.
A string builder doesn't do all that copying. Rather, it maintains a mutable array that is hoped to be larger than the final string, and stuffs new things into the array as necessary.
What happens when the guess is wrong and the array gets full? There are two strategies. In the previous version of the framework, the string builder reallocated and copied the array when it got full, and doubled its size. In the new implementation, the string builder maintains a linked list of relatively small arrays, and appends a new array onto the end of the list when the old one gets full.
Also, as you have conjectured, the string builder can do tricks with "unsafe" code to improve its performance. For example, the code which writes the new data into the array can already have checked that the array write is going to be within bounds. By turning off the safety system it can avoid the per-write check that the jitter might otherwise insert to verify that every write to the array is safe. The string builder does a number of these sorts of tricks to do things like ensuring that buffers are reused rather than reallocated, ensuring that unnecessary safety checks are avoided, and so on. I recommend against these sorts of shenanigans unless you are really good at writing unsafe code correctly, and really do need to eke out every last bit of performance.

StringBuilder's implementation has changed between versions, I believe. Fundamentally though, it maintains a mutable structure of some form. I believe it used to use a string which was still being mutated (using internal methods) and would just make sure it would never be mutated after it was returned.
The reason StringBuilder is faster than using string concatenation in a loop is precisely because of the mutability - it doesn't require a new string to be constructed after each mutation, which would mean copying all the data within the string etc.
For just a single concatenation, it's actually slightly more efficient to use + than to use StringBuilder. It's only when you're performing multiple operations and you don't really need the intermediate results that StringBuilder shines.
See my article on StringBuilder for more information.

The Microsoft CLR does do some operations with internal call (not quite the same as unsafe code). The biggest performance benefit over a bunch of + concatenated strings is that it writes to a char[] and doesn't create as many intermediate strings. When you call ToString (), it builds a completed, immutable string from your contents.

The StringBuilder uses a string buffer that can be altered, compared to a regular String that can't be. When you call the ToString method of the StringBuilder it will just freeze the string buffer and convert it into a regular string, so it doesn't have to copy all the data one extra time.
As the StringBuilder can alter the string buffer, it doesn't have to create a new string value for each and every change to the string data. When you use the + operator, the compiler turns that into a String.Concat call that creates a new string object. This seemingly innocent piece of code:
str += ",";
compiles into this:
str = String.Concat(str, ",");

C# Why doesn't the resulting string from concatenation using variables get interned the way it does when concatenating inline? [duplicate]

I want to know the process and internals of string interning specific to .Net framework. Would also like to know the benefits of using interning and the scenarios/situations where we should use string interning to improve the performance. Though I have studied interning from the Jeffery Richter's CLR book but I am still confused and would like to know it in more detail.
[Editing] to ask a specific question with a sample code as below:
private void MethodA()
{
string s = "String"; // line 1 - interned literal as explained in the answer
//s.intern(); // line 2 - what would happen in line 3 if we uncomment this line, will it make any difference?
}
private bool MethodB(string compareThis)
{
if (compareThis == "String") // line 3 - will this line use interning (with and without uncommenting line 2 above)?
{
return true;
}
return false;
}

In general, interning is something that just happens, automatically, when you use literal string values. Interning provides the benefit of only having one copy of the literal in memory, no matter how often it's used.
That being said, it's rare that there is a reason to intern your own strings that are generated at runtime, or ever even think about string interning for normal development.
There are potentially some benefits if you're going to be doing a lot of work with comparisons of potentially identical runtime generated strings (as interning can speed up comparisons via ReferenceEquals). However, this is a highly specialized usage, and would require a fair amount of profiling and testing, and wouldn't be an optimization I'd consider unless there was a measured problem in place.

This is an "old" question, but I have a different angle on it.
If you're going to have a lot of long-lived strings from a small pool, interning can improve memory efficiency.
In my case, I was interning another type of object in a static dictionary because they were reused frequently, and this served as a fast cache before persisting them to disk.
Most of the fields in these objects are strings, and the pool of values is fairly small (much smaller than the number of instances, anyway).
If these were transient objects, it wouldn't matter because the string fields would be garbage collected often. But because references to them were being held, their memory usage started to accumulate (even when no new unique values were being added).
So interning the objects reduced the memory usage substantially, and so did interning their string values while they were being interned.

Interning is an internal implementation detail. Unlike boxing, I do not think there is any benefit in knowing more than what you have read in Richter's book.
Micro-optimisation benefits of interning strings manually are minimal hence is generally not recommended.
This probably describes it:
class Program
{
const string SomeString = "Some String"; // gets interned
static void Main(string[] args)
{
var s1 = SomeString; // use interned string
var s2 = SomeString; // use interned string
var s = "String";
var s3 = "Some " + s; // no interning
Console.WriteLine(s1 == s2); // uses interning comparison
Console.WriteLine(s1 == s3); // do NOT use interning comparison
}
}

Interned strings have the following characteristics:
Two interned strings that are identical will have the same address in memory.
Memory occupied by interned strings is not freed until your application terminates.
Interning a string involves calculating a hash and looking it up in a dictionary which consumes CPU cycles.
If multiple threads intern strings at the same time they will block each other because accesses to the dictionary of interned strings are serialized.
The consequences of these characteristics are:
You can test two interned strings for equality by just comparing the address pointer which is a lot faster than comparing each character in the string. This is especially true if the strings are very long and start with the same characters. You can compare interned strings with the Object.ReferenceEquals method, but it is safer to use the string == operator because it checks to see if the strings are interned first.
If you use the same string many times in your application, your application will only store one copy of the string in memory reducing the memory required to run your application.
If you intern many different strings this will allocate memory for those strings that will never be freed, and your application will consume ever increasing amounts of memory.
If you have a very large number of interned strings, string interning can become slow, and threads will block each other when accessing the interned string dictionary.
You should use string interning only if:
The set of strings you are interning is fairly small.
You compare these strings many times for each time that you intern them.
You really care about minute performance optimizations.
You don't have many threads aggressively interning strings.

Internalization of strings affects memory consumption.
For example if you read strings and keep them it in a list for caching; and the exact same string occurs 10 times, the string is actually stored only once in memory if string.Intern is used. If not, the string is stored 10 times.
In the example below, the string.Intern variant consumes about 44 MB and the without-version (uncommented) consumes 1195 MB.
static void Main(string[] args)
{
var list = new List<string>();
for (int i = 0; i < 5 * 1000 * 1000; i++)
{
var s = ReadFromDb();
list.Add(string.Intern(s));
//list.Add(s);
}
Console.WriteLine(Process.GetCurrentProcess().PrivateMemorySize64 / 1024 / 1024 + " MB");
}
private static string ReadFromDb()
{
return "abcdefghijklmnopqrstuvyxz0123456789abcdefghijklmnopqrstuvyxz0123456789abcdefghijklmnopqrstuvyxz0123456789" + 1;
}
Internalization also improves performance for equals-compare. The example below the intern version takes about 1 time units while the non-intern takes 7 time units.
static void Main(string[] args)
{
var a = string.Intern(ReadFromDb());
var b = string.Intern(ReadFromDb());
//var a = ReadFromDb();
//var b = ReadFromDb();
int equals = 0;
var stopwatch = Stopwatch.StartNew();
for (int i = 0; i < 250 * 1000 * 1000; i++)
{
if (a == b) equals++;
}
stopwatch.Stop();
Console.WriteLine(stopwatch.Elapsed + ", equals: " + equals);
}

Worth having a StringBuilder for 5 concatenations? [duplicate]

This question already has answers here:
Closed 13 years ago.
Possible Duplicate:
At what point does using a StringBuilder become insignificant or an overhead?
Related/Duplicate Questions
String vs StringBuilder
At what point does using a StringBuilder become insignificant or an overhead?
As plain as possible I have this method 1:
cmd2.CommandText = ("insert into " + TableName + " values (" + string.Join(",", insertvalues) + ");");
I am wondering if method 2 be faster if I would do:
StringBuilder sb2 = new StringBuilder();
sb2.Append("insert into ");
sb2.Append(TableName);
sb2.Append(" values (");
sb2.Append(string.Join(",", insertvalues));
sb2.Append(");");
cmd2.CommandText = sb2.ToString();

You could also try String.Format, which I believe uses a StringBuilder internally but has increased readability.
cmd2.CommandText = string.Format("insert into {0} values ({1});", TableName, string.Join(",", insertvalues));
(This is for C#)

For small programs this will be a premature optimization.
If you want to take into consideration these kinds of optimization then better measure it, because this depends on the size of the string concatenated also, apart from the number or appends.

Besides that IMO the StringBuilder method looks and reads better the StringBuilder does outperform string concatenation after 5 to 10 added strings according to http://dotnetperls.com/stringbuilder-performance

In C# an expression in the form "a" + b + "c" is optimized by the compiler into String.Concat("a", b, "c") so you will not get intermediary strings. This would be more efficient than a StringBuilder.

From here:
The Java language provides special
support for the string concatenation
operator ( + ), and for conversion of
other objects to strings. String
concatenation is implemented through
the StringBuffer class and its append
method.
So it would seem that the compiler is using StringBuffer on your behalf anyway.

A good compiler should optimize this for you - but don't take my word for it when you can easily find out for yourself.
Unless you are doing this in a tight loop, then the difference in performance is likely to be insignificant.
String concatenation of values is usually a bad way to construct SQL statements when you could use bind variables instead. This allows the database to optimize the queries. Using bind is likely to make a much bigger difference than optimizing your string construction - and with bind you only need to construct the string once per session instead of once per query.

Use String.format.
String.format("insert into {0} values({1});",TableName,string.Join(",", insertvalues));
It's more readable.

Your two methods will not differ in performance, the reason is that your string is concatenaled in 1 expression, and the compiler will create a StringBuilder for that expression. The following are equivalent and result in the same code:
String s1 = "five" + '=' + 5;
String s2 = new StringBuilder().append("five").append('=').append(5).toString();
If your code splits up the expresion, for instance in a loop, creating your own StringBuilder will perform better, the naive version using string + concatenation results after compilation in code like:
String s3 = "";
for (int n = 0; n < 5; n++) {
s3 = new StringBuilder(s3).append(getText(n)).append('=').append(n).append("\n").toString();
}
creating your method using an explicit StringBuilder can save creation of unnecessary StringBuilder objects.
For simple methods you normally do not have to optimise string concatenation yourself, but in those situations where the code is on the critical path, or where you are forced to use many different string expression to build up your end result, it is a good to know what happens under the hood so you can decide whether it is worth the extra effort.
Note that StringBuffer is thread-safe, while StringBuilder is not. StringBuilder is the faster choice for situations where multi-threaded access is not an issue.

String Concatenation unsafe in C#, need to use StringBuilder?

My question is this: Is string concatenation in C# safe? If string concatenation leads to unexpected errors, and replacing that string concatenation by using StringBuilder causes those errors to disappear, what might that indicate?
Background: I am developing a small command line C# application. It takes command line arguments, performs a slightly complicated SQL query, and outputs about 1300 rows of data into a formatted XML file.
My initial program would always run fine in debug mode. However, in release mode it would get to about the 750th SQL result, and then die with an error. The error was that a certain column of data could not be read, even through the Read() method of the SqlDataReader object had just returned true.
This problem was fixed by using StringBuilder for all operations in the code, where previously there had been "string1 + string2". I'm not talking about string concatenation inside the SQL query loop, where StringBuilder was already in use. I'm talking about simple concatenations between two or three short string variables earlier in the code.
I had the impression that C# was smart enough to handle the memory management for adding a few strings together. Am I wrong? Or does this indicate some other sort of code problem?

To answer your question:
String contatenation in C# (and .NET in general) is "safe", but doing it in a tight loop as you describe is likely to cause severe memory pressure and put strain on the garbage collector.
I would hazard a guess that the errors you speak of were related to resource exhaustion of some sort, but it would be helpful if you could provide more detail — for example, did you receive an exception? Did the application terminate abnormally?
Background:
.NET strings are immutable, so when you do a concatenation like this:
var stringList = new List<string> {"aaa", "bbb", "ccc", "ddd", //... };
string result = String.Empty;
foreach (var s in stringList)
{
result = result + s;
}
This is roughly equivalent to the following:
string result = "";
result = "aaa"
string temp1 = result + "bbb";
result = temp1;
string temp2 = temp1 + "ccc";
result = temp2;
string temp3 = temp2 + "ddd";
result = temp3;
// ...
result = tempN + x;
The purpose of this example is to emphasise that each time around the loop results in the allocation of a new temporary string.
Since the strings are immutable, the runtime has no alternative options but to allocate a new string each time you add another string to the end of your result.
Although the result string is constantly updated to point to the latest and greatest intermediate result, you are producing a lot of these un-named temporary string that become eligible for garbage collection almost immediately.
At the end of this concatenation you will have the following strings stored in memory (assuming, for simplicity, that the garbage collector has not yet run).
string a = "aaa";
string b = "bbb";
string c = "ccc";
// ...
string temp1 = "aaabbb";
string temp2 = "aaabbbccc";
string temp3 = "aaabbbcccddd";
string temp4 = "aaabbbcccdddeee";
string temp5 = "aaabbbcccdddeeefff";
string temp6 = "aaabbbcccdddeeefffggg";
// ...
Although all of these implicit temporary variables are eligible for garbage collection almost immediately, they still have to be allocated. When performing concatenation in a tight loop this is going to put a lot of strain on the garbage collector and, if nothing else, will make your code run very slowly. I have seen the performance impact of this first hand, and it becomes truly dramatic as your concatenated string becomes larger.
The recommended approach is to always use a StringBuilder if you are doing more than a few string concatenations. StringBuilder uses a mutable buffer to reduce the number of allocations that are necessary in building up your string.

String concatenation is safe though more memory intensive than using a StringBuilder if contatenating large numbers of strings in a loop. And in extreme cases you could be running out of memory.
It's almost certainly a bug in your code.
Maybe you're contatenating a very large number of strings. Or maybe it's something else completely different.
I'd go back to debugging without any preconceptions of the root cause - if you're still having problems try to reduce it to the minimum needed to repro the problem and post code.

Apart from what you're doing is probably best done with XML APIs instead of strings or StringBuilder I doubt that the error you see is due to string concatenation. Maybe switching to StringBuilder just masked the error or went over it gracefully, but I doubt using strings really was the cause.

How long would it take the concatenation version vs the string builder version? It's possible that your connection to the DB is being closed. If you are doing a lot of concatenation, i would go w/ StringBuilder as it is a bit more efficient.

One cause may be that strings are immutable in .Net so when you do an operation on one such as concatenation you are actually creating a new string.
Another possible cause is that string length is an int so the maximum possible length is Int32.MaxValue or 2,147,483,647.
In either case a StringBuilder is better than "string1 + string2" for this type of operation. Although, using the built-in XML capabilities would be even better.

string.Concat(string[]) is by far the fastest way to concatenate strings. It litterly kills StringBuilder in performance when used in loops, especially if you create the StringBuilder in each iteration.
There are loads of references if you Google "c# string format vs stringbuilder" or something like that.
http://www.codeproject.com/KB/cs/StringBuilder_vs_String.aspx gives you an ideer about the times. Here string.Join wins the concatenation test but I belive this is because the string.Concat(string, string) is used instead of the overloaded version that takes an array.
If you take a look at the MSIL code that is generated by the different methods you'll see what going on beneath the hood.

Here is my shot in the dark...
Strings in .NET (not stringbuilders) go into the String Intern Pool. This is basically an area managed by the CLR to share strings to improve performance. There has to be some limit here, although I have no idea what that limit is. I imagine all the concatenation you are doing is hitting the ceiling of the string intern pool. So SQL says yes I have a value for you, but it can't put it anywhere so you get an exception.
A quick and easy test would be to nGen your assembly and see if you still get the error. After nGen'ing, you application no longer will use the pool.
If that fails, I'd contact Microsoft to try and get some hard details. I think my idea sounds plausible, but I have no idea why it works in debug mode. Perhaps in debug mode strings aren't interned. I am also no expert.

When compounding strings together I always use StringBuilder. It's designed for it and is more efficient that simply using "string1 + string2".

automatic incrementing filename

I have a filename that has the following format:
timestamp-username-1
This file is constantly etting written to, but before it gets too large I want to create a new file.
timestamp-username-2
How can I acheive this with using least amount of memory (ie, no or little variables)
here is my version:
private void Split() {
char[] strArr = FlowArgs.Filename.ToCharArray();
int num;
//get the last number
if(Int32.TryParse(strArr[strArr.Length - 1].ToString(), out num)) {
num += 1;
}
//replace the old number with the new number
char.TryParse(num.ToString(), out strArr[strArr.Length - 1]);
FlowArgs.Filename = strArr.ToString();
}
Edit:
I have added a "version" property (int) in the FlowArgs class. However my new problem is that how can I append this at the end of thefilename

I think you should just store the counter in an int. I understand that you want to save memory space but to be honest, an extra int is really in the "acceptable" category. I mean, the Int32 parser is probably wasting way much more memory. Don't forget that on x86, the memory space is spitted to 4096 byte pages so there is much more memory wasted than these 4 bytes.
EDIT: You probably want to have a method like GetNextFileName() in your class that generates you the next filename (being able to refactor your code into small bits is important, much more important than saving memory space):
private int nextFileNumber = 0;
private string GetNextFileName(string userName)
{
return String.Format("{0}-{1}-{2}", DateTime.Now, userName,
nextFileNumber++);
}

"least amount of memory" is NOT equals to "no or little variables"
Local variables only takes little memory itself.
But object creation in heap takes a lot more, and they require GC to do cleanup.
In your example, your ToCharArray() and ToString() have created 4 object (indirect created object not included).

your string variable is already character array:
int num=0;
//get the last number
if (Int32.TryParse(FolwArgs.Filename[FolwArgs.Filename.Length-1].ToString(), out num))
num++;
//replace the old number with the new number
char.TryParse(num.ToString(), out FolwArgs.Filename[FolwArgs.Filename.Length-1]]);

Instead of using a running counter, consider using datetime of creation as the changing part of your filename. This way, you don't have to store and retrieve the previous value.
Using the ToBinary() method, you can get a numeric representation of the time.
Of course, any time format that is acceptable in a filename can be used - see custom date and time format strings.