StringBuilder.ToString () throws an OutOfMemoryException

Question

I have a StringBuilder length "132370292", when I try to get a string using the ToString() method, it throws an OutOfMemoryException .

  StringBuilder SB = new StringBuilder(); for(int i =0; i<=5000; i++) { SB.Append("Some Junk Data for testing. My Actual Data is created from different sources by Appending to the String Builder."); } try { string str = SB.ToString(); //Throws OOM mostly Console.WriteLine("String Created Successfully"); } catch(OutOfMemoryException ex) { StreamWriter sw = new StreamWriter(@"c:\memo.txt", true); sw.Write(SB.ToString()); //Always writes to the file without any error Console.WriteLine("Written to File Successfully"); } 

What is the reason for OOM when creating a new line and why does it not throw OOM while writing to a file?

Machine Details: 64-bit, Windows-7, RAM 2 GB Note. .NET Version 2.0

Answer 1

What is the reason for OOM when creating a new line

Because you are running out of memory - or at least the CLR cannot allocate an object with the requested size. It's really that simple. If you want to avoid errors, do not try to create strings that do not fit into memory. Note that even if you have a lot of memory, and even if you use the 64-bit CLR, there are limits on the size of objects that you can create.

and why doesn't it throw OOM while writing to a file?

Because you have more disk space than memory.

I'm sure the code is not quite the way you describe. This line will not be able to compile:

 sw.write(SB.ToString()); 

... because the method is Write , not Write . And if you actually call SB.ToString() , then this is as strong as str = SB.ToString() .

It seems more likely that you are actually writing the file in streaming mode, for example.

 using (var writer = File.CreateText(...)) { for (int i = 0; i < 5000; i++) { writer.Write(mytext); } } 

Thus, you never need to have a huge amount of text in memory - it just writes it to disk, as it is, perhaps with some buffering, but not enough for memory problems.

Answer 2

Workaround: Suppose you want to write a large string stored in a StringBuilder in a StreamWriter, I would write to avoid SB.ToString OOM exception. But if your OOM exception is related to adding StringBuilder content, you should work on this.

 public const int CHUNK_STRING_LENGTH = 30000; while (SB.Length > CHUNK_STRING_LENGTH ) { sw.Write(SB.ToString(0, CHUNK_STRING_LENGTH )); SB.Remove(0, CHUNK_STRING_LENGTH ); } sw.Write(SB); 

Answer 3

You must remember that strings in .NET are stored in memory in 16-bit Unicode. This means that a string 132370292 in length will require 260 MB of RAM.

Also performing

 string str = SB.ToString(); 

you create a COPY of your string (another 260 MB).

Keep in mind that each process has its own RAM limit, so an OutOfMemoryException can be thrown even if you have free RAM.