Workaround for C # CodeDom causing stack overflow (CS1647) in csc.exe?

Question

I have a situation where I need to create a class with a large string const. The code outside my control causes the generated CodeDom tree to be emitted to a C # source, and then later compiled as part of a larger assembly.

Unfortunately, I came across a situation where the length of this line exceeds 335440 characters in Win2K8 x64 (926240 in Win2K3 x86), the C # compiler comes out with a fatal error:

fatal error CS1647: expression is too long or too complicated to compile next to 'int'

MSDN says that CS1647 is "stack overflow in the compiler" (no pun intended!). Looking more closely, I determined that CodeDom “beautifully” wraps my const string with 80 characters. This forces the compiler to concatenate over 4,193 string fragments, which appear to be the depth of the C # compiler stack in x64 NetFx. CSC.exe must internally recursively evaluate this expression to "rehydrate" my single line.

My initial question is this: “ does anyone know about the work to change the way the code generator is generated? ” I cannot control the fact that the external system uses the C # source as an intermediate and I want this to be a constant ( rather than concatenating strings at runtime).

Alternatively, how can I formulate this expression in such a way that after a certain number of characters I can still create a constant, but it consists of several large pieces?

Full play here:

// this string breaks CSC: 335440 is Win2K8 x64 max, 926240 is Win2K3 x86 max string HugeString = new String('X', 926300); CodeDomProvider provider = CodeDomProvider.CreateProvider("C#"); CodeCompileUnit code = new CodeCompileUnit(); // namespace Foo {} CodeNamespace ns = new CodeNamespace("Foo"); code.Namespaces.Add(ns); // public class Bar {} CodeTypeDeclaration type = new CodeTypeDeclaration(); type.IsClass = true; type.Name = "Bar"; type.Attributes = MemberAttributes.Public; ns.Types.Add(type); // public const string HugeString = "XXXX..."; CodeMemberField field = new CodeMemberField(); field.Name = "HugeString"; field.Type = new CodeTypeReference(typeof(String)); field.Attributes = MemberAttributes.Public|MemberAttributes.Const; field.InitExpression = new CodePrimitiveExpression(HugeString); type.Members.Add(field); // generate class file using (TextWriter writer = File.CreateText("FooBar.cs")) { provider.GenerateCodeFromCompileUnit(code, writer, new CodeGeneratorOptions()); } // compile class file CompilerResults results = provider.CompileAssemblyFromFile(new CompilerParameters(), "FooBar.cs"); // output reults foreach (string msg in results.Output) { Console.WriteLine(msg); } // output errors foreach (CompilerError error in results.Errors) { Console.WriteLine(error); } 

Answer 1

Using the CodeSnippetExpression expression and the quoted string manually, I was able to emit a source that I would like to see from Microsoft.CSharp.CSharpCodeGenerator.

To answer the question above, replace this line:

 field.InitExpression = new CodePrimitiveExpression(HugeString); 

with this:

 field.InitExpression = new CodeSnippetExpression(QuoteSnippetStringCStyle(HugeString)); 

And finally, change the private line quoting the Microsoft.CSharp.CSharpCodeGenerator.QuoteSnippetStringCStyle method to not wrap after 80 characters:

 private static string QuoteSnippetStringCStyle(string value) { // CS1647: An expression is too long or complex to compile near '...' // happens if number of line wraps is too many (335440 is max for x64, 926240 is max for x86) // CS1034: Compiler limit exceeded: Line cannot exceed 16777214 characters // theoretically every character could be escaped unicode (6 chars), plus quotes, etc. const int LineWrapWidth = (16777214/6) - 4; StringBuilder b = new StringBuilder(value.Length+5); b.Append("\r\n\""); for (int i=0; i<value.Length; i++) { switch (value[i]) { case '\u2028': case '\u2029': { int ch = (int)value[i]; b.Append(@"\u"); b.Append(ch.ToString("X4", CultureInfo.InvariantCulture)); break; } case '\\': { b.Append(@"\\"); break; } case '\'': { b.Append(@"\'"); break; } case '\t': { b.Append(@"\t"); break; } case '\n': { b.Append(@"\n"); break; } case '\r': { b.Append(@"\r"); break; } case '"': { b.Append("\\\""); break; } case '\0': { b.Append(@"\0"); break; } default: { b.Append(value[i]); break; } } if ((i > 0) && ((i % LineWrapWidth) == 0)) { if ((Char.IsHighSurrogate(value[i]) && (i < (value.Length - 1))) && Char.IsLowSurrogate(value[i + 1])) { b.Append(value[++i]); } b.Append("\"+\r\n"); b.Append('"'); } } b.Append("\""); return b.ToString(); } 

Answer 2

I am correctly saying that you have a C # source file with something like:

 public const HugeString = "xxxxxxxxxxxx...." + "yyyyy....." + "zzzzz....."; 

and then you try to compile it?

If so, I would try to edit the text file (in the code, of course) before compiling. This should be relatively simple, as they are likely to follow a rigidly defined pattern (compared to human-generated source code). Convert it to have one massive line for each constant. Let me know if you want to try some sample code.

By the way, your player succeeds without errors on my box - what version of the frame are you using? (There is beta 4.0 in my box, which can affect things.)

EDIT: How about changing it so as not to be a string constant? You need to break it yourself and release it as a read-only public static field:

 public static readonly HugeString = "xxxxxxxxxxxxxxxx" + string.Empty + "yyyyyyyyyyyyyyyyyyy" + string.Empty + "zzzzzzzzzzzzzzzzzzz"; 

Actually, string.Empty is a public static readonly field, not a constant. This means that the C # compiler simply issues a call to string.Concat , which may well be in order. This will happen only once at runtime, of course, more slowly than doing it at compile time, but it may be an easier workaround than anything else.

Answer 3

Note that if you declare a string as const, it will be copied in every assembly that uses this string in its code.

You might be better off with static readonly.

Another way is to declare a readonly property that returns a string.

Answer 4

I don't know how to change the behavior of the code generator, but you can change the size of the stack that the compiler uses with / stack EditBin.EXE .

Example:

 editbin /stack:100000,1000 csc.exe <options> 

The following is an example of its use:

 class App { private static long _Depth = 0; // recursive function to blow stack private static void GoDeep() { if ((++_Depth % 10000) == 0) System.Console.WriteLine("Depth is " + _Depth.ToString()); GoDeep(); return; } public static void Main() { try { GoDeep(); } finally { } return; } } editbin /stack:100000,1000 q.exe Depth is 10000 Depth is 20000 Unhandled Exception: StackOverflowException. editbin /stack:1000000,1000 q.exe Depth is 10000 Depth is 20000 Depth is 30000 Depth is 40000 Depth is 50000 Depth is 60000 Depth is 70000 Depth is 80000 Unhandled Exception: StackOverflowException. 

Answer 5

Verify that application pools in IIS include 32-bit applications. This is all I needed to fix this problem while trying to compile a 32-bit application in 64-bit Win7. Oddly enough (or not), Microsoft could not provide this answer. After a full day of searching, I found this fix link on the Iron Speed ​​Designer forum:

http://darrell.mozingo.net/2009/01/17/running-iis-7-in-32-bit-mode/