Live debugging

Question

I have a managed Windows Service application that sometimes crashes into production due to a managed StackOverFlowException. I know this because I ran adplus in crash mode and analyzed the crash popup using SoS. I even attached a windbg debugger and set it for a "go unhandled exception".

My problem: I do not see any of the managed stacks or switch to any of the threads. They all break down by the time the debugger breaks.

I am not a Windbg expert, and not to mention installing Visual Studio on a live system or remotely debugging and debugging with this tool, does anyone have any suggestions on how I can get a stack trace due to an insult a thread?

That's what I'm doing.

! Threads

...

XXXX 11 27c 000000001b2175f0 b220 Disabled 00000000072c9058: 00000000072cad80 0000000019bdd3f0 0 Ukr System.StackOverflowException (0000000000c010d0)

...

And at this point you will see the identifier XXXX indicating that the stream is pretty dead.

Answer 1

Once you hit the stack overflow, you are pretty unlucky for debugging the problem - flushing your stack space leaves your program in a non-deterministic state, so you cannot rely on any information in this place - any stack trace you are trying to get can be damaged and can easily point you in the wrong direction. That is, as soon as a StackOverflowException occurs, it is too late.

Also, according to the documentation , you cannot catch a StackOverflowException with .Net 2.0, so other suggestions to surround your attempt / catch code for this probably won't work. That makes sense, given the side effects (I'm surprised .Net has ever let you catch it).

Your only real option is to get bored with code analysis, search for something that could potentially lead to a stack overflow, and place some markers so that you can get an idea of ​​where they occur before they happen. For example, it’s obvious that all recursive methods are the first place to start, so give them a depth counter and throw your own exception if they get some kind of “unreasonable” value that you define, so you can actually get a valid stack trace .

Answer 2

Is it possible to wrap my code with a try-catch that writes to EventLog (or a file or something else) and runs this debug one-time?

 try { ... } catch(SOE) { EventLog.Write(...); throw; } 

You cannot debug, but you will get a stack trace.

Answer 3

One option is to use the try / catch block at a high level, and then print or write the stack trace provided by the exception. Each exception has a StackTrace property that can tell you where it came from. This will not allow you to do any kind of interactive debugging, but it should give you a place to start.

Answer 4

For what costs, starting with .NET 4.0, Visual Studio (and any debuggers that rely on ICorDebug api) get the opportunity to debug mini-disks. This means that you can load the crash dump into the VS debugger on another computer and view managed stacks similar to how you attached the debugger during the crash. See the PDC Discussion Section or Rick Byers' blog for more information. Unfortunately, this will not help you solve this problem, but maybe it will be the next time you encounter this problem.

Answer 5

Take a look at your ADPLUS debug log. See if there are any access violations or true source exceptions that are thrown before the StackOverflowException is thrown.

My guess is that there is an exception in the thread stack that you delay until the thread exits.

You can also use DebugDiag from www.iis.net, and then set a crash rule and create a complete dump file for violated access rights (sxe av) and your own stack exceptions (sxe sov)

Thanks Aaron

Answer 6

I have a RecursionChecker class for this kind of thing. I hereby waive the copyright to the code below.

He complains that he too often gets on the check for the target. This is not all-all-over; For example, loops can cause false positives. You can avoid this by calling another call after the risky code, telling the controller that it can reduce its recursion call for the target. That would still not be bulletproof.

To use it, I just call

 public void DangerousMethod() { RecursionChecker.Check(someTargetObjectThatWillBeTheSameIfWeReturnHereViaRecursion); // recursion-risky code here. } 

Here is the RecursionChecker class:

 /// <summary>If you use this class frequently from multiple threads, expect a lot of blocking. In that case, /// might want to make this a non-static class and have an instance per thread.</summary> public static class RecursionChecker { #if DEBUG private static HashSet<ReentrancyInfo> ReentrancyNotes = new HashSet<ReentrancyInfo>(); private static object LockObject { get; set; } = new object(); private static void CleanUp(HashSet<ReentrancyInfo> notes) { List<ReentrancyInfo> deadOrStale = notes.Where(info => info.IsDeadOrStale()).ToList(); foreach (ReentrancyInfo killMe in deadOrStale) { notes.Remove(killMe); } } #endif public static void Check(object target, int maxOK = 10, int staleMilliseconds = 1000) { #if DEBUG lock (LockObject) { HashSet<ReentrancyInfo> notes = RecursionChecker.ReentrancyNotes; foreach (ReentrancyInfo note in notes) { if (note.HandlePotentiallyRentrantCall(target, maxOK)) { break; } } ReentrancyInfo newNote = new ReentrancyInfo(target, staleMilliseconds); newNote.HandlePotentiallyRentrantCall(target, maxOK); RecursionChecker.CleanUp(notes); notes.Add(newNote); } #endif } } 

helper classes below:

 internal class ReentrancyInfo { public WeakReference<object> ReentrantObject { get; set;} public object GetReentrantObject() { return this.ReentrantObject?.TryGetTarget(); } public DateTime LastCall { get; set;} public int StaleMilliseconds { get; set;} public int ReentrancyCount { get; set;} public bool IsDeadOrStale() { bool r = false; if (this.LastCall.MillisecondsBeforeNow() > this.StaleMilliseconds) { r = true; } else if (this.GetReentrantObject() == null) { r = true; } return r; } public ReentrancyInfo(object reentrantObject, int staleMilliseconds = 1000) { this.ReentrantObject = new WeakReference<object>(reentrantObject); this.StaleMilliseconds = staleMilliseconds; this.LastCall = DateTime.Now; } public bool HandlePotentiallyRentrantCall(object target, int maxOK) { bool r = false; object myTarget = this.GetReentrantObject(); if (target.DoesEqual(myTarget)) { DateTime last = this.LastCall; int ms = last.MillisecondsBeforeNow(); if (ms > this.StaleMilliseconds) { this.ReentrancyCount = 1; } else { if (this.ReentrancyCount == maxOK) { throw new Exception("Probable infinite recursion"); } this.ReentrancyCount++; } } this.LastCall = DateTime.Now; return r; } } public static class DateTimeAdditions { public static int MillisecondsBeforeNow(this DateTime time) { DateTime now = DateTime.Now; TimeSpan elapsed = now.Subtract(time); int r; double totalMS = elapsed.TotalMilliseconds; if (totalMS > int.MaxValue) { r = int.MaxValue; } else { r = (int)totalMS; } return r; } } public static class WeakReferenceAdditions { /// <summary> returns null if target is not available. </summary> public static TTarget TryGetTarget<TTarget> (this WeakReference<TTarget> reference) where TTarget: class { TTarget r = null; if (reference != null) { reference.TryGetTarget(out r); } return r; } }