How to avoid the possible async / await in this program?

Question

The following is an example of a program that may exhibit stack overflow when using the DependsOnPreviousTaskAsync method, which is surprising because I do not believe that explicit synchronous recursion exists.

You can see an example of a stack before overflow at the breakpoint encoded in the DoSomething method. There will be a large long chain of main dependent async state calls.

The logical chain exists because of the relationship between the task and its predecessor, but I am very surprised that this chain of asynchronous calls appears recursively in the call stack!

To get around the problem, I encoded the method as DependsOnPreviousTaskAsync2 , which uses the old-style ContinueWith continuation handling instead of async / await. In this case, the call stack observed at the breakpoint is never very deep.

My question is: is there something I am missing regarding using async / await that will prevent stack overflows? Or did I just hit the edge register, which requires the use of a workaround to break the awesome recursion inherent in the async / wait state machine?

EDIT : I added TaskContinuationOptions.ExecuteSynchronously to the workaround, and although the observed call stack may be deeper, I don't see any StackOverflowExceptions using this method. Regardless of which discovery logic is successfully applied to ContinueWith , while ExecuteSynchronous not applied in the async / wait version.

 class Program { public async static Task<int> DependsOnPreviousTaskAsync(Task<int> previousTask) { if (previousTask == null) return 0; var result = await DoSomethingAsync(previousTask).ConfigureAwait(false); Console.WriteLine(result); return result; } public static Task<int> DependsOnPreviousTaskAsync2(Task<int> previousTask) { // this is a non async/await version of DependsOnPreviousTaskAsync if (previousTask == null) return Task.FromResult(0); var tcs = new TaskCompletionSource<int>(); DoSomethingAsync(previousTask) .ContinueWith(t => { if (t.IsCanceled) { tcs.TrySetCanceled(); } else if (t.IsFaulted) { tcs.TrySetException(t.Exception); } else { Console.WriteLine(t.Result); tcs.TrySetResult(t.Result); } }, TaskContinuationOptions.ExecuteSynchronously); return tcs.Task; } public async static Task<int> DoSomethingAsync(Task<int> previousTask) { var tasksToWaitOn = new Task[] { previousTask, SomethingElseAsync() }; await Task.WhenAll(tasksToWaitOn).ConfigureAwait(false); var previous = ((Task<int>)tasksToWaitOn[0]).Result; if (previous == 500) Debugger.Break(); return previous + 1; } public async static Task SomethingElseAsync() { await Task.Run(() => { Thread.Sleep(2); }); } static void Main(string[] args) { const bool causePossibleStackOverflow = true; Task<int> previous = null; for (var i = 0; i < 100000; i++) { previous = causePossibleStackOverflow ? DependsOnPreviousTaskAsync(previous) : DependsOnPreviousTaskAsync2(previous); } Console.WriteLine(previous.Result); } } 

Here is an example call stack at a breakpoint when using DependsOnPreviousTaskAsync :

 TestAsyncRecursion.exe!TestAsyncRecursion.Program.DoSomethingAsync(System.Threading.Tasks.Task<int> previousTask) Line 62 C# [Resuming Async Method] mscorlib.dll!System.Runtime.CompilerServices.AsyncMethodBuilderCore.MoveNextRunner.InvokeMoveNext(object stateMachine) Unknown mscorlib.dll!System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state, bool preserveSyncCtx) Unknown mscorlib.dll!System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state, bool preserveSyncCtx) Unknown mscorlib.dll!System.Runtime.CompilerServices.AsyncMethodBuilderCore.MoveNextRunner.Run() Unknown mscorlib.dll!System.Runtime.CompilerServices.AsyncMethodBuilderCore.OutputAsyncCausalityEvents<System.Runtime.CompilerServices.AsyncTaskMethodBuilder<int>>.AnonymousMethod__0() Unknown mscorlib.dll!System.Runtime.CompilerServices.AsyncMethodBuilderCore.ContinuationWrapper.Invoke() Unknown mscorlib.dll!System.Runtime.CompilerServices.TaskAwaiter.OutputWaitEtwEvents.AnonymousMethod__0() Unknown mscorlib.dll!System.Runtime.CompilerServices.AsyncMethodBuilderCore.ContinuationWrapper.Invoke() Unknown mscorlib.dll!System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction(System.Action action, bool allowInlining, ref System.Threading.Tasks.Task currentTask) Unknown mscorlib.dll!System.Threading.Tasks.Task.FinishContinuations() Unknown mscorlib.dll!System.Threading.Tasks.Task.FinishStageThree() Unknown mscorlib.dll!System.Threading.Tasks.Task<System.Threading.Tasks.VoidTaskResult>.TrySetResult(System.Threading.Tasks.VoidTaskResult result) Unknown mscorlib.dll!System.Threading.Tasks.Task.WhenAllPromise.Invoke(System.Threading.Tasks.Task completedTask) Unknown mscorlib.dll!System.Threading.Tasks.Task.FinishContinuations() Unknown mscorlib.dll!System.Threading.Tasks.Task.FinishStageThree() Unknown mscorlib.dll!System.Threading.Tasks.Task<System.Threading.Tasks.VoidTaskResult>.TrySetResult(System.Threading.Tasks.VoidTaskResult result) Unknown mscorlib.dll!System.Runtime.CompilerServices.AsyncTaskMethodBuilder<System.Threading.Tasks.VoidTaskResult>.SetResult(System.Threading.Tasks.VoidTaskResult result) Unknown mscorlib.dll!System.Runtime.CompilerServices.AsyncTaskMethodBuilder.SetResult() Unknown TestAsyncRecursion.exe!TestAsyncRecursion.Program.SomethingElseAsync() Line 73 C# [Resuming Async Method] mscorlib.dll!System.Runtime.CompilerServices.AsyncMethodBuilderCore.MoveNextRunner.InvokeMoveNext(object stateMachine) Unknown mscorlib.dll!System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state, bool preserveSyncCtx) Unknown mscorlib.dll!System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state, bool preserveSyncCtx) Unknown mscorlib.dll!System.Runtime.CompilerServices.AsyncMethodBuilderCore.MoveNextRunner.Run() Unknown mscorlib.dll!System.Runtime.CompilerServices.AsyncMethodBuilderCore.OutputAsyncCausalityEvents<System.Runtime.CompilerServices.AsyncTaskMethodBuilder<System.Threading.Tasks.VoidTaskResult>>.AnonymousMethod__0() Unknown mscorlib.dll!System.Runtime.CompilerServices.AsyncMethodBuilderCore.ContinuationWrapper.Invoke() Unknown mscorlib.dll!System.Runtime.CompilerServices.TaskAwaiter.OutputWaitEtwEvents.AnonymousMethod__0() Unknown mscorlib.dll!System.Runtime.CompilerServices.AsyncMethodBuilderCore.ContinuationWrapper.Invoke() Unknown mscorlib.dll!System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction(System.Action action, bool allowInlining, ref System.Threading.Tasks.Task currentTask) Unknown mscorlib.dll!System.Threading.Tasks.Task.FinishContinuations() Unknown mscorlib.dll!System.Threading.Tasks.Task.FinishStageThree() Unknown mscorlib.dll!System.Threading.Tasks.Task.FinishStageTwo() Unknown mscorlib.dll!System.Threading.Tasks.Task.Finish(bool bUserDelegateExecuted) Unknown mscorlib.dll!System.Threading.Tasks.Task.ExecuteWithThreadLocal(ref System.Threading.Tasks.Task currentTaskSlot) Unknown mscorlib.dll!System.Threading.Tasks.Task.ExecuteEntry(bool bPreventDoubleExecution) Unknown mscorlib.dll!System.Threading.Tasks.Task.System.Threading.IThreadPoolWorkItem.ExecuteWorkItem() Unknown mscorlib.dll!System.Threading.ThreadPoolWorkQueue.Dispatch() Unknown mscorlib.dll!System.Threading._ThreadPoolWaitCallback.PerformWaitCallback() Unknown [Async Call] TestAsyncRecursion.exe!TestAsyncRecursion.Program.DependsOnPreviousTaskAsync(System.Threading.Tasks.Task<int> previousTask) Line 18 C# [Async Call] TestAsyncRecursion.exe!TestAsyncRecursion.Program.DoSomethingAsync(System.Threading.Tasks.Task<int> previousTask) Line 58 C# [Async Call] TestAsyncRecursion.exe!TestAsyncRecursion.Program.DependsOnPreviousTaskAsync(System.Threading.Tasks.Task<int> previousTask) Line 18 C# [Async Call] TestAsyncRecursion.exe!TestAsyncRecursion.Program.DoSomethingAsync and so on ... 

Here is an example call stack at a breakpoint when using DependsOnPreviousTaskAsync2 :

 TestAsyncRecursion.exe!TestAsyncRecursion.Program.DoSomethingAsync(System.Threading.Tasks.Task<int> previousTask) Line 62 C# [Resuming Async Method] mscorlib.dll!System.Runtime.CompilerServices.AsyncMethodBuilderCore.MoveNextRunner.InvokeMoveNext(object stateMachine) Unknown mscorlib.dll!System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state, bool preserveSyncCtx) Unknown mscorlib.dll!System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state, bool preserveSyncCtx) Unknown mscorlib.dll!System.Runtime.CompilerServices.AsyncMethodBuilderCore.MoveNextRunner.Run() Unknown mscorlib.dll!System.Runtime.CompilerServices.AsyncMethodBuilderCore.OutputAsyncCausalityEvents<System.Runtime.CompilerServices.AsyncTaskMethodBuilder<int>>.AnonymousMethod__0() Unknown mscorlib.dll!System.Runtime.CompilerServices.AsyncMethodBuilderCore.ContinuationWrapper.Invoke() Unknown mscorlib.dll!System.Runtime.CompilerServices.TaskAwaiter.OutputWaitEtwEvents.AnonymousMethod__0() Unknown mscorlib.dll!System.Runtime.CompilerServices.AsyncMethodBuilderCore.ContinuationWrapper.Invoke() Unknown mscorlib.dll!System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction(System.Action action, bool allowInlining, ref System.Threading.Tasks.Task currentTask) Unknown mscorlib.dll!System.Threading.Tasks.Task.FinishContinuations() Unknown mscorlib.dll!System.Threading.Tasks.Task.FinishStageThree() Unknown mscorlib.dll!System.Threading.Tasks.Task<System.Threading.Tasks.VoidTaskResult>.TrySetResult(System.Threading.Tasks.VoidTaskResult result) Unknown mscorlib.dll!System.Threading.Tasks.Task.WhenAllPromise.Invoke(System.Threading.Tasks.Task completedTask) Unknown mscorlib.dll!System.Threading.Tasks.Task.FinishContinuations() Unknown mscorlib.dll!System.Threading.Tasks.Task.FinishStageThree() Unknown mscorlib.dll!System.Threading.Tasks.Task<int>.TrySetResult(int result) Unknown mscorlib.dll!System.Threading.Tasks.TaskCompletionSource<int>.TrySetResult(int result) Unknown TestAsyncRecursion.exe!TestAsyncRecursion.Program.DependsOnPreviousTaskAsync2.AnonymousMethod__4(System.Threading.Tasks.Task<int> t) Line 44 C# mscorlib.dll!System.Threading.Tasks.ContinuationTaskFromResultTask<int>.InnerInvoke() Unknown mscorlib.dll!System.Threading.Tasks.Task.Execute() Unknown mscorlib.dll!System.Threading.Tasks.Task.ExecutionContextCallback(object obj) Unknown mscorlib.dll!System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state, bool preserveSyncCtx) Unknown mscorlib.dll!System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state, bool preserveSyncCtx) Unknown mscorlib.dll!System.Threading.Tasks.Task.ExecuteWithThreadLocal(ref System.Threading.Tasks.Task currentTaskSlot) Unknown mscorlib.dll!System.Threading.Tasks.Task.ExecuteEntry(bool bPreventDoubleExecution) Unknown mscorlib.dll!System.Threading.Tasks.Task.System.Threading.IThreadPoolWorkItem.ExecuteWorkItem() Unknown mscorlib.dll!System.Threading.ThreadPoolWorkQueue.Dispatch() Unknown mscorlib.dll!System.Threading._ThreadPoolWaitCallback.PerformWaitCallback() Unknown 

Answer 1

According to the comments, there is logic to throw a StackOverflowException when using .ContinueWith , however, as stated in Stephen Toub:

@Luke Horsley: Yes, the logic of checking and forcing asynchronous continuations, if that exists for ContinueWith, but not for waiting. You can add it (at the expense of some performance overhead), it just has not been done.

His justification for the reason that this was not done is as follows:

The main reason for this is that measuring performance affects the study (even with in-place optimization, to do it only after a certain depth), and the main use case for asynchronous methods is to implement asynchronous versions of synchronous methods, and in this case the deepest that you hit the stack is in the same order as in your synchronous call stack. This only happens when you start playing asynchronous tricks and use asynchronous data structures in which you run the risk of much deeper call stacks, and most of these situations are properly handled by the logic in the data structures used, rather than making everyone wait. This, combined with the fact that most codes not only have one of the problems, it led us to the fact that we did not enable stack checking for expectations. This does not mean that we could not or could not include it in the future (this is a rather trivial change).

One solution is to use

 await Task.Yield(); 

inside DependsOnPreviousTaskAsync before returning the result , this ensures that the continuations are performed asynchronously and the stack trace is actually reset.

Here is a simpler example that I developed to reproduce your problem:

 internal class Program { private static void Main() { bool useAsync = true; var tcs = new TaskCompletionSource<object>(); Task previous = tcs.Task; for (var i = 0; i < 100000; ++i) { previous = useAsync ? DoSomethingUsingAsync(previous) : DoSomethingUsingContinuation(previous); } tcs.SetResult(null); previous.Wait(); } private static Task DoSomethingUsingContinuation(Task previousTask) { return previousTask.ContinueWith(_ => { }, TaskContinuationOptions.ExecuteSynchronously); } private static async Task DoSomethingUsingAsync(Task previousTask) { await previousTask.ConfigureAwait(false); // Uncomment the next line to solve! // await Task.Yield(); } } 

This sample also throws a StackOverflowException , uncommenting await Task.Yield() , solves the problem.