.NET TPL CancellationToken memory leak

I developed a library that implements a producer / consumer pattern for work items. The work is canceled, and a separate task with the continuation of failure and success is created for each processed work object.

Continuing tasks reloads the work item after its completion (or failure) of its operation.

The entire library has one central CancellationTokenSource , which is launched when the application terminates.

Now I am facing a serious memory leak. If tasks are created using the undo marker as a parameter, then the tasks seem to remain in memory until the undo source is started (and later deleted).

This can be reproduced in this code sample (VB.NET). The main task is the task that will wrap the work item, and the continuation tasks will handle the rescheduling.

 Dim oCancellationTokenSource As New CancellationTokenSource Dim oToken As CancellationToken = oCancellationTokenSource.Token Dim nActiveTasks As Integer = 0 Dim lBaseMemory As Long = GC.GetTotalMemory(True) For iteration = 0 To 100 ' do this 101 times to see how much the memory increases Dim lMemory As Long = GC.GetTotalMemory(True) Console.WriteLine("Memory at iteration start: " & lMemory.ToString("N0")) Console.WriteLine(" to baseline: " & (lMemory - lBaseMemory).ToString("N0")) For i As Integer = 0 To 1000 ' 1001 iterations to get an immediate, measurable impact Interlocked.Increment(nActiveTasks) Dim outer As Integer = i Dim oMainTask As New Task(Sub() ' perform some work Interlocked.Decrement(nActiveTasks) End Sub, oToken) Dim inner As Integer = 1 Dim oFaulted As Task = oMainTask.ContinueWith(Sub() Console.WriteLine("Failed " & outer & "." & inner) ' if failed, do something with the work and re-queue it, if possible ' (imagine code for re-queueing - essentially just a synchronized list.add) ' Does not help: ' oMainTask.Dispose() End Sub, oToken, TaskContinuationOptions.OnlyOnFaulted, TaskScheduler.Default) ' if not using token, does not cause increase in memory: 'End Sub, TaskContinuationOptions.OnlyOnFaulted) ' Does not help: ' oFaulted.ContinueWith(Sub() ' oFaulted.Dispose() ' End Sub, TaskContinuationOptions.NotOnFaulted) Dim oSucceeded As Task = oMainTask.ContinueWith(Sub() ' success ' re-queue for next iteration ' (imagine code for re-queueing - essentially just a synchronized list.add) ' Does not help: ' oMainTask.Dispose() End Sub, oToken, TaskContinuationOptions.OnlyOnRanToCompletion, TaskScheduler.Default) ' if not using token, does not cause increase in memory: 'End Sub, TaskContinuationOptions.OnlyOnRanToCompletion) ' Does not help: ' oSucceeded.ContinueWith(Sub() ' oSucceeded.Dispose() ' End Sub, TaskContinuationOptions.NotOnFaulted) ' This does not help either and makes processing much slower due to the thrown exception (at least one of these tasks is cancelled) 'Dim oDisposeTask As New Task(Sub() ' Try ' Task.WaitAll({oMainTask, oFaulted, oSucceeded, oFaultedFaulted, oSuccededFaulted}) ' Catch ex As Exception ' End Try ' oMainTask.Dispose() ' oFaulted.Dispose() ' oSucceeded.Dispose() ' End Sub) oMainTask.Start() ' oDisposeTask.Start() Next Console.WriteLine("Memory after creating tasks: " & GC.GetTotalMemory(True).ToString("N0")) ' Wait until all main tasks are finished (may not mean that continuations finished) Dim previousActive As Integer = nActiveTasks While nActiveTasks > 0 If previousActive <> nActiveTasks Then Console.WriteLine("Active: " & nActiveTasks) Thread.Sleep(500) previousActive = nActiveTasks End If End While Console.WriteLine("Memory after tasks finished: " & GC.GetTotalMemory(True).ToString("N0")) Next 

I measured memory usage with ANTI Memory Profiler and saw a significant increase in System.Threading.ExecutionContext, which returns to the continuation of the task and CancellationCallbackInfo .

As you can see, I already tried to get rid of tasks that use the cancellation token, but this does not seem to have any effect.

Edit

I am using .NET 4.0

Update

Even with a simple connection of the main task with continued failure, the memory usage is constantly increasing. The continuation of the task, apparently, hinders the deregistration of the undo marker.

So, if a task is chained with a continuation that fails (due to TaskContinuationOptions ), then there seems to be a memory leak. If there is only one continuation that is running, then I did not notice a memory leak.

Bypass

As a workaround, I can make one continuation without TaskContinuationOptions and handle the state of the parent task:

 oMainTask.ContinueWith(Sub(t) If t.IsCanceled Then ' ignore ElseIf t.IsCompleted Then ' reschedule ElseIf t.IsFaulted Then ' error handling End If End Sub) 

I will need to check how this happens in case of cancellation, but this seems to do the trick. I almost suspect a bug in the .NET Framework. Canceling jobs with mutually exclusive terms is not something that can be so rare.

+7
memory-leaks task-parallel-library
source share
2 answers

Some observations

  • A potential leak seems only present when there is a β€œbranch” of a task that does not start. In your example, if you comment out the oFaulted task, the leak will go away for me. If you update the code to have an oMainTask error so that the oFaulted task starts and the oSucceeded task oSucceeded not start, then commenting on oSucceeded prevents a leak.
  • This may not be useful, but if you call oCancellationTokenSource.Cancel() after all the tasks have been completed, memory is freed. Dispose does not help, and no combination of Disposing cancellation sources along with tasks.
  • I took a look at http://referencesource.microsoft.com/ , which is 4.5.2 (is there a way to view the early frames?) I know that this is not necessarily the same, but it is useful to know what things are happening. Basically, when you pass a cancellation token to a task, the task is registered using the token cancel cancel source. Thus, the cancellation source contains links to all your tasks. I still don't understand why your script seems to be leaking. I will be updated after I have the opportunity to look in more detail if I find anything.

Bypass

Move the branch logic to a continuation that always runs.

 Dim continuation As Task = oMainTask.ContinueWith( Sub(antecendent) If antecendent.Status = TaskStatus.Faulted Then 'Handle errors ElseIf antecendent.Status = TaskStatus.RanToCompletion Then 'Do something else End If End Sub, oToken, TaskContinuationOptions.None, TaskScheduler.Default) 

There is a good chance that this is easier than another approach. In both cases, one continuation is always performed, but with this code only one continuation task is created instead of 2.

+4
source share

I was able to fix the problem in .net 4.0 by moving these 2 lines

 Dim oCancellationTokenSource As New CancellationTokenSource Dim oToken As CancellationToken = oCancellationTokenSource.Token 

inside the first cycle

then at the end of this cycle

 oToken = Nothing oCancellationTokenSource.Dispose() 

also i moved

 Interlocked.Decrement(nActiveTasks) 

inside each "final" task, since

 While nActiveTasks > 0 

will not be exact.

here is the code that works

 Imports System.Threading.Tasks Imports System.Threading Module Module1 Sub Main() Dim nActiveTasks As Integer = 0 Dim lBaseMemory As Long = GC.GetTotalMemory(True) For iteration = 0 To 100 ' do this 101 times to see how much the memory increases Dim oCancellationTokenSource As New CancellationTokenSource Dim oToken As CancellationToken = oCancellationTokenSource.Token Dim lMemory As Long = GC.GetTotalMemory(True) Console.WriteLine("Memory at iteration start: " & lMemory.ToString("N0")) Console.WriteLine(" to baseline: " & (lMemory - lBaseMemory).ToString("N0")) For i As Integer = 0 To 1000 ' 1001 iterations to get an immediate, measurable impact Dim outer As Integer = iteration Dim inner As Integer = i Interlocked.Increment(nActiveTasks) Dim oMainTask As New Task(Sub() ' perform some work End Sub, oToken, TaskCreationOptions.None) oMainTask.ContinueWith(Sub() Console.WriteLine("Failed " & outer & "." & inner) Interlocked.Decrement(nActiveTasks) End Sub, oToken, TaskContinuationOptions.OnlyOnFaulted, TaskScheduler.Default) oMainTask.ContinueWith(Sub() If inner Mod 250 = 0 Then Console.WriteLine("Success " & outer & "." & inner) Interlocked.Decrement(nActiveTasks) End Sub, oToken, TaskContinuationOptions.OnlyOnRanToCompletion, TaskScheduler.Default) oMainTask.Start() Next Console.WriteLine("Memory after creating tasks: " & GC.GetTotalMemory(True).ToString("N0")) Dim previousActive As Integer = nActiveTasks While nActiveTasks > 0 If previousActive <> nActiveTasks Then Console.WriteLine("Active: " & nActiveTasks) Thread.Sleep(500) previousActive = nActiveTasks End If End While oToken = Nothing oCancellationTokenSource.Dispose() Console.WriteLine("Memory after tasks finished: " & GC.GetTotalMemory(True).ToString("N0")) Next Console.WriteLine("Final Memory after finished: " & GC.GetTotalMemory(True).ToString("N0")) Console.Read() End Sub End Module 
0
source share

All Articles