How to use a dump file to diagnose a memory leak?

Question

I have a .NET service with a regular private working set of about 80 MB. During a recent load test, the process reached 3.5 GB of memory, which caused the entire machine to be low in physical memory (3.9 out of 4 GB), and the memory was not released long after the load test was stopped. Using the task manager, I took the process dump file and opened it in Visual Studio 2010 SP1, and I can start debugging it.

How do I diagnose a memory problem? I have dotTrace Memory 3.x at my disposal, does it support memory profiling in dump files? If not, will the memory profiling features of Visual Studio 2010 Premium help (currently I have Professional)? Can WinDbg Help?

UPDATE: The new version of Visual Studio 2013 Ultimate can now diagnose memory problems with dump files. See this blog post for more details.

Answer 1

Install WinDbg. You must make sure that you get the correct version of x86 or x64 depending on your dump. Here is a direct download link for x86.

In doing so, you need to make sure that you have the correct dump. You can use the task manager to create a dump file (right-click the process → Create dump file). If you are on a 64-bit version and your process is x86, use the 32-bit version of the task manager (C: \ Windows \ SysWOW64 \ taskmgr.exe) to take the dump file. See my article for more information on taking dump files, for example if you are in XP and you need to use windbg to create a dump file.

warning there is a pretty steep learning curve, and everything may not work as described here, so go back to any problems.

I assume that you are using .NET4, since you can open the dump in Visual Studio. Here is a very quick guide to help you work with your dmp file:

1) Run WinDbg, set the path to the characters (File → character search path) to

SRV*c:\symbols*http://msdl.microsoft.com/download/symbols 

2) Open Reset or drag the .DMP file to WinDbg.

3) enter this in the command window

 .loadby sos clr 

(FYI, for .NET 2 the command should be .loadby sos mscorwks )

4) enter this

 !dumpheap -stat 

which lists the types of objects and their number. It looks something like this:

You will have to analyze this in the context of your application and see if something seems unusual.

There is much more to windbg, google is your friend.

Answer 2

Typically, if you have a leak in a managed application, this means that something is not going to. Common sources include

• Event handlers. If the subscriber is not deleted, the publisher will continue it.

• Statics

• Finalizers: A locked finalizer will prevent the finalizer thread from starting up with any other finalizers, and thus prevent the collection of these instances.

• Similarly, a deadlock thread will be supported by any roots that it executes. Of course, if you have deadlock threads that are likely to affect the application at several levels.

To fix this problem, you need to check the managed heap. WinDbg + SOS (or PSSCOR) will allow you to do this. The !dumpheap -stat displays the entire managed heap.

You need to have an idea of ​​the number of instances of each type expected on the heap. When you find something strange, you can use the command !dumpheap -mt <METHOD TABLE> to display all instances of a given type.

The next step is to analyze the root of these instances. Choose one randomly and do !gcroot . This will show how this particular instance is being implemented. Look at event handlers and pinned objects (usually static links). If you see the finalizer queue, you need to learn what the finalizer stream does. To do this, use the !clrstack and !clrstack .

If everything looks fine for this instance, you move on to another instance. If this does not work, you may need to come back to look at the pile again and repeat from there.

Other sources of leaks include: assemblies that are not unloaded and fragmentation of the heap of a large object. SOS / PSSCOR can help you find them, but for now I will not list the details.

If you want to know more, I recommend the Tess blog . I also made some videos about using WinDbg + SOS ( here and here ).

If you have the ability to debug a process while it is starting, I recommend PSSCOR instead. PSSCOR is essentially a private SOS source branch that has been complemented by additional commands, and many of the existing SOS commands have also been improved. For example. The PSSCOR version of the !dumpheap has a very useful delta column, which greatly simplifies troubleshooting for memory leaks.

To use it, you need to start your process, connect WinDbg and download PSSCOR and execute !dumpheap -stat . Then you start the process again so that the distributions are made. Break the execution and reissue the command. Now PSSCOR will show you the number of instances that have been added / removed since the previous inspection.

Answer 3

Starting with version 2017.2, JetBrains dotMemory supports analysis of Windows memory dumps with all the features and a graphical interface.

Answer 4

http://msdn.microsoft.com/en-us/library/ee817660.aspx

Microsoft has a guide. However, this is too complicated for beginners.

dotTrace can generate visual memory graphics (better than WinDbg), but never use them for dumps.