Python memory leak?

Question

I am writing a python extension that seems to be causing a memory leak. I am trying to understand the problem problem using valgrind.

However, it seems that python itself has a memory leak according to valgrind. Using the following simple script:

hello.py

print "Hello World!" 

and making

 > valgrind --tool=memcheck python ./hello.py (...) ==7937== ERROR SUMMARY: 580 errors from 34 contexts (suppressed: 21 from 1) ==7937== malloc/free: in use at exit: 721,878 bytes in 190 blocks. ==7937== malloc/free: 2,436 allocs, 2,246 frees, 1,863,631 bytes allocated. ==7937== For counts of detected errors, rerun with: -v ==7937== Use --track-origins=yes to see where uninitialised values come from ==7937== searching for pointers to 190 not-freed blocks. ==7937== checked 965,952 bytes. ==7937== ==7937== LEAK SUMMARY: ==7937== definitely lost: 0 bytes in 0 blocks. ==7937== possibly lost: 4,612 bytes in 13 blocks. ==7937== still reachable: 717,266 bytes in 177 blocks. ==7937== suppressed: 0 bytes in 0 blocks. ==7937== Rerun with --leak-check=full to see details of leaked memory. 

Does anyone have an explanation for this behavior of suffering? Is python interpreter really a memory leak?

What tool do python developers use to debug memory leaks?

Answer 1

There is a whole README.valgrind in Python that explains the various caveats trying to use Valgrind with Python:

http://svn.python.org/projects/python/trunk/Misc/README.valgrind

 Python uses its own small-object allocation scheme on top of malloc, called PyMalloc. Valgrind may show some unexpected results when PyMalloc is used. Starting with Python 2.3, PyMalloc is used by default. You can disable PyMalloc when configuring python by adding the --without-pymalloc option. If you disable PyMalloc, most of the information in this document and the supplied suppressions file will not be useful. As discussed above, disabling PyMalloc can catch more problems. If you use valgrind on a default build of Python, you will see many errors like: ==6399== Use of uninitialised value of size 4 ==6399== at 0x4A9BDE7E: PyObject_Free (obmalloc.c:711) ==6399== by 0x4A9B8198: dictresize (dictobject.c:477) These are expected and not a problem. 

Answer 2

The leak is most likely coming from your own extension, not from Python. Large systems often come out with dedicated memory, simply because you should not explicitly free it if the process ends anyway.