Print all available tuples in python from debugger

Question

I realized that there is a memory leak in one python script. At first it took about 25 MB, and after 15 days - more than 500 MB.

I followed in many different ways and could not get to the root of the problem, like a python rookie ...

Finally, I got the following

objgraph.show_most_common_types(limit=20) tuple 37674 function 9156 dict 3935 list 1646 wrapper_descriptor 1468 weakref 888 builtin_function_or_method 874 classobj 684 method_descriptor 551 type 533 instance 483 Kind 470 getset_descriptor 404 ImmNodeSet 362 module 342 IdentitySetMulti 333 PartRow 331 member_descriptor 264 cell 185 FontEntry 170 

I set a breakpoint, and after each iteration this happens ...

 objgraph.show_growth() tuple 37674 +10 

What is the best way to continue?

 (Pdb) c (Pdb) objgraph.show_growth() tuple 37684 +10 

I assume I am printing all the tuples, and cross-checking - that every 10 tuples are added every time, will give me a hint? Please let me know how to do this.

Or is there any other way to find out about this memory leak. I am using python 2.4.3 and due to many other product dependencies. Unfortunately, I cannot / should not be updated.

Answer 1

Am I reading correctly that the same script runs for 15 days without stops?

Periodic restarting is good practice for such lengthy processes, and it is much easier than fixing all memory leaks.

Update: look at this answer , it seems to do exactly what you need - print all newly added objects that were not garbage collected.

Answer 2

My first thought: perhaps you create new script objects in yourself and accumulate them in some kind of global list. It is generally easier to skip your script and make sure you are not generating persistent data other than debugging garbage. I think that the utility you use, objgraph, also allows you to print a garbage object with a number of references to it. You could try this.