Deep version of sys.getsizeof

Question

I want to calculate the memory used by an object. sys.getsizeof fine, but not shallow (for example, it is called in a list, it will not contain memory occupied by list items).

I would like to write a general “deep” version of sys.getsizeof . I understand that there is some ambiguity in the definition of "deep"; I am completely satisfied with the definition and then copy.deepcopy .

Here is my first attempt:

 def get_deep_sizeof(x, level=0, processed=None): if processed is None: # we're here only if this function is called by client code, not recursively processed = set() processed.add(id(x)) mem = sys.getsizeof(x) if isinstance(x, collections.Iterable) and not isinstance(x, str): for xx in x: if id(xx) in processed: continue mem += get_deep_sizeof(xx, level+1, processed) if isinstance(x, dict): mem += get_deep_sizeof(x[xx], level+1, processed) return mem 

He suffers from two known problems and an unknown series of unknown problems:

• I do not know how to cross a common container in such a way as to capture all related objects. So I repeated using in and hard-coded the dictionary case (to include values, not just keys). Obviously this will not work for other classes such as a dictionary.
• I had to hard code the str exception (which is iterable and has no references to any other objects). Again, this will break if there are more such objects.

I suspect using in is not a good idea, but I'm not sure what else to do.

Answer 1

I think Pympler has already beaten you to hit on this.

From their documentation:

 >>> from pympler.asizeof import asizeof >>> obj = [1, 2, (3, 4), 'text'] >>> asizeof(obj) 176 

If you need a specific example, you can find the source for asizeof here .