Python has a stack / heap and how is memory managed?

Question

How are variables and memory managed in Python? Does it have a stack and a heap and what algorithm is used to manage memory? Given this knowledge, are there any memory management recommendations for large amounts / data crunches?

Answer 1

How variables and memory are managed in Python.

automagically! No, really, you just create an object, and the Python virtual machine processes the necessary memory and where it should be placed in the memory layout.

Does it have a stack and a bunch, and what algorithm is used to manage memory?

When we talk about CPython , it uses a private heap to store objects. From the official Python documentation :

Python memory management includes a private heap containing all Python Objects and data structures. This private heap is managed by the Python internal memory manager. The Python memory manager has various components that have dynamic aspects of storage management, such as sharing, segmentation, preallocation or caching.

The algorithm used to collect garbage is called Link Counting . This Python VM stores an internal log of how many references the object references, and automatically garbage collects it when no more references refer to it.

NOTE. . Keep in mind that this information is CPython . Other python implementations, such as pypy , iron python , jython and others, may differ from each other and from CPython when it comes to their implementation features. To understand which is better, this can help to understand that there is a difference between the semantics of Python (the language) and the underlying implementation

Given this knowledge, are there any memory management recommendations for large amounts / crunches of data?

Now I can’t talk about it, but I’m sure that NumPy (the most popular python library for number crunching) has mechanisms that correctly handle memory consumption.

If you want to learn more about Python Internals, check out these resources:

• Going through CPython (video)
• Presentation about the internal components of a Python virtual machine
• True Hacker-style Python Object Allocator Source Code

Answer 2

Python does not have any such thing.

Python is a language and does not indicate exactly how implementations should achieve the semantics defined by the Python language.

Each implementation (CPython, PyPy, IronPython, Stackless, Jython ...) is free to do its job!

In C Python, all objects are on the heap:

Python memory management includes a private heap containing all Python objects and data structures. ^one

The CPython virtual machine is stack-based:

 >>> def g(): x = 1 y = 2 return f(x, y) >>> import dis >>> dis.dis(g) 2 0 LOAD_CONST 1 (1) # Push 1 onto the stack 3 STORE_FAST 0 (x) # Stores top of stack into local var x 3 6 LOAD_CONST 2 (2) # Push 2 onto stack 9 STORE_FAST 1 (y) # Store TOS into local var y 4 12 LOAD_GLOBAL 0 (f) # Push f onto stack 15 LOAD_FAST 0 (x) # Push x onto stack 18 LOAD_FAST 1 (y) # Push y onto stack 21 CALL_FUNCTION 2 # Execute function with 2 # f return value is pushed on stack 24 RETURN_VALUE # Return TOS to caller (result of f) 

Keep in mind that this is specific to CPython. The stack does not contain actual values, but retains references to these objects.

¹ : Source