How python handles an instance of an object in a 'for' loop

Question

I have a very complex class:

class C: pass 

And I have this test code:

 for j in range(10): c = C() print c 

What gives:

 <__main__.C instance at 0x7f7336a6cb00> <__main__.C instance at 0x7f7336a6cab8> <__main__.C instance at 0x7f7336a6cb00> <__main__.C instance at 0x7f7336a6cab8> <__main__.C instance at 0x7f7336a6cb00> <__main__.C instance at 0x7f7336a6cab8> <__main__.C instance at 0x7f7336a6cb00> <__main__.C instance at 0x7f7336a6cab8> <__main__.C instance at 0x7f7336a6cb00> <__main__.C instance at 0x7f7336a6cab8> 

It is easy to see that Python includes two different values. In some cases, this can be catastrophic (for example, if we store objects in some other complex object).

Now, if I store the objects in a list:

 lst = [] for j in range(10): c = C() lst.append(c) print c 

I get this:

 <__main__.C instance at 0x7fd8f8f7eb00> <__main__.C instance at 0x7fd8f8f7eab8> <__main__.C instance at 0x7fd8f8f7eb48> <__main__.C instance at 0x7fd8f8f7eb90> <__main__.C instance at 0x7fd8f8f7ebd8> <__main__.C instance at 0x7fd8f8f7ec20> <__main__.C instance at 0x7fd8f8f7ec68> <__main__.C instance at 0x7fd8f8f7ecb0> <__main__.C instance at 0x7fd8f8f7ecf8> <__main__.C instance at 0x7fd8f8f7ed40> 

What solves this case.

So now I have to ask a question ... Can someone explain with complex words (I mean, deeply) how Python behaves with references to objects? I suppose this is an optimization issue (to free up memory or prevent leaks, ...)

Many thanks.

EDIT: Well, let me be more specific. I understand perfectly that python sometimes collects garbage ... But in my case:

I had a list returned by a specific Cython: class 'Network' class that manages a Node list (both Network and Node are defined in the Cython extension ). Each Node has an object [then it is placed in an object (void *)] 'userdata'. The Nodes list is populated from within cython, and the UserData is populated inside the Python script. So in python, I had the following:

 ... def some_python_class_method(self): nodes = self.netBinding.GetNetwork().get_nodes() ... for item in it: a_site = PyLabSiteEvent() #l_site.append(a_site) # WARN : Required to get an instance on 'a_site' # that persits - workaround... item.SetUserData(a_site) 

Reusing this Node list in the same python class using the same cython recipient:

 def some_other_python_class_method(self, node): s_data = node.GetUserData() ... 

So it looks like with the repository made in the Node UserDatas user list, my python script was completely blind and freed / reused memory. It worked by referencing a second time (but apparently the first to the python side) using an optional list (here: 'l_site'). That's why I should have learned a little more about Python itself, but it seems that the way I implemented the connection between Python and Cython is responsible for the problems I had to deal with.

Answer 1

There is no need to be “complicated”: in the first example, you do not specify another link to the object referenced by the name "c" - when you run the code in the line "c = C ()" at subsequent iterations of the loop, one link previously contained in "c" is lost.

Since standard Python uses reference counting to track when it should remove objects from memory, since at that moment the reference count for the object of the previous cycle interval reaches 0, it is destroyed, and its memory becomes available for other objects.

Why do you have 2 mutable values? Since the object is currently being created in a new iteration, i.e. When Python executes the expression on the right-hand side of = in c = C() , the object of the previous iteration still exists, which is referenced by the name c - so a new object is built in a different place in memory. Python then moves on to assigning a new object c , after which the previous object is destroyed, as described above, which means that at the next (third) iteration, this memory will be available for a new instance of c .

In the second example, newly created objects never lose a link, therefore their memory is not freed at all - new objects always occupy a new memory cell.

Most important: The goal of using a high-level language like Python or others is not worried about memory allocation. Language takes care of this for you. In this case, the CPython implementation (standard) does exactly what you noticed. Other implementations, such as Pypy or Jython, may have completely different “memory location” behavior for each instance in the examples above, but all relevant implementations (including these 3) will behave exactly the same from the “point of view” Python program : (1) She has access to the instances to which she refers, (2) the data of these instances is in any case not damaged or distorted.

Answer 2

It does not seem complicated.

In the first example, the second time through the loop, the memory at 0x7f7336a6cb00 is occupied by the C instance created during the first iteration. Therefore, Python allocates the following memory block 0x7f7336a6cab8 for the new C object.

However, once you create a second C object and assign it c , references to the axes remaining now will not be left at 0x7f7336a6cb00. Therefore, for the third time through the Python loop, you can reuse memory in this place for a new object. As soon as this happens, of course, the object in 0x7f7336a6cab8 no longer refers to it, and this memory location becomes available for reuse for the fourth time through the loop.

In your second example, however, by adding an object to the list, you keep a link to each object you create. Since these objects always have at least one reference to them, the memory in which they "live" can never be freed and recycled. Therefore, Python allocates new memory every time.

The illusion of danger created in the first example is just an illusion. As long as there is a reference to the object you are creating, the object will be saved. When references do not exist, it is safe for Python to free the memory used by the object, since your program can no longer use the object.

Answer 3

During each cycle, you change the object c , so the original is not available, and Python can freely get rid of it (since what is any object for if you can never get it again?). At this point, he has free memory in the same place and seems to be reusing it. I'm not sure what is surprising. If the interpreter never used memory again, you would be done quickly.

This does not happen when you add an object to the list because the object is still available, so Python cannot get rid of it (since you can use it again).

This should never cause problems, because Python will not get rid of the object while you can still use it, so if you “store objects in some complex object”, they will remain available and their memory won't be reused (by at least until the object leaves).