Memory usage @on_trait_change vs _foo_changed ()

I created an application with Enthought Traits that uses too much memory. I think the problem is caused by value notifications:

There seems to be a fundamental difference in memory usage for events caught by @on_trait_change, or through a special naming convention (e.g. _foo_changed ()). I made a small example with two classes Foo and FooDecorator, which, as I assumed, show exactly the same behavior. But they do not!

from traits.api import * class Foo(HasTraits): a = List(Int) def _a_changed(self): pass def _a_items_changed(self): pass class FooDecorator(HasTraits): a = List(Int) @on_trait_change('a[]') def bar(self): pass if __name__ == '__main__': n = 100000 c = FooDecorator a = [c() for i in range(n)] 

When running this script with c = Foo, the Windows task manager shows memory usage for the entire python 70MB process, which remains constant to increase n. For c = FooDecorator, the python process uses 450 MB, increasing for higher n.

Could you explain me this behavior?

EDIT: Maybe I should rephrase: why would anyone choose FooDecorator over Foo?

EDIT 2: I just uninstalled python (x, y) 2.7.9 and installed the latest version of the dome with features 4.5.0. Now 450 MB have become 750 MB.

EDIT 3: Compiled traits-4.6.0.dev0-py2.7-win-amd64. The result is the same as in EDIT 2. Therefore, despite all the plausibility of https://github.com/enthought/traits/pull/248/files , it does not seem to be the reason.

+4
source share
2 answers

What happens here is that Traits has two different ways to handle notifications: static notifications and dynamic notifications.

Static notifiers (for example, created by specially selected methods _*_changed() ) are quite easy: each instance line has a list of notifications about t, which are basically functions or lightweight shell methods.

Dynamic notifiers (for example, those created with on_trait_change() and extended name conventions , such as a[] , are much more powerful and flexible, but as a result they are much heavier. In particular, in addition to the shell object they create, they also create an analysis view the extended name of the attribute and the handler object, some of which are in the HasTraits queue HasTraits instances of the subclass.

As a result, even for a simple expression like a[] , many new Python objects will be created, and these objects must be created for each on_trait_change listener for each instance separately in order to properly handle corner elements, such as instance features. The relevant code is here: https://github.com/enthought/traits/blob/master/traits/has_traits.py#L2330

Based on the reported numbers, most of the differences in memory usage that you see is to create this dynamic listener infrastructure for each instance and each on_trait_change decorator.

It is worth noting that there is a short circuit for on_trait_change in the case when you use a simple attribute name, in which case it generates a static attribute attribute instead of a dynamic notifier. Therefore, if you should write something like:

 class FooSimpleDecorator(HasTraits): a = List(Int) @on_trait_change('a') def a_updated(self): pass @on_trait_change('a_items') def a_items_updated(self): pass 

you should see similar memory performance for specially named methods.

To answer the rephrased question "why use on_trait_change ", in FooDecorator you can write one method instead of two if your answer to changing the list or any items in the list is the same. This greatly simplifies the debugging and maintenance of the code, and if you do not create thousands of these objects, the additional memory usage is negligible.

This becomes an even more important factor when you consider more complex advanced feature name templates in which dynamic listeners automatically process changes that would otherwise require significant manual (and error prone) code to connect and remove listeners from intermediate objects and features. The strength and simplicity of this approach usually outweighs concerns about memory usage.

+3
source

I believe that you see a memory leak effect that has been fixed recently: https://github.com/enthought/traits/pull/248/files

As for the use of the decorator, in this particular case the two versions are almost equivalent.

In general, the decorator is more flexible: you can specify a list of attributes to listen to, and you can use the extension name designation, as described here: http://docs.enthought.com/traits/traits_user_manual/notification.html#semantics

For example, in this case:

 class Bar(HasTraits): b = Str class FooDecorator(HasTraits): a = List(Bar) @on_trait_change('a.b') def bar(self): print 'change' 

bar notifier will be called to change characteristic a , its elements and to change characteristic b in each of bar elements. Extended names can be quite powerful.

+5
source

All Articles