Python: How to extend a huge class with minimal lines of code?

Question

Description of the original problem

The problem arises when I implement some machine learning algorithm with numpy . I need a new ludmo class that works the same as numpy.ndarray but has a few more properties. For example, with the new ludmo.foo property. I have tried several methods below, but none of them are satisfactory.

1. Packing

First I created a wrapper class for numpy.ndarray since

 import numpy as np class ludmo(object): def __init__(self) self.foo = None self.data = np.array([]) 

But when I use some function (in scikit-learn , which I cannot change) to manipulate the list of the np.ndarray instance, I must first extract all the data fields of each ludmo object and collect them into a list. After that, the list is sorted, and I lost the correspondence between the data and original ludmo .

2. Inheritance

Then I tried to make ludmo subclass of numpy.ndarray , since

 import numpy as np class ludmo(np.ndarray): def __init__(self, shape, dtype=float, buffer=None, offset=0, strides=None, order=None) super().__init__(shape, dtype, buffer, offset, strides, order) self.foo = None 

But another problem arises: the most common way to create a numpy.ndarray object is numpy.array(some_list) , which returns a numpy.ndarray object, and I have to convert it to a ludmo object. But so far I have not found a good way to do this; simply changing the __class__ attribute will result in an error.

I am new to Python and numpy, so there must be some elegant way that I don't know. Any advice is appreciated.

It is better if someone can give a general solution that applies not only to the numpy.ndarray class, but also to any classes.

Answer 1

Since you are asking for a general solution, here is a general wrapper class that you can use: (from http://code.activestate.com/recipes/577555-object-wrapper-class/ )

 class Wrapper(object): ''' Object wrapper class. This a wrapper for objects. It is initialiesed with the object to wrap and then proxies the unhandled getattribute methods to it. Other classes are to inherit from it. ''' def __init__(self, obj): ''' Wrapper constructor. @param obj: object to wrap ''' # wrap the object self._wrapped_obj = obj def __getattr__(self, attr): # see if this object has attr # NOTE do not use hasattr, it goes into # infinite recurrsion if attr in self.__dict__: # this object has it return getattr(self, attr) # proxy to the wrapped object return getattr(self._wrapped_obj, attr) 

how it works:

when, for example, skicit calls ludmo.data python actually calls ludmo.__getattribute__('data') if ludmo doesn't have the attribute 'data', python will call ludmo.__getattr__('data')

__getattr__ function that you intercept this call, check if your ludmo has a data attribute (again, you can go into recursion otherwise) and send the call to your internal object. So you should have covered all the possible calls for your numpy inner object.

Update : You will also have to implement __setattr__ in the same way, or you will get this

 >>> class bla(object): ... def __init__(self): ... self.a = 1 ... def foo(self): ... print self.a ... >>> d = Wrapper(bla()) >>> da 1 >>> d.foo() 1 >>> da = 2 >>> da 2 >>> d.foo() 1 

and you probably also want to set up a new metaclass that intercepts calls to magic functions of new style classes (for the full class see https://github.com/hpcugent/vsc-base/blob/master/lib/vsc/utils/wrapper .py for information see How to intercept calls to the “magic” python? methods in new-style classes? ) however this is only necessary if you still want to have access to x.__name__ or x.__file__ and get the magic attribute from wrapped class, not for your class.

 # create proxies for wrapped object double-underscore attributes class __metaclass__(type): def __init__(cls, name, bases, dct): def make_proxy(name): def proxy(self, *args): return getattr(self._obj, name) return proxy type.__init__(cls, name, bases, dct) if cls.__wraps__: ignore = set("__%s__" % n for n in cls.__ignore__.split()) for name in dir(cls.__wraps__): if name.startswith("__"): if name not in ignore and name not in dct: setattr(cls, name, property(make_proxy(name))) 

Answer 2

As described in the docs , you can add your own methods in np.ndarray :

 import numpy as np class Ludmo(np.ndarray): def sumcols(self): return self.sum(axis=1) def sumrows(self): return self.sum(axis=0) def randomize(self): self[:] = np.random.rand(*self.shape) 

and then instantiate using the np.ndarray.view() method :

 a = np.random.rand(4,5).view(Ludmo) 

And use the __array_finalize__() method to define new attributes:

 def __array_finalize__(self, arr): self.foo = 'foo'