Array operator and __rmul__ in Python Numpy

Question

In the project, I created a class, and I needed an operation between this new class and the real matrix, so I overloaded an __rmul__ function like this

 class foo(object): aarg = 0 def __init__(self): self.aarg = 1 def __rmul__(self,A): print(A) return 0 def __mul__(self,A): print(A) return 0 

but when I called it, the result was not what I expected

 A = [[i*j for i in np.arange(2) ] for j in np.arange(3)] A = np.array(A) R = foo() C = A * R 

Output:

 0 0 0 1 0 2 

It seems that the function is called 6 times, once for each element.

Instead, the __mul__ function works significantly

 C = R * A 

Output:

 [[0 0] [0 1] [0 2]] 

If A not an array, but just a list of lists, both work fine

 A = [[i*j for i in np.arange(2) ] for j in np.arange(3)] R = foo() C = A * R C = R * A 

Exit

 [[0, 0], [0, 1], [0, 2]] [[0, 0], [0, 1], [0, 2]] 

I would really like my __rmul__ function __rmul__ work with arrays as well (my original multiplication function is not commutative). How can I solve it?

Answer 1

Behavior is expected.

First of all, you need to understand how an operation such as x*y is actually performed. The python interpreter will first try to compute x.__mul__(y) . If this call returns NotImplemented it will try to compute y.__rmul__(x) . y.__rmul__(x) y is a suitable subclass of type x , in this case the interpreter will first consider y.__rmul__(x) and then x.__mul__(y) .

Now it happens that numpy processes the arguments differently, depending on whether it considers the argument scalar or massive.

When working with arrays * performs element-wise multiplication, while scalar multiplication multiplies the entire array record by a given scalar.

In your case, foo() is considered a numpy scalar, and thus numpy multiplies all elements of the array by foo . Moreover, since numpy does not know about the type foo it returns an array with dtype=object , so the returned object:

 array([[0, 0], [0, 0], [0, 0]], dtype=object) 

Note: the numpy array does not return NotImplemented when trying to calculate the product, so the interpreter calls the numpy array __mul__ , which, as we said, performs scalar multiplication. At this point, numpy will try to multiply each array entry by your “scalar” foo() , and that’s where __rmul__ your __rmul__ method, because the numbers in the array return NotImplemented when their __mul__ is called with the foo argument.

Obviously, if you change the order of the arguments to the initial multiplication, your __mul__ method will be called immediately and you will not have __mul__ any problems.

So, to answer your question, one way to handle this is to use foo inherit from ndarray , so the second rule applies:

 class foo(np.ndarray): def __new__(cls): # you must implement __new__ # code as before 

However, a warning that subclassing ndarray not simple . Moreover, you may have other side effects since your class is now ndarray .

Answer 2

I could not explain the main problem as accurately as Bakuriu, but there may be another solution.

You can force numpy to use your evaluation method by specifying __array_priority__ . As described here in numpy docs.

In your case, you had to change the class definition:

 MAGIC_NUMBER = 15.0 # for the necessary lowest values of MAGIC_NUMBER look into the numpy docs class foo(object): __array_priority__ = MAGIC_NUMBER aarg = 0 def __init__(self): self.aarg = 1 def __rmul__(self,A): print(A) return 0 def __mul__(self,A): print(A) return 0 

Answer 3

You can define the __numpy_ufunc__ function in your class. This works even without subclassing np.ndarray . You can find the documentation here .

Here is an example based on your case:

 class foo(object): aarg = 0 def __init__(self): self.aarg = 1 def __numpy_ufunc__(self, *args): pass def __rmul__(self,A): print(A) return 0 def __mul__(self,A): print(A) return 0 

And if we try this,

 A = [[i*j for i in np.arange(2) ] for j in np.arange(3)] A = np.array(A) R = foo() C = A * R 

Exit:

 [[0 0] [0 1] [0 2]] 

It works!