Logical indices in numpy throw exception

Question

I'm trying to write code that uses numpy logical arrays to index a larger array, similar to how MATLAB allows indexing arrays using logical arrays.

import numpy as np m = 4 n = 4 unCov = np.random.randint(10, size = (m,n) ) rowCov = np.zeros( m, dtype = bool ) colCov = np.ones( n, dtype = bool ) >>> unCov[rowCov, rowCov] [] # as expected >>> unCov[colCov, colCov] [0 8 3 3] # diagonal values of unCov, as expected >>> unCov[rowCov, colCov] ValueError: shape mismatch: objects cannot be broadcast to a single shape 

For this final evaluation, I was expecting an empty array, similar to what MATLAB returns. I would prefer not to check rowCov / colCov for True elements before indexing. Why is this happening, and is there a better way to do this?

Answer 1

As I understand it, numpy will translate your 2d logical indexes into actual index vectors: arr[[True,False],[False,True]] will become arr[0,1] for the ndarray form (2,2) . However, in your last case, the second index array is filled with False , so it corresponds to an array of indexes of length 0. This is associated with another full index index True , corresponding to an array of indexes of length 4.

From the numpy manual :

If the index arrays do not have the same shape, there is an attempt to translate them into the same form. If they cannot be submitted in the same form, an exception is thrown:

In your case, the error is precisely because of this:

 --------------------------------------------------------------------------- IndexError Traceback (most recent call last) <ipython-input-1411-28e41e233472> in <module>() ----> 1 unCov[colCov,rowCov] IndexError: shape mismatch: indexing arrays could not be broadcast together with shapes (4,) (0,) 

MATLAB, on the other hand, automatically returns an empty array if the index array is empty at any given size.

---

This actually underlines the fundamental difference between logical indexing in MATLAB and numpy. In MATLAB, index index vectors always cut subarrays. That is, both

 arr([1,2],[1,2]) 

and

 arr([true,true],[true,true]) 

will return the submatrix 2 x 2 matrix arr . If the logical index vectors are shorter than the specified array size, the missing False index elements are considered. An interesting fact: the index vector can also be longer than a given size, if all the extra elements are all False . Thus, the above is also equivalent

 arr([true,true,false,false],[true,true]) 

and

 arr([true,true,false,false,false,false,false],[true,true]) 

for a 4 x 4 array (for an argument).

However, in numpy, indexing with a boolean value numpy arrays thus try to extract the vector. In addition, logical index vectors must be the same length as the dimension into which they are indexed. In the 4 x 4 example

 unCov[np.array([True,True]),np.array([True,True])] 

and

 unCov[np.array([True,True,False,False,False]),np.array([True,True,False,False,False])] 

both return the first two diagonal elements, therefore not a submatrix, but rather a vector. In addition, they also give a less favorable warning in accordance with

 /usr/bin/ipython:1: VisibleDeprecationWarning: boolean index did not match indexed array along dimension 0; dimension is 4 but corresponding boolean dimension is 5 

So, in numpy, your logical index vectors should be the same length as the corresponding ndarray sizes. And then what I wrote above is true: logical values ​​are translated into indices, and the result is expected to be a vector. The length of this vector is the number of True elements in each index vector, so if your logical index vectors have a different number of True elements, then the link does not make sense, and you get the error you get.