Strange numpy array indexing behavior

Question

I noticed some confusing behavior when indexing a numpy flat array with a list of tuples (using python 2.7.8 and numpy 1.9.1). I assume this is due to the maximum number of dimensions of the array (I believe it is 32), but I could not find the documentation.

>>> a = np.arange(100) >>> tuple_index = [(i,) for i in a] >>> a[tuple_index] # This works (but maybe it shouldn't) >>> a[tuple_index[:32]] # This works too >>> a[tuple_index[:31]] # This breaks for 2 <= i < 32 Traceback (most recent call last): File "<stdin>", line 1, in <module> IndexError: too many indices for array >>> a[tuple_index[:1]] # This also works... 

Is a tuple list "flattened" if it has 32 elements or more? Is it documented anywhere?

Answer 1

The difference is that the first examples trigger fancy indexing (which simply selects indexes on the list from the same size), while tuple_index[:31] instead treated as an indexing tuple (which implies choosing from multiple axes).

As you noted, the maximum number of dimensions for a NumPy array is (usually) 32:

 >>> np.MAXDIMS 32 

According to the following comment in the mapping.c file (which contains code for interpreting the index passed by the user), any sequence of tuples shorter than 32 is smoothed out with an index tuple:

 /* * Sequences < NPY_MAXDIMS with any slice objects * or newaxis, Ellipsis or other arrays or sequences * embedded, are considered equivalent to an indexing * tuple. (`a[[[1,2], [3,4]]] == a[[1,2], [3,4]]`) */ 

(I have not yet found a link for this in the official documentation on the SciPy website.)

This makes a[tuple_index[:3]] equivalent to a[(0,), (1,), (2,)] , so the error is “too many indexes” (because a has only one dimension, but we mean that there are three of them).

On the other hand, a[tuple_index] matches a[[(0,), (1,), (2,), ..., (99,)]] , which leads to a 2D array.