Numpy reformatting a multidimensional array through an arbitrary axis

Question

so this is a question about using reshape and how these functions use each axis on a multidimensional scale .

Suppose I have the following array containing matrices indexed by the first index. What I want to achieve is instead index the columns of each matrix with the first index . To illustrate this problem, consider the following example in which a given numpy array that indexes matrices with its first index is z.

x = np.arange(9).reshape((3, 3)) y = np.arange(9, 18).reshape((3, 3)) z = np.dstack((x, y)).T 

Where z looks like this:

 array([[[ 0, 3, 6], [ 1, 4, 7], [ 2, 5, 8]], [[ 9, 12, 15], [10, 13, 16], [11, 14, 17]]]) 

And its shape (2, 3, 3) . Here, the first index is two images, and three x three are the matrix.

Then the question, more specifically formulated, is how to use reshape to get the following desired result :

 array([[ 0, 1, 2], [ 3, 4, 5], [ 6, 7, 8], [ 9, 10, 11], [12, 13, 14], [15, 16, 17]]) 

Whose figure (6, 3) . This ensures that the size of the array indexes the columns of the matrix x and y, as shown above. My natural tendency was to use the conversion directly to z as follows:

 out = z.reshape(2 * 3, 3) 

But its conclusion is this: indexes the rows of the matrices, not the columns:

 array([[ 0, 3, 6], [ 1, 4, 7], [ 2, 5, 8], [ 9, 12, 15], [10, 13, 16], [11, 14, 17]] 

Is it possible to change the form to obtain the required output above? Or more generally, can you control how each axis is used when using the change function ?

Two things:

• I know how to solve a problem. I can go through each element of the large matrix (z) transposed, and then apply the change form above. This slightly increases the computation time and is actually not problematic. But it does not generalize and does not feel python. So I was wondering if there is a standard enlightened way to do this.

• I did not know how to formulate this question. If anyone has a suggestion on how to better articulate this problem, I’m all ears.

Answer 1

Each array has a natural (1D flattened) order for its elements. When you modify the array, it is as if first smoothed out (thus, a natural order is obtained), and then it changes:

 In [54]: z.ravel() Out[54]: array([ 0, 3, 6, 1, 4, 7, 2, 5, 8, 9, 12, 15, 10, 13, 16, 11, 14, 17]) In [55]: z.ravel().reshape(2*3, 3) Out[55]: array([[ 0, 3, 6], [ 1, 4, 7], [ 2, 5, 8], [ 9, 12, 15], [10, 13, 16], [11, 14, 17]]) 

Note that in the "natural order" 0 and 1 are far apart. However, you change it, 0 and 1 will not be next to each other along the last axis, what you want in the desired array:

 desired = np.array([[ 0, 1, 2], [ 3, 4, 5], [ 6, 7, 8], [ 9, 10, 11], [12, 13, 14], [15, 16, 17]]) 

This requires some reordering, which in this case can be done using swapaxes :

 In [53]: z.swapaxes(1,2).reshape(2*3, 3) Out[53]: array([[ 0, 1, 2], [ 3, 4, 5], [ 6, 7, 8], [ 9, 10, 11], [12, 13, 14], [15, 16, 17]]) 

because swapaxes(1,2) puts the values ​​in the desired order

 In [56]: z.swapaxes(1,2).ravel() Out[56]: array([ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17]) In [57]: desired.ravel() Out[57]: array([ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17]) 

Note that the reshape method also has an order parameter that can be used to control (C- or F-) the order by which the elements are read from the array and placed in the reconfigured array. However, I do not think this will help in your case.

---

Another way to think about reshape boundaries is to say that all changes followed by ravel are the same:

 In [71]: z.reshape(3,3,2).ravel() Out[71]: array([ 0, 3, 6, 1, 4, 7, 2, 5, 8, 9, 12, 15, 10, 13, 16, 11, 14, 17]) In [72]: z.reshape(3,2,3).ravel() Out[72]: array([ 0, 3, 6, 1, 4, 7, 2, 5, 8, 9, 12, 15, 10, 13, 16, 11, 14, 17]) In [73]: z.reshape(3*2,3).ravel() Out[73]: array([ 0, 3, 6, 1, 4, 7, 2, 5, 8, 9, 12, 15, 10, 13, 16, 11, 14, 17]) In [74]: z.reshape(3*3,2).ravel() Out[74]: array([ 0, 3, 6, 1, 4, 7, 2, 5, 8, 9, 12, 15, 10, 13, 16, 11, 14, 17]) 

So, if the ravel of the desired array is different, there is no way to get it just for modification.

---

The same applies to changing the form with order='F' if you also use ravel with order='F' :

 In [109]: z.reshape(2,3,3, order='F').ravel(order='F') Out[109]: array([ 0, 9, 1, 10, 2, 11, 3, 12, 4, 13, 5, 14, 6, 15, 7, 16, 8, 17]) In [110]: z.reshape(2*3*3, order='F').ravel(order='F') Out[110]: array([ 0, 9, 1, 10, 2, 11, 3, 12, 4, 13, 5, 14, 6, 15, 7, 16, 8, 17]) In [111]: z.reshape(2*3,3, order='F').ravel(order='F') Out[111]: array([ 0, 9, 1, 10, 2, 11, 3, 12, 4, 13, 5, 14, 6, 15, 7, 16, 8, 17]) 

---

You can get the desired array using two forms:

 In [83]: z.reshape(2, 3*3, order='F').reshape(2*3, 3) Out[83]: array([[ 0, 1, 2], [ 3, 4, 5], [ 6, 7, 8], [ 9, 10, 11], [12, 13, 14], [15, 16, 17]]) 

but I stumbled upon it aimlessly.

---

If I do not completely understand your question, and x and y are givens (not z ), then you can get the desired array using row_stack instead of dstack :

 In [88]: z = np.row_stack([x, y]) In [89]: z Out[89]: array([[ 0, 1, 2], [ 3, 4, 5], [ 6, 7, 8], [ 9, 10, 11], [12, 13, 14], [15, 16, 17]]) 

Answer 2

You look at the dstack code, which you find that

 np.dstack((x, y)).T 

effective:

 np.concatenate([i[:,:,None] for i in (x,y)],axis=2).transpose([2,1,0]) 

It converts each array of components and then connects them along this new axis. Finally, he carries the axis.

Your goal is the same as (line stack)

 np.concatenate((x,y),axis=0) 

So, with a little reverse engineering, we can create it from z using

 np.concatenate([i[...,0] for i in np.split(zT,2,axis=2)],axis=0) np.concatenate([iT[:,:,0] for i in np.split(z,2,axis=0)],axis=0) 

or

 np.concatenate(np.split(zT,2,axis=2),axis=0)[...,0] 

or with partial transposition, we can first save the split-and-rejoin axis and just use concatenate :

 np.concatenate(z.transpose(0,2,1),axis=0) 

or its equivalent conversion

 (z.transpose(0,2,1).reshape(-1,3))