Numpy - number of equal arrays

Question

I want to count the number of equal matrices that I encounter after splitting a large matrix.

mat1 = np.zeros((4, 8)) split4x4 = np.split(mat1, 4) 

Now I want to know how many equal matrices are in split4x4, but collections.Counter(split4x4) throws an error. Is there a numpy built-in way for this?

Answer 1

This can be done in a completely vectorized way using the numpy_indexed package (disclaimer: I am its author):

 import numpy_indexed as npi unique_rows, row_counts = npi.count(mat1) 

This should be significantly faster than using collections. Counter.

Answer 2

Perhaps the easiest way is to use np.unique and flatten the broken arrays to compare them as a tuple:

 import numpy as np # Generate some sample data: a = np.random.uniform(size=(8,3)) # With repetition: a = np.r_[a,a] # Split a in 4 arrays s = np.asarray(np.split(a, 4)) s = [tuple(e.flatten()) for e in s] np.unique(s, return_counts=True) 

Note: return_counts argument is np.unique new in version 1.9.0.

Another clean numpy inspired solution posted

 # Generate some sample data: In: a = np.random.uniform(size=(8,3)) # With some repetition In: a = r_[a,a] In: a.shape Out: (16,3) # Split a in 4 arrays In: s = np.asarray(np.split(a, 4)) In: print s Out: [[[ 0.78284847 0.28883662 0.53369866] [ 0.48249722 0.02922249 0.0355066 ] [ 0.05346797 0.35640319 0.91879326] [ 0.1645498 0.15131476 0.1717498 ]] [[ 0.98696629 0.8102581 0.84696276] [ 0.12612661 0.45144896 0.34802173] [ 0.33667377 0.79371788 0.81511075] [ 0.81892789 0.41917167 0.81450135]] [[ 0.78284847 0.28883662 0.53369866] [ 0.48249722 0.02922249 0.0355066 ] [ 0.05346797 0.35640319 0.91879326] [ 0.1645498 0.15131476 0.1717498 ]] [[ 0.98696629 0.8102581 0.84696276] [ 0.12612661 0.45144896 0.34802173] [ 0.33667377 0.79371788 0.81511075] [ 0.81892789 0.41917167 0.81450135]]] In: s.shape Out: (4, 4, 3) # Flatten the array: In: s = asarray([e.flatten() for e in s]) In: s.shape Out: (4, 12) # Sort the rows using lexsort: In: idx = np.lexsort(sT) In: s_sorted = s[idx] # Create a mask to get unique rows In: row_mask = np.append([True],np.any(np.diff(s_sorted,axis=0),1)) # Get unique rows: In: out = s_sorted[row_mask] # and count: In: for e in out: count = (e == s).all(axis=1).sum() print e.reshape(4,3), count Out:[[ 0.78284847 0.28883662 0.53369866] [ 0.48249722 0.02922249 0.0355066 ] [ 0.05346797 0.35640319 0.91879326] [ 0.1645498 0.15131476 0.1717498 ]] 2 [[ 0.98696629 0.8102581 0.84696276] [ 0.12612661 0.45144896 0.34802173] [ 0.33667377 0.79371788 0.81511075] [ 0.81892789 0.41917167 0.81450135]] 2