Find multiple maximum values ​​in a 2d array fast

Question

The situation is as follows:

I have a numpy 2D array. Its form (1002, 1004). Each element contains a value from 0 to Inf. Now I want to determine the first 1000 maximum values ​​and store the corresponding indexes in a list with the name x and a list with the name y. This is because I want to build the maximum values, and the indices really correspond to the real time x and y positions of the value.

What I still have:

x = numpy.zeros(500)
y = numpy.zeros(500)

for idx in range(500):
    x[idx] = numpy.unravel_index(full.argmax(), full.shape)[0]
    y[idx] = numpy.unravel_index(full.argmax(), full.shape)[1]
    full[full == full.max()] = 0.

print os.times()

Here's the full array of 2D numpy. As you can see from the for loop, I only determine the first 500 maximum values ​​at a given moment. This, however, already takes about 5 s. For the first 1000 maximum values, the user time should be about 0.5 s. I noticed that the very laborious part sets the previous maximum value to 0 each time. How can I speed up the process?

Thank you very much!

Answer 1

If you have numpy 1.8, you can use a function or method . Here is a script that calculates and : argpartitionxy

import numpy as np

# Create an array to work with.
np.random.seed(123)
full = np.random.randint(1, 99, size=(8, 8))

# Get the indices for the largest 'num_largest' values.
num_largest = 8

indices = (-full).argpartition(num_largest, axis=None)[:num_largest]
# OR, if you want to avoid the temporary array created by '-full':
# indices = full.argpartition(full.size - num_largest, axis=None)[-num_largest:]

x, y = np.unravel_index(indices, full.shape)

print("full:")
print(full)
print("x =", x)
print("y =", y)
print("Largest values:", full[x, y])
print("Compare to:    ", np.sort(full, axis=None)[-num_largest:])

Exit:

full:
[[67 93 18 84 58 87 98 97]
 [48 74 33 47 97 26 84 79]
 [37 97 81 69 50 56 68  3]
 [85 40 67 85 48 62 49  8]
 [93 53 98 86 95 28 35 98]
 [77 41  4 70 65 76 35 59]
 [11 23 78 19 16 28 31 53]
 [71 27 81  7 15 76 55 72]]
x = [0 2 4 4 0 1 4 0]
y = [6 1 7 2 7 4 4 1]
Largest values: [98 97 98 98 97 97 95 93]
Compare to:     [93 95 97 97 97 98 98 98]

Answer 2

, @Inspired, NumPy , , NumPy, NumPy C/Fortran, -by-item Python.

, O(n log n), , O(n) Python. np.unique :

import numpy as np

def nlargest_indices(arr, n):
    uniques = np.unique(arr)
    threshold = uniques[-n]
    return np.where(arr >= threshold)

full = np.random.random((1002,1004))
x, y = nlargest_indices(full, 10)
print(full[x, y])
print(x)
# [  2   7 217 267 299 683 775 825 853]
print(y)
# [645 621 132 242 556 439 621 884 367]

---

nlargest_indices ()

def nlargest_indices_orig(full, n):
    full = full.copy()
    x = np.zeros(n)
    y = np.zeros(n)

    for idx in range(n):
        x[idx] = np.unravel_index(full.argmax(), full.shape)[0]
        y[idx] = np.unravel_index(full.argmax(), full.shape)[1]
        full[full == full.max()] = 0.
    return x, y


In [97]: %timeit nlargest_indices_orig(full, 500)
1 loops, best of 3: 5 s per loop

In [98]: %timeit nlargest_indices(full, 500)
10 loops, best of 3: 133 ms per loop

timeit nlargest_indices_orig, full .

:

def base(full, n):
    full = full.copy()

In [102]: %timeit base(full, 500)
100 loops, best of 3: 4.11 ms per loop

, 4 5s nlargest_indices_orig.

---

: nlargest_indices nlargest_indices_orig , arr .

nlargest_indices n arr, x y, .

nlargest_indices_orig n arr, x y . x y, , , , .

, , .

Answer 3

n max/min 2d, ( )

indx = divmod((-full).argpartition(num_largest,axis=None)[:3],full.shape[0])

, 2d .

Nevermind. , , , num_largest = 3.

Answer 4

, - . , 1002 * 1004 500 , 500 .

, , : 1000 ( ) - 2D- ( ). , - ( heapq), .