Geek Answers Handbook

Scipy ndimage morphology operators saturate my computer's memory (8 GB)

I need to calculate the morphological discovery for a 3D shape array (400,401,401), size 64320400 bytes, using a three-dimensional structure element with a radius of 17 or higher. The size of an ndarray structure element is 42875 bytes. Using scipy.ndimage.morphology.binary_opening, the whole process consumes 8 GB of RAM.

I read scipy/ndimage/morphology.pyon GitHub, and as far as I can tell, the morphology erosion operator is implemented in pure C. I find it difficult to understand the source ni_morphology.c, so I have not found any part of this code that leads to such a huge memory usage. Adding more RAM is not a workable solution, since memory usage can increase exponentially with the radius of the structural element.

To reproduce the problem:

import numpy as np
from scipy import ndimage

arr_3D = np.ones((400,401,401),dtype="bool")

str_3D = ndimage.morphology.generate_binary_structure(3,1)
big_str_3D = ndimage.morphology.iterate_structure(str_3D,20)

arr_out_3D = ndimage.morphology.binary_opening(arr_3D, big_str_3D)

It takes approximately 7 GB of RAM.

Does anyone have any suggestions on how to calculate the morphology in the example described above?

+4
source share
2 answers

, . , R ^ 6, R - . ! , , ( ) , SciPy , Python, SciPy C, .

OP: " → - O (n log n) ". , . . , , .

, - () , .

, ., A ( ) B, . ( ) , , . , A B, , , 0,5, A B. ( , 0, , , Kukal et al. > 0,5 scipy.ndimage.binary_dilation . , , , ref 1 )

, , , scipy.signal.fftconvolve(A,B,'same') - , , . " " , A, ( B).

, :

from scipy.signal import fftconvolve
def dilate(A,B):
    return fftconvolve(A,B,'same')>0.5

: A, B, , . , scipy.ndimage.binary_erosion - 1s , , R B. , , scipy.ndimage.binary_erosion. ( , , .)

from scipy.signal import fftconvolve
import numpy as np
def erode_v1(A,B,R):
    #R should be the radius of the spherical kernel, i.e. half the width of B
    A_inv = np.logical_not(A)
    A_inv = np.pad(A_inv, R, 'constant', constant_values=1)
    tmp = fftconvolve(A_inv, B, 'same') > 0.5
    #now we must un-pad the result, and invert it again
    return np.logical_not(tmp[R:-R, R:-R, R:-R])

, ., , > -0,5, - "B ( , ). erode_v1, , :

from scipy.signal import fftconvolve
import numpy as np
def erode_v2(A,B):
    thresh = np.count_nonzero(B)-0.5
    return fftconvolve(A,B,'same') > thresh

, - . :

  • 2D, 3D, , scipy.ndimage ( skimage, ndimage).
  • (R = 21) 30 ! 20 .
  • , - , .

:

: , erode_v1. 1s , . , , , , . , "", - . ( , ), , , .

: , - . B . , , . (, ), B .

, , , :

  1. . , . . , . . , . . http://www.cs.utep.edu/vladik/misha5.pdf
  2. . , . , . . Http://http%3A%2F%2Fwww2.humusoft.cz%2Fwww%2Fpapers%2Ftcp07%2F001_kukal.pdf
+2

, - . . 400x400x400 tbh...

AFAIK, /, , , 3 : + / + ...

, ... , ...

+1

More articles:


All Articles