Tricky filling holes in the image

Question

I need to fill holes in images using python. This is an image with the objects that I managed to get - they really are the edges of the objects that I want, so I need to fill them.

It looked very simple using ndimage.binary_fill_holes(A) , but the problem is that it produces this (manually filled in red):

But I need this:

In any case, can this be solved?

This is the first image without axes if you want to try:

Answer 1

I think I found a solution. This is a little longer since I did not have enough time, but maybe it helps. I coded, if only for this problem, but for many images this should be easy to generalize.

Some naming conventions first:

• I define “first level areas” as compact areas that are surrounded by a background. Such first level areas may consist of different subregions.
• A first-level region consisting of more than one subdomain is called a critical region.

My main idea is to compare the contour lengths of two subregions that are part of the same critical region. However, I do not compare their full length of the contour, but only a segment that is close to the background . One with a shorter contour close to the background is considered a hole.

First, I'll start with images of the results.

Some idea of ​​what we are talking about vizualizing naming conventions above:

Two subdomains of the critical region. Two border segments of each of the areas that are close to the background are marked with different colors (very thin, blue and dark red, but visible). These segments are obviously not ideal (thin areas cause errors), but enough to compare their lengths:

Final result. In case you want the hole to be closed, let me know, you just need to assign the original black outlines to the regions instead of the background ([EDIT] I included three marked lines of code that assign the borders to the regions as you wanted):

The code is attached here. I used the OpenCV loop function, which is quite complicated, and some masking methods. The code is legthy because of its visualization, sorry for its limited readability, but there seems to be no two-line solution to this problem.

Some concluding remarks: I first tried matching paths using sets of dots, which would avoid loops and allow the use of set.intersection to define two path segments close to the background, but since your black lines are rather thick, the paths are slipperyly incompatible. I tried skeletonizing the contours, but this opened up another can of worms, so I worked with the dump approach, looping and calculating the distance between the contour points. There may be a more convenient way to do this part, but it works.

I also looked at using Shapely , there may be ways to take advantage of this, but I did not find them, so I dropped it again.

 import numpy as np import scipy.ndimage as ndimage from matplotlib import pyplot as plt import cv2 img= ndimage.imread('image.png') # Label digfferentz original regions labels, n_regions = ndimage.label(img) print "Original number of regions found: ", n_regions # count the number of pixels in each region ulabels, sizes = np.unique(labels, return_counts=True) print sizes # Delete all regions with size < 2 and relabel mask_size = sizes < 2 remove_pixel = mask_size[labels] labels[remove_pixel] = 0 labels, n_regions = ndimage.label(labels) #,s) print "Number of regions found (region size >1): ", n_regions # count the number of pixels in each region ulabels, sizes = np.unique(labels, return_counts=True) print ulabels print sizes # Determine large "first level" regions first_level_regions=np.where(labels ==1, 0, 1) labeled_first_level_regions, n_fl_regions = ndimage.label(first_level_regions) print "Number of first level regions found: ", n_fl_regions # Plot regions and first level regions fig = plt.figure() a=fig.add_subplot(2,3,1) a.set_title('All regions') plt.imshow(labels, cmap='Paired', vmin=0, vmax=n_regions) plt.xticks([]), plt.yticks([]), plt.colorbar() a=fig.add_subplot(2,3,2) a.set_title('First level regions') plt.imshow(labeled_first_level_regions, cmap='Paired', vmin=0, vmax=n_fl_regions) plt.xticks([]), plt.yticks([]), plt.colorbar() for region_label in range(1,n_fl_regions): mask= labeled_first_level_regions!=region_label result = np.copy(labels) result[mask]=0 subregions = np.unique(result).tolist()[1:] print region_label, ": ", subregions if len(subregions) >1: print " Element 4 is a critical element: ", region_label print " Subregions: ", subregions #Critical first level region crit_first_level_region=np.ones(labels.shape) crit_first_level_region[mask]=0 a=fig.add_subplot(2,3,4) a.set_title('Crit. first level region') plt.imshow(crit_first_level_region, cmap='Paired', vmin=0, vmax=n_regions) plt.xticks([]), plt.yticks([]) #Critical Region Contour im = np.array(crit_first_level_region * 255, dtype = np.uint8) _, contours0, hierarchy = cv2.findContours( im.copy(), cv2.RETR_TREE, cv2.CHAIN_APPROX_NONE) crit_reg_contour = [contours0[0].flatten().tolist()[i:i+2] for i in range(0, len(contours0[0].flatten().tolist()), 2)] print crit_reg_contour print len(crit_reg_contour) #First Subregion mask2= labels!=subregions[1] first_subreg=np.ones(labels.shape) first_subreg[mask2]=0 a=fig.add_subplot(2,3,5) a.set_title('First subregion: '+str(subregions[0])) plt.imshow(first_subreg, cmap='Paired', vmin=0, vmax=n_regions) plt.xticks([]), plt.yticks([]) #First Subregion Contour im = np.array(first_subreg * 255, dtype = np.uint8) _, contours0, hierarchy = cv2.findContours( im.copy(), cv2.RETR_TREE, cv2.CHAIN_APPROX_NONE) first_sub_contour = [contours0[0].flatten().tolist()[i:i+2] for i in range(0, len(contours0[0].flatten().tolist()), 2)] print first_sub_contour print len(first_sub_contour) #Second Subregion mask3= labels!=subregions[0] second_subreg=np.ones(labels.shape) second_subreg[mask3]=0 a=fig.add_subplot(2,3,6) a.set_title('Second subregion: '+str(subregions[1])) plt.imshow(second_subreg, cmap='Paired', vmin=0, vmax=n_regions) plt.xticks([]), plt.yticks([]) #Second Subregion Contour im = np.array(second_subreg * 255, dtype = np.uint8) _, contours0, hierarchy = cv2.findContours( im.copy(), cv2.RETR_TREE, cv2.CHAIN_APPROX_NONE) second_sub_contour = [contours0[0].flatten().tolist()[i:i+2] for i in range(0, len(contours0[0].flatten().tolist()), 2)] print second_sub_contour print len(second_sub_contour) maxdist=6 print "Points in first subregion close to first level contour:" close_1=[] for p1 in first_sub_contour: for p2 in crit_reg_contour: if (abs(p1[0]-p2[0])+abs(p1[1]-p2[1]))<maxdist: close_1.append(p1) break print close_1 print len(close_1) print "Points in second subregion close to first level contour:" close_2=[] for p1 in second_sub_contour: for p2 in crit_reg_contour: if (abs(p1[0]-p2[0])+abs(p1[1]-p2[1]))<maxdist: close_2.append(p1) break print close_2 print len(close_2) for p in close_1: result[p[1],p[0]]=1 for p in close_2: result[p[1],p[0]]=2 if len(close_1)>len(close_2): print "first subregion is considered a hole:", subregions[0] hole=subregions[0] else: print "second subregion is considered a hole:", subregions[1] hole=subregions[1] #Plot Critical region with subregions a=fig.add_subplot(2,3,3) a.set_title('Critical first level region with subregions') plt.imshow(result, cmap='Paired', vmin=0, vmax=n_regions) plt.xticks([]), plt.yticks([]) result2=result.copy() #Plot result fig2 = plt.figure() a=fig2.add_subplot(1,1,1) a.set_title('Critical first level region with subregions and bordering contour segments') plt.imshow(result2, cmap='flag', vmin=0, vmax=n_regions) plt.xticks([]), plt.yticks([]) #Plot result mask_hole=np.where(labels ==hole, True, False) labels[mask_hole]=1 labels=np.where(labels > 1, 2, 1) # [Edit] Next two lines include black borders into final result mask_borders=np.where(img ==0, True, False) labels[mask_borders]=2 fig3 = plt.figure() a=fig3.add_subplot(1,1,1) a.set_title('Final result') plt.imshow(labels, cmap='flag', vmin=0, vmax=n_regions) plt.xticks([]), plt.yticks([]) plt.show()