How to use Kinect depth data in a watershed image segmentation

Question

I have RGB images and depths from Kinect in png format. I'm trying to use depth data with watershed segmentation, but I don't know how to combine both data and get a more accurate result. I checked some documents, but I did not understand the results or could not find a solution written specifically for the watershed algorithm. How to include depth data as a control point in the segmentation process?

I am using the MatLab Image Processing Toolbox.

Images from Nathan Silberman et al. Database on the Silberman website

An example of an RGB image and its corresponding depth file are shown below (note that the depth image, originally a binary image, is converted to uint8):

Update: I tried to create a grayscale image with an RGB source along with depth data, taking each channel (red, green, blue and depth) and calculating their weights; then including values ​​multiplied by their weights for each corresponding pixel. But the resulting image in shades of gray does not significantly improve the result. This is no better than just RGB-based segmentation. What else can I do if I follow this approach? Alternatively, how can I see the effects of depth data?

Answer 1

if you did not have error loading, the depth image is black and does not contain depth data. Keep in mind that (Dutch term) you are comparing apples and pears here. Images with images are not depth images; they are contour selections.

Then the following thing arises, in which you are mistaken, depth images have a lower resulting ability than color images. for kinect v2, it is only 512x424, and kinekt is one true vision of depth even lower than its return raster image size (this is low resolution depth, and not every pixel in the measurement, unlike kinect v2). But then v2 has an even better video output.

If you need a better rgb image watershed, then align a few camera frames to get rid of camera noise.

PS I recommend you download windows kinect sdk and see the samples provided with it.