Comparing similar images with photos - detecting the difference, diff image

Question

The situation is unique from all that I have already managed to find, and consists in the following: if I took a photo with two similar images, I would like to emphasize the distinctive features of the two images. For example, the following two halves of children determine the difference:

Differences in images will be missing / bits and / or color changes added, as well as the type of differences that are easily detected from the source image files without doing anything more smart than comparing in half. However, the fact that they are subject to fluctuations in light and inaccurate photos, I will need a much softer / smarter algorithm.

As you can see, the images will not necessarily match perfectly if superimposed.

This question is labeled as agnostic, as I expect the answers to point to the appropriate algorithms, however I would also be interested in current implementations if they exist, especially in Java, Ruby or C.

Answer 1

The following approach should work. All of these features are available in OpenCV. Take a look at this example for calculating homographs.

• Identify key points in two images using an angular detector.
• Retrieve the descriptors (SIFT / SURF) for the key points.
• Match key points and calculate homography with RANSAC, which aligns the second image to the first.
• Apply homography to the second image so that it matches the first.
• Now just calculate the difference in pixels between the two images, and the difference image will highlight everything that has changed from the first to the second.

Answer 2

My general approach would be to use an optical stream to align both images and perform pixel-by-pixel comparisons after they are aligned.

However, for specifics, standard optical streams (OpenCV, etc.) are more likely to fail if the two images are significantly different, as in your case. If this really fails, there are recent optical flow methods that should work, even if the images are very different from each other. For example, you can look at an article on SIFT flows of Ce Liu and others , which solves this problem with sparse correspondences.