OpenCV: comparing multiple images using ORB

Question

I am trying to create a C ++ program where there are many images in the list compared to a single input image. I got everything that works, and the program creates DMatch matches.

Now I'm trying to determine which of the image lists that are compared with the original image is the best match. At first I tried to do this by simply comparing the number of matches between the images, but the problem is that when the generated image has many key points; they also tend to have many matches, at least in my program.

So, how can I determine which image array is the best fit for the original image? I use this loop to determine matches, but it really doesn't work:

 vector< vector<DMatch> > filteredMatches; vector<int> goodIds; Ptr<DescriptorMatcher> matcher(new BFMatcher(NORM_HAMMING, false)); printf("bad matches: "); for(size_t i = 0; i < images.size();i++){ vector<DMatch> matches, good_matches; matcher->clear(); matcher->match(images[i], tex_des, matches); if(matches.size() < 8){ printf("F%d,", (int)i + 1); continue; } double min_dist = 100; for(size_t j = 0; j < matches.size(); j++ ){ double dist = matches[j].distance; if( dist < min_dist ) min_dist = dist; } if(min_dist > 50.0){ printf("D%d,", (int)i + 1); continue; } double good_dist = min_dist * 3; for(size_t j = 0; j < matches.size(); j++ ){ if(matches[j].distance < good_dist) good_matches.push_back(matches[j]); } size_t size = good_matches.size(); if(size < 8){ printf("M%d,", (int)i + 1); continue; } vector<Point2f> srcPoints(size); vector<Point2f> dstPoints(size); for(size_t j = 0; j < size; j++){ srcPoints[j] = destination[good_matches[j].trainIdx].pt; dstPoints[j] = keyPoints[i][good_matches[j].queryIdx].pt; } vector<unsigned char> inliersMask(srcPoints.size()); Mat H = findHomography(srcPoints, dstPoints, CV_FM_RANSAC, 3.0, inliersMask); vector<DMatch> inliers; for(size_t j = 0; j < inliersMask.size(); j++){ if(inliersMask[j]){ inliers.push_back(good_matches[j]); } } if(inliers.size() < 4){ printf("S%d,", (int)i + 1); continue; } filteredMatches.push_back(inliers); goodIds.push_back((int)i); H.release(); } printf(" good matches: "); int best = -1; int amount = 0; for(size_t i = 0; i < filteredMatches.size(); i++){ int size = (int)filteredMatches.at(i).size(); if(size < 8) continue; printf("%d,", goodIds[i] + 1); if(amount < size){ amount = size; best = i; } } if(best >= 0) printf(" best match on image: %d, keypoints: %d, ", goodIds[best] + 1, amount); 

If someone can point me to a function or logic that I should use, I would really appreciate it!

Answer 1

There is no direct answer. For best results, you need to implement some kind of transformation and do clustering on the converted map, and not just summarize the distances. This is difficult and even publication .

Otherwise, you will have to use more practical methods, such as filtering sizes and histograms. You can take a look at the OpenCV stitcher , isolate the module you are interested in, and customize the source code to suit your needs.

Answer 2

You should choose only truly stable matches. I would recommend reading: OpenCV 2 Computer Vision Application Programming Cookbook - Chapter 9 - Matching Images Using Random Selective Consensus ( http://opencv-cookbook.googlecode.com/svn/trunk/Chapter%2009/ ).

Answer 3

A short search for your problem provided me with the following entry in the opencv answer sections:

/ forum with CV-response

which seems to give an answer to the question that you have. To filter the results obtained in the answer, I would like to take a look at the RANSAC algorithm to find the best results in the choice of correspondence.

RANSAC desctiption Wikipedia

At the very least, this should point you in the right direction.

Answer 4

It depends on what the image is in the list. You cannot have one solution for every vision problem in the world. For example, the project I'm working on should recognize the material on the walls. You cannot just compare it with a different picture of the wall with another material and hope for a match.

In my case, I needed to create descriptors. Descriptors is an algorithm that outputs a value that can be compared with another value of another image. OpenCV has many descriptors such as LBP, SURF, etc. Simply put, you no longer compare the image, you compare the output value of the image descriptor 1 with the value descriptor of all the images in the list.

You need to pick up handles that use your eyes / brain to find a match in real life. For example, if the match is based on color, you can use CLD or DCD. If the mapping is texture based, use LBP. You can also do what I did in my project and use many descriptors and use Machine Learning using prepared data algorithms to find the best fit.

So, to summarize, there is no silver bullet that can fix the whole problem of vision. You need to adapt the solution to the problem.

Hope this helps!