How to remove false matches from FLANNBASED Matcher in OPENCV?

Question

[Ask you to read the details of the question before marking a duplication or voting. I carefully searched and could not find a solution, and therefore posted a question here.]

I am trying to compare one image with several images and get a list of ALL matching images. I DO NOT want to draw key points between the images.

My solution is based on the following source code:

https://github.com/Itseez/opencv/blob/master/samples/cpp/matching_to_many_images.cpp

The above source code corresponds to a single image with multiple images and obtains the best matching image.

I modified the above sample and generated:

vector<vector<DMatch>> matches; vector<vector<DMatch>> good_matches; 

Now my question is: how to apply the nearest neighbor search relation to get good matches for multiple images?

Change 1:

My implementation is as follows:

• For each image in the dataset, compute SURF descriptors.

• Combine all descriptors into one large matrix.

• Create a FLANN index from a concatenated matrix.

• Calculate descriptors for the request image.

• Run a KNN search on the FLANN index to find the best matching image in the top 20 or less . K is given as 20.

• Filter out any inappropriate matches calculated in the previous step. (How??)

I successfully completed steps 1 to 5. I ran into a problem in step 6, where I cannot remove false matches.

Answer 1

There are two answers to your problem. First, you must use a completely different technique, the second answer is how to do what you asked.

Use another method

You want to find duplicates of the given image. Traditionally, you do this by comparing global image descriptors rather than local function descriptors.

The easiest way to do this is to combine the local function descriptors into a local descriptor. The standard method here is the "bag of visual words." In OpenCV, this is called Bag-Of-Words (e.g. BOWTrainer , BOWImgDescriptorExtractor , etc.). See the documentation for using this.

In samples/cpp/bagofwords_classification.cpp

There is an example code.

The benefits will be that you will get more reliable results (depending on the implementation of what you are doing now), and that the match is faster.

Use your method

I understand that you want to remove points from the input that lead to false positives in your mapping.

You cannot delete points from FLANN ( 1 , 2 , 3 ). FLANN creates a tree for quick searches. Depending on the type of tree, deleting a node becomes impossible. Guess that FLANN uses a KD tree that does not make it easy to delete points.

FlannBasedMatcher does not support masking of valid matches for descriptor sets because the :: Index flange does not support this.

I would suggest using a radius search instead of a simple search. Alternatively, look at the L2 distance of the matches found and write a function in the code to see if the distance falls below the threshold.

Edit

I should also note that you can rebuild your flann-tree. Obviously, when doing this, a performance penalty occurs. But if you have a large number of requests and some functions that occur as false positives, too often, it may make sense to do this once.

To do this, you need the functions DescriptorMatcher::clear() , and then DescriptorMatcher::add(const vector<Mat>& descriptors) . Referenz .

Answer 2

First, you need to define "inadequate matches," without this definition you cannot do anything.

There are several free definitions that spring:

1: inadequate matches = match with distance> target distance

In this case, a search for the lunar radius may be more appropriate, since it will only give you those indices within a predetermined radius from the target:

http://docs.opencv.org/trunk/modules/flann/doc/flann_fast_approximate_nearest_neighbor_search.html#flann-index-t-radiussearch

2: inadequate correspondence = comparison with distance> dynamically determined distance based on the obtained k-nn

This is more complicated and from head to head I can think of two possible solutions:

2a: Determine some test coefficient based on the distance of the first 1-NN, for example:

base distance = distance to 1NN

inadequate match_k = match distance_k> = a * base distance;

2b: Use a dynamic threshold method such as the Otsu threshold for normalized distance distribution for k-nn, thus splitting k-nn into two groups, a group that contains 1-nn is a suitable group, the other is an inadequate group.

http://en.wikipedia.org/wiki/Otsu 's_method,

http://docs.opencv.org/modules/imgproc/doc/miscellaneous_transformations.html#threshold .