OpenCV - image stitching

Question

I use the following stitching code to input images. For an unknown reason, the result is a result of crap! It seems that the homography matrix is ​​wrong (or wrong) because the transformed image is like an “exploited star”! I commented on the part that, it seems to me, is the source of the problem, but I can’t understand this. Any help or point is appreciated!

Have a nice day, Ali

void Stitch2Image(IplImage *mImage1, IplImage *mImage2) { // Convert input images to gray IplImage* gray1 = cvCreateImage(cvSize(mImage1->width, mImage1->height), 8, 1); cvCvtColor(mImage1, gray1, CV_BGR2GRAY); IplImage* gray2 = cvCreateImage(cvSize(mImage2->width, mImage2->height), 8, 1); cvCvtColor(mImage2, gray2, CV_BGR2GRAY); // Convert gray images to Mat Mat img1(gray1); Mat img2(gray2); // Detect FAST keypoints and BRIEF features in the first image FastFeatureDetector detector(50); BriefDescriptorExtractor descriptorExtractor; BruteForceMatcher<L1<uchar> > descriptorMatcher; vector<KeyPoint> keypoints1; detector.detect( img1, keypoints1 ); Mat descriptors1; descriptorExtractor.compute( img1, keypoints1, descriptors1 ); /* Detect FAST keypoints and BRIEF features in the second image*/ vector<KeyPoint> keypoints2; detector.detect( img1, keypoints2 ); Mat descriptors2; descriptorExtractor.compute( img2, keypoints2, descriptors2 ); vector<DMatch> matches; descriptorMatcher.match(descriptors1, descriptors2, matches); if (matches.size()==0) return; vector<Point2f> points1, points2; for(size_t q = 0; q < matches.size(); q++) { points1.push_back(keypoints1[matches[q].queryIdx].pt); points2.push_back(keypoints2[matches[q].trainIdx].pt); } // Create the result image result = cvCreateImage(cvSize(mImage2->width * 2, mImage2->height), 8, 3); cvZero(result); // Copy the second image in the result image cvSetImageROI(result, cvRect(mImage2->width, 0, mImage2->width, mImage2->height)); cvCopy(mImage2, result); cvResetImageROI(result); // Create warp image IplImage* warpImage = cvCloneImage(result); cvZero(warpImage); /************************** Is there anything wrong here!? *******************/ // Find homography matrix Mat H = findHomography(Mat(points1), Mat(points2), 8, 3.0); CvMat HH = H; // Is this line converted correctly? // Transform warp image cvWarpPerspective(mImage1, warpImage, &HH); // Blend blend(result, warpImage); /*******************************************************************************/ cvReleaseImage(&gray1); cvReleaseImage(&gray2); cvReleaseImage(&warpImage); } 

Answer 1

Here is what I suggest you try in the following order:

1) Use the CV_RANSAC option for homography. See http://opencv.willowgarage.com/documentation/cpp/calib3d_camera_calibration_and_3d_reconstruction.html

2) Try using other descriptors, especially SIFT or SURF, that come with OpenCV. For some images, the FAST or BRIEF descriptors are not sufficiently different. EDIT (Aug '12): BRIEF-based ORB descriptors are pretty good and fast!

3) Try to look at the homograph matrix (go into debug mode or print it) and see if it is compatible.

4) If the above does not give you clues, try looking at the created matches. Does it correspond to one point in one image with several points in another image? If so, the problem should again be with descriptors or a detector.

My guess is that these descriptors (so 1) or 2) should fix it).

Answer 2

Also switch to Hamming distance instead of L1 distance in BruteForceMatcher. SHORT descriptors should be compared using the Hamming distance.

Answer 3

Your homography may be calculated on the basis of incorrect matches, and thus constitutes a bad assignment. I suggest skipping the matrix by additionally checking the interdependence between its rows.

You can use the following code:

 bool cvExtCheckTransformValid(const Mat& T){ // Check the shape of the matrix if (T.empty()) return false; if (T.rows != 3) return false; if (T.cols != 3) return false; // Check for linear dependency. Mat tmp; T.row(0).copyTo(tmp); tmp /= T.row(1); Scalar mean; Scalar stddev; meanStdDev(tmp,mean,stddev); double X = abs(stddev[0]/mean[0]); printf("std of H:%g\n",X); if (X < 0.8) return false; return true; }