InitUndistortRectifyMap and Remap

Question

I am currently writing an openCV program for a pair of stereo cameras. Camera calibration and stereo calibration are in progress.

The next step is to find the position of the object in space from the two images that I get. That is why I have to stereo image rectify and do my calculations afterwards.

The problem I am encountering with initUndistortRectifyMap is this:

-If I go through R1 or R2 computed using stereoRectify() before initUndistortRectifyMap() , I get black images after reassignment.

-If I pass r (empty matrix) to initUndistortRectifyMap() , I get unrectified images after reassignment. The images I get are a little distorted.

I need to pass R1 and R2 to initUndistortRectifyMap() to fix 2 cameras, otherwise when passing through an empty matrix, the stereo heads will not rotate in the same plane.

Below is my code:

 stereoRectify(intrinsic[0], distCoeffs[0], intrinsic[1], distCoeffs[1], imageSize, R, T_Stereo, R1, R2, newP1, newP2, Q, CV_CALIB_ZERO_DISPARITY, -1, imageSize); if (x_Cam.GetSerial()=="4002678487") { initUndistortRectifyMap(intrinsic[0], distCoeffs[0], R1, newP1, imageSize, CV_16SC2 , mapx1, mapy1); remap(x_Image, imageRectified[0],mapx1, mapy1, INTER_LINEAR); return imageRectified[0]; } if (x_Cam.GetSerial()=="4002702131") { //flip(in, in, -1); initUndistortRectifyMap(intrinsic[1], distCoeffs[1], R2, newP2, imageSize, CV_16SC2 , mapx2, mapy2); remap(x_Image, imageRectified[1],mapx2, mapy2, INTER_LINEAR, BORDER_CONSTANT, 0); return imageRectified[1]; } 

I checked all the matrix values ​​going to stereoRectify() and they are correct. The rotation matrices R1 and R2 seem correct. I just get black images as output.

I tried passing garbage values ​​to initUndistortRectifyMap() for R1 and R2 (R1 * R2, for example) to see the effect simply, and I got strange results, but not black images.

Answer 1

Well, I somehow circumvented the problem and thought that I would share the solution if anyone could use it.

InitUndistortRectifyMap displayed blank images when using the Rotation Matrix R1 and R2 matrices created by StereoRectify.

So I tried using StereoRectifyUncalibrated, which produces 2 homography matrices H1 and H2, and I calculated the rotations according to the OpenCV documentation:

R1 = inv (CamMatrix1) * H1 * CamMatrix1 R2 = inv (CamMatrix2) * H2 * CamMatrix2

I passed the new R1 and R2 to InitUndistortRectifyMap and reassigned, and the results were satisfactory.

Answer 2

I am going to dig this old post because I recently made some efforts to research this problem. Namely, I encountered the same problem - after the correct stereo calibration and correction (all opencv provided stereo-vision equipment), finally, the "distorted" images were black. Let me explain why this is so.

First of all, how remapping works (src, dst, mapX, mapY). If you go to the documentation , you will see that

And if one of the values ​​mapX (x, y) or mapY (x, y) is outside src, dst (x, y) = 0 (black!). This means that all pairs (x, y) have unvalid values ​​in mapX or mapY. For example, in my case, for reasons that I cannot explain, I have all the values ​​in mapY negative.

But the reason for this is that this is happening, in particular, in satellite configurations. I made several examples when the cameras were aligned so that the rotation angles between them were small (up to several degrees). In this case, opencv procedures work well. The "black" exit problems that I had when the two Euler angles were the same, and one angle (rotation around Y) was 17 degrees. In this case, I did some experiment - I translated mapX and mapY as follows:

 for(int i=0;i<_imageSize.width;i++) for(int j=0;j<_imageSize.height;j++) { _mapXA.at<float>(j, i) -= 1200; _mapYA.at<float>(j, i) -= 1200; _mapXB.at<float>(j, i) += 2500; _mapYB.at<float>(j, i) += 2500; } 

(due to the fact that I saw _mapXA, it was too big, and _mapYB had negative values ​​around -1500). Surprisingly, the output after reassignment () was not black. The image looked exactly like it was rotated around a certain angle (maybe 17, I definitely did not check it), but it was not corrected at all.

It is possible that opencv cannot control stereoscopic non-parallel cameras well. I mean, this works for some examples, but not for some - but this means that this method is not real. In addition, you can find such information as Gary Bradsky and Adrian Kaeler Learning OpenCV:

Once again, this is one of the reasons you will achieve better results if you camera should be as close to the front as possible (at least until you have a stereo view)