Emgu (OpenCV for C #) - creating a mismatch map using cvStereoRectify

Question

I use the large Emgu C # shell for OpenCV to collect images from the built-in stereo. Two webcams are tied to a piece of wood, 35 cm apart, in order to hopefully allow me to produce depth maps in the range of 10-20 m. I set them as parallel as I can (about 89.3 degrees from the triangulation test )

I am trying to make a map of the differences from them, and although the process works like code, the results are very random. By this, I mean that every time I try to run a stereo fix, I get very different results, and the image is often so distorted that almost nothing is visible on the screen.

As I understand it, the way to do this is as follows:

---

1) Print a drawing of a chessboard (for example, 6 by 8 internal corners) - attach to something flat.

2) Take a set of 10 photos from camera 1, holding the chessboard in full screen, but in different positions.

3) use CameraCalibration.FindChessboardCorners to find the inside corners (6 by 8)

4) use img.FindCornerSubPix () to refine these angular locations at subpixel level

5) use CameraCalibration.CalibrateCamera () to calculate internal camera information and save it as an XML file

6) Repeat above for camera 2.

7) Now that you have internal information about camera distortion, you can take a pair of stereo pairs and use CameraCalibration.StereoCalibrate () with internal data that was previously calculated to generate external information (offsets and rotations between cameras 1 and 2).

8) Use CvInvoke.cvStereoRectify () and CvInvoke.cvInitUndistortRectifyMap () and then CvInvoke.cvRemap () to create an output image that needs to be lined up in Y so you can perform one of the stereo matching tests.

I found that you need to use Emgu 2.1 ver 806 to get cvStereoRectify to work without an access violation error.

---

I think my questions are:

A) Is my process right? I performed the internal calibration of the camera as a separate process, because since the cameras are 35 cm apart, it is not easy to get a chessboard taking into account both of them in the office and move it a lot ... since it will soon go towards one of the types of cameras. I realized that since the values ​​are integral, they are connected to the camera and therefore must transfer the OK procedure to the stereo. Is it correct?

It seems that the internal values ​​change during the cvStereoRectify process and change completely differently.

eg. Distortion values ​​from the first stage = 0.22, -1,2,0,01, -0,01,2,6 after cvStereoRectify values ​​were changed to = 10, -489, -0,03, -0,09,13208

I'm not an expert, but the first set seems to be more like what I saw from other people's comments, and the second set looks pretty good!

B) Is there a way to stop updating the built-in + distortion values ​​during cvStereoRectify?

C) Does this correspond correctly to the internal values ​​(937,0,290,0,932,249,0,0,1)?

Thank you very much for any advice ... I've been stuck with this for a long time ... and I'm really not sure which part of the process is causing errors. Any advice or suggestions would be greatly appreciated ...

Answer 1

I did not use EMGU after a couple of years, but I used it to calibrate the camera and Kinect.

I found that there might be some confusion about ordering the corners of the rectangle, if the rectangle is completely horizontal - I never understood why, and it just seemed to me useful. Try setting the board so that the rectangle is on a roll of 45 degrees (i.e., from the front you would look more at the shape of the diamond, if that makes sense).

My very common understanding is that this should make it easier for both cameras to ensure that they choose the same angle as the first angle from which to start counting.

I can’t guarantee that this will fix your problem - perhaps it was because I had something deeply wrong in my code, and that’s why it couldn’t achieve regular results when the chessboard was flat. Again, you don’t need much time for testing and may just work.