Bad results when undistorted dots use OpenCV in Python

Question

I am having problems with undistorted dots on an image taken with a calibrated camera using Python bindings for OpenCV. Faulty points have completely different coordinates than the original points found in the image.

Here's the breaking call:

undistorted = cv2.undistortPoints(image_points, camera_matrix, distortion_coefficients) 

where image_points is an array with many chessboard corners detected returned by cv2.findChessboardCorners and modified to fit the cv2.undistortPoints size cv2.undistortPoints , and camera_matrix and distortion_coefficients were returned by cv2.calibrateCamera .

camera_matrix and distortion_coefficients seem okay to me, as well as image_points . However, distorted does not seem to be related to image_points . Here's a summary of the values:

 >>> image_points array([[[ 186.95303345, 163.25502014]], [[ 209.54478455, 164.62690735]], [[ 232.26443481, 166.10734558]], ..., [[ 339.03695679, 385.97784424]], [[ 339.20108032, 400.38635254]], [[ 339.13067627, 415.30780029]]], dtype=float32) >>> undistorted array([[[-0.19536583, -0.07900728]], [[-0.16608481, -0.0772614 ]], [[-0.13660771, -0.07537176]], ..., [[ 0.00228534, 0.21044853]], [[ 0.00249786, 0.22910291]], [[ 0.00240568, 0.24841554]]], dtype=float32) >>> camera_matrix array([[ 767.56947802, 0. , 337.27849576], [ 0. , 767.56947802, 224.04766824], [ 0. , 0. , 1. ]]) >>> distortion_coefficients array([[ 0.06993424, -0.32645465, 0. , 0. , -0.04310827]]) 

I work with C reference code and everything is consistent until I make this call. What's wrong?

Answer 1

I think you forgot to specify a new camera matrix when calling undistortPoints . If you look at the documentation for the function , it says that the signature:

 Python: cv.UndistortPoints(src, dst, cameraMatrix, distCoeffs, R=None, P=None) → None 

where dst is the array of points after non-erasure, and "if P is identical or omitted, then it contains the normalized coordinates of the point", which means before projection using the calibration matrix.

The function should do what you expect if you set P to cameraMatrix .