Finding the fourth point of the quadrangle. (After converting perspectives)

Question

There is an image of a square, and a perspective transformation is applied to this image.

Thus, the perspective image of the square will not be square, but quadrangular. Question:

If I know the three corner points of the perspective image (quad), then what will be the fourth corner point of the quad?

The problem also arises due to the fact that corner points are indicated in the image space, not in the world coordinate system. Thus, they have only two coordinates, for example: N1 = (x1, y1), N2 = (x2, y2), N3 = (x3, y3)

And I need to find N4 = (x4, y4), using the information that the real image in world coordinates was the square before the perspective transformation.

Can this fourth point be found? I think that there will not be only one solution to this problem, but I want to know how many solutions we have, and I want to get these solutions.

I read almost the same problem here, but the author did not say that it was the square before the conversion.

Answer 1

In general, this is not possible. You want to define homography (perspective transformation) between the plane in which the original square lies and the image plane, using knowledge of the three correspondences between the points, but you need four correspondences between the points to uniquely determine the plane homography.

An example of ambiguity: suppose that the original square has its vertices at the points with uniform coordinates [0, 0, 1], [1, 0, 1], [1, 1, 1] and [0, 1, 1]. Now consider two homologies defined by the following matrices:

H1 = [1, -2, 0; 0, -1, 0; 0, -2, 1] H2 = [-1, 2, 0; 0, 1, 0; -2, 2, 1] 

Both transformations leave the first three vertices invariant (remember that two points in a projective space are equal if and only if their vectors differ by a nonzero scale factor), but they transform the fourth vertex to different points.

Answer 2

Suppose we use the Wikipedia projection transformation matrix and assume that you have four points p = {px,py,pz} , p+u , p+v , and p+u+v . Suppose the last one is the one you want to find, and u and v are the edge vectors of the square. After the conversion, the points are displayed on p -> {px/pz, py/pz, 1} , p+u -> {(px+ux)/(pz+uz), (py+uy)/(pz+uz), 1} and similarly for p+v and p+u+v . All this, of course, assumes that neither u nor v are in the zero transformation space (they are not “aligned towards the camera”). Thus, you have 4 new coordinate pairs, which are rational functions of the original coordinates of 3D real space. Your problem is basically how you calculate

 (px+ux+vx)/(pz+uz+vz) and (py+uy+vy)/(pz+uz+vz) 

from

 px/pz, py/pz, (px+ux)/(pz+uz), (py+uy)/(pz+uz), (px+vx)/(pz+vz), and (py+vy)/(pz+vz) 

I don’t think there is a way to do this. In addition, this page seems to indicate that any quadrangle can be matched to any other quadrangle using a perspective transform, so it is likely that your problem is -posed.

Answer 3

If all you know is the coordinates of the image after projection, then I don’t think you can find the coordinates of the fourth point. If you know the side of the square (for example, 5 m in three-dimensional space) and the image coordinates wrt the center of the projection (and not just the x, y pixel values, as well as the distance of the image plane from the center of the projection), then you can use any of the methods described in Haralick et al. to find the coordinates of the fourth point.

Answer 4

But this problem has some solutions, I heard that there are several ways to find the four possible fourth corners of the quadrangle. So maybe there are not many solutions to this problem? And what additional information will be enough to find not so many solutions to this problem? Also now I'm reading about calibrating the camera, but I think I will need to calibrate the camera’s runtime to solve the problem. Therefore, even if I remember the calibration parameters and make a program that automatically calibrates the camera, what if the state of the camera changes and the calibration parameters also change? In this situation, the parameters of the old camera will be incorrect.