Calculation of the direction of the camera's rays per pixel of the 3D world

Question

I want to calculate the directions of the rays from the camera to the positions of the pixels (in world coordinates) with a given height, as described in this article . The camera image size is 640,480. I calibrated the internal parameters of the camera and distorted each image. I measured the physical distance between the camera and the background plane (29 cm) as described here:

1 cm translates into 25 pixels (assuming quadratic pixels). My first approach was to calculate based on the position of the pixel and camera as follows:

float3 pixPos = (float3)(u, v, z); float3 camPos = (float3)(height/2, width/2, 29*25); float3 ray = normalize(pixPos-camPos); 

where u, v are the coordinates of the image from 0.0 to height, width and z are my (already estimated) height values. This does not seem to be the right way. I have already done a search on SO and found this answer , but the solution described there does not contain pixel heights (here).

Answer 1

I solved the same problem several years ago when I was writing my thesis. Here is a part of it that describes how to create a camera for ray tracing.

First of all, you need to determine the camera coordinate system. This is an orthonormal coordinate system, which means that all three basis vectors are perpendicular to each other and have the same size (not necessarily “one”). You also need to indicate if your camera coordinate system is right-handed or left-handed (I'll talk about left-handed people). First you need to define up vector (the vector shows that you have the y axis) and camera direction vector (so the vector is from the position of the eyes to the middle of your projection plane). Then you compute the left vector (or the right vector if you need) (which indicates exactly where the x axis is on your screen) on cross product up vector and camera direction vector . Now, since only the camera direction vector and the left vector perpendicular, you should make one more cross product camera direction vector and left vector (vector y in the image).

So you get a camera coordination system like this

No, you need to determine how large your projection screen is in world coordinates. It can be difficult sometimes, so you can also define it at two angles ( phi and theta ) and the distance from the position of the eyes (lets call it d ).

You'll get and . (the vector x is the vector that defines the x axis on the screen, so it is a left vector or a right vector, depends on manualness) Using a linear combination of these two vectors u and v you can calculate any position on the projection screen. Odds alpha and although they represent the point distance from the middle of the projection screen.

So and , where s and r are the x and y coordinates on your computed image, and imageWidth and imageHeight are the corresponding sizes.

So, as you can see in this image

The final position of any point on the design plane .

Then computing the requested vector .