You need two direction vectors. One is along the line AB given
Vector3D e = Normalize(BA)
and one to determine the upward direction for the cross section. This can be specified or it can be calculated using the following algorithm (with preference to + y)
if( eX != 0 || eZ != 0 ) { // choose direction perpendicular to line closest to +y direction Vector3D n = [-eX*eY, eX*e.X+eZ*eZ, -eZ*eY]; } else { // if line along +y already then choose +z for up vector Vector3D n = [ 0, 0, 1]; }
Now you can calculate the 3rd direction to form a coordinate system
Vector3D k = Normalize( Cross(e,n) )
And you collect a 3 × 3 rotation matrix that converts local coordinates to world coordinates using
| kX nX eX | R = | kY nY eY | | kZ nZ eZ |
The local coordinate has a direction along the line as + z such that
Point3D a = A + R*[w,h,0] Point3D b = A + R*[-w,h,0] Point3D c = A + R*[w,-h,0] Point3D d = A + R*[-w,-h,0] Point3D e = B + R*[w,h,0] Point3D f = B + R*[-w,h,0] Point3D g = B + R*[w,-h,0] Point3D h = B + R*[-w,-h,0]
where R*[x,y,z] denotes the multiplication of the matrix vector, w and h are the width and height of the rectangular section, and A , B are the position vectors of point A and B. The addition between the vectors is element by element.
I checked the code in my own code and it works. vec3 is an alias for a 3D vector, mat3 is an alias for a 3 × 3 matrix.

vec3 u=(BA).Normalized(); vec3 n = vec3.O; if(Math.Abs(uX)<=Math.Abs(uY)&&Math.Abs(uX)<=Math.Abs(uZ)) { n=new vec3(uY*u.Y+uZ*uZ, -uY*uX, -uZ*uX); } else if(Math.Abs(uY)<=Math.Abs(uX)&&Math.Abs(uY)<=Math.Abs(uZ)) { n=new vec3(-uX*uY, uX*u.X+uZ*uZ, -uZ*uY); } else if(Math.Abs(uZ)<=Math.Abs(uX)&&Math.Abs(uZ)<=Math.Abs(uY)) { n=new vec3(-uX*uZ, -uY*uZ, uX*u.X+uY*uY); } vec3 v=n.Cross(u); mat3 R=mat3.Combine(v, n, u); var a=A+R*new vec3(wt, ht, 0); var b=A+R*new vec3(-wt, ht, 0); var c=A+R*new vec3(wt, -ht, 0); var d=A+R*new vec3(-wt, -ht, 0); var e=B+R*new vec3(wt, ht, 0); var f=B+R*new vec3(-wt, ht, 0); var g=B+R*new vec3(wt, -ht, 0); var h=B+R*new vec3(-wt, -ht, 0);