There are two relatively simple approaches. The first approach is to draw a ray from a point to "infinity" (in fact, to any point outside the polygon) and calculate how many sides of the polygon the ray intersects. A point inside a polygon if and only if the counter is odd.
The second approach is to go around the polygon in order and for each pair of vertices v i and v i + 1 (wrapping the first vertex if necessary), calculate the value (x - x i ) * (y i + 1 - y i ) - (x i + 1 - x i ) * (y - y i ). If these values have the same sign, the point is inside the polygon. (These quantities are the Z-component of the transverse product of the vectors (v i + 1 - v i ) and (p - v i ). The condition that they all have the same sign coincides with the condition that p is on the same side ( left or right) of each rib.)
Both approaches must deal with the fact that the point is exactly on the edge or on the top. First you need to decide whether you want to count points such as those inside the polygon or not. Then you need to configure the tests accordingly. Keep in mind that small round-off errors can give a false answer anyway. This is just what you have to live with.
Since you have a convex quadrangle, there is another approach. Select any three vertices and calculate the barycentric coordinates of the point and the fourth vertex relative to the triangle formed by the three selected vertices. If the barycentric coordinates of the point are all positive and less and less than the barycentric coordinates of the fourth vertex, then the point is inside the quadrangle.
PS I just found a nice page here that lists quite a few strategies. Some of them are very interesting.
source share