The demo from your link does not seem to use real gesture recognition. It simply distinguishes between two different positions of the hands (open / closed), which is much simpler, and tracks the position of the hand. Given how he holds his hands in a demonstration (in front of the body, in front of the kintset when they are open), perhaps this is what he does. Since you did not specify which language you are using, I will use the C function names in openCV, but they should be similar in other languages. I also suggest that you can get the depth map from kinect (possibly using the callback function if you use libfreenect).
Depth threshold to select only close points (hands). You can achieve this either on your own or directly using openCV to get the binary image (cvThreshold () with CV_THRESH_BINARY). Display the image that you get after the threshold value and adjust the threshold value according to your configuration (try to avoid being too close to kinect as there is more interference in this area).
Get hand contour with cvFindContour ()
This is the basis. Now that you have the contours of the hands, depending on what you want to do, you can take different directions. If you just want to detect between open and closed hands, you can probably do:
Get the convex hull of the hands using cvConvexHull2 ()
Get bulge defects using cvConvexityDefect () on the contours and convex hull you got before.
Examine the bulge defects: if there are large defects, the hand is open (because the shape is concave between the fingers), if the hand is not closed.
But you can also find fingers! This is what I did last week, it does not require much effort and is likely to increase your demo! A cheap but fairly reliable way to do this:
Zoom in on the contours of a hand using a polygon. Use cvApproxPoly () on the loop. You will need to adjust the accuracy parameter so that the polygon is as simple as possible, but it does not mix fingers (about 15 should be nice, but draw it on the image using cvDrawContours () to check what you get)
Examine the contour to find sharp convex corners. You will have to do it manually. This is the hardest part because:
- The data structures used in openCV may get a little confused at first. If you work too much with the CvSeq framework, cvCvtSeqToArray () may help.
- You will finally get some (basic) math to find convex angles. Remember that you can use a point product to determine how sharp an angle is, and a vector product to distinguish between convex and concave angles.
Here you are, sharp convex corners - your fingertips!
This is a simple finger detection algorithm, but there are many ways to enhance it. For example, you can try to apply a median filter on the depth map to โsmooth outโ everything a little or try to use a more accurate approximation of the polygon, but then filter the contour to combine the points that should be closed at your fingertips, etc.
Good luck and have fun!
Jules ollรฉon
source share