OpenKinect Maturity

Question

I am interested in writing homebrew code for the Microsoft Kinect console. I have several applications that I think will translate well to the platform. I played with the idea of ​​doing this using OpenKinect drivers and libraries . Obviously, that would be a lot of work, but I wonder how much. Does anyone have experience with OpenKinect? Do you only receive raw video / audio data from the device, or did someone write higher-level abstractions to simplify common tasks?

Answer 1

The OpenKinect library is basically a driver — at least for now — so it doesn't expect significant features from it . You will more or less receive raw data both from depth and from cameras.
This is basically an array received in the callback function every time a frame arrives.

You can try following the instructions on the OpenKinect website, it is really quick to install and try, and you can play a bit with glview to get an idea of ​​what is possible. I created some demos using opencv and got pretty nice results, although I didn’t have much background in computer vision, so I can only suggest you try it yourself!

Alternatively, if you are looking for more advanced features, the OpenNI infrastructure was released this week and contains some impressive high-level algorithms, such as skeleton tracking and multiple gesture recognition . Part of the structure is PrimeSense's own algorithms (for example, a powerful skeleton tracking module ...). I haven’t tried yet and don’t know how well it integrates with Kinect and various OSs, but with a bunch of guys from different groups (OpenKinect, Willow Garage ...) they are working hard on this, which shouldn’t be "This will be a problem for a week.

Answer 2

Further developing what Jules Ollen wrote, I worked with OpenNI ( http://www.openni.org ) and its algorithms (NITE), and I strongly recommend using this framework. Both structures are well documented and come with numerous samples from which you can start.

Basically, OpenNI abstracts the details of the lower level of working with the sensor and its driver for you and gives you a convenient way to get what you want from the "generator" (for example, xn :: DepthGenerator for receiving depth data). OpenNI is open source and can be used in any application. OpenNI also handles platform abstraction for you. OpenNI is currently supported and works fine for Windows 32/64 and Linux and is currently porting to OSX. Bindings are available for use in several programming languages ​​(C, C ++, .NET, Python, and some others, which I suppose).

NITE has additional interfaces built above OpenNI that give you higher level results (for example, track manual point, skeletons, scene analysis, etc.). You want to test the nuances of the NITE license as to when / where you can use it, but this is probably the easiest and fastest way to get an analysis (like a skeleton) at the moment. NITE is a closed source, so PrimeSense needs to provide a binary version for use. Currently available versions of Windows and Linux.

Answer 3

I have not worked with OpenKinect, but for several months now I have been working with OpenNI and SensorKinect for my research. If you plan to work with raw Kinect data, it works great with giving you depth and video (they do not support engine control). I used it with C ++ and OpenGL on both Windows 64bit and Ubuntu 32bit without any code changes. It is very easy to find out if you know basic C ++. Installing it may be a slight headache.

For more advanced features such as skeleton detection, gesture recognition, etc., I highly recommend using middlewares such as NITE with OpenNI or those listed here: Middlewares developed around OpenNI , rather than reinventing the wheel. Nite is also very easy to use when you are running OpenNI; for example, joint recognition is about 10-20 additional lines of code.

Something that I would recommend to my young "I" is to learn and work with the main game engine (for example, Unity), and not directly with OpenGL. This would give you much better and more pleasant graphics, without any unnecessary hassle, and also make it easy to integrate your program with other tools such as PhysX . I have not tried, but I know that there are some plugins for using Kinect drivers in Unity.