Best approach for recognizing 2D gestures in VR?

Question

I’m the developer of a game that uses gesture recognition using the VR headset for HTC Vive, and I’m trying to improve the accuracy of gesture recognition.

(Game for context: http://store.steampowered.com//app/488760 . This is a game in which you cast spells by drawing characters in the air.)

I am currently using a $ 1 algorithm to recognize a two-dimensional gesture and using a spelling camera attached to the horizontal rotation of the player to smooth out the gesture that the player draws in space.

However, I am sure that there should be more effective approaches to the problem!

I should represent the gestures in 2D in the instructions, so ideally I would like to:

Find the optimal vector on which to smooth the gesture. Flatten it in 2D space. Use the best gesture recognition algorithm to recognize what kind of gesture it is.

It would be very helpful to approach 100% accuracy under any circumstances. Currently, for example, the game tends to get confused when players try to draw a circle in the midst of a battle, and it is assumed that they draw a Z shape instead.

All suggestions are welcome. Thank you in advance.

Answer 1

It seems like this plugin called Gesture Recognizer 3.0 can give you a great idea of ​​what step you should take

Gesture Recognizer 3.0

Also, I found this javascript gesture tag in github

Jester

Hope this helps.

Answer 2

Believe me or not, but I found this message two months ago and decided to test my VR / AI skills by preparing a Unity package for recognizing magic gestures in VR. Now I'm back with a full VR demo: https://ravingbots.itch.io/vr-magic-gestures-ai

The recognition system tracks the gesture vector, and then projects it onto a two-dimensional grid. You can also easily customize the 3D grid if you want the system to work with three-dimensional shapes, but do not forget to provide the correct set of workouts, fixing a large number of shape variations.

Of course, the package is universal, and you can use it for a non-magic application. The code is well documented. The online documentation provided in PDF has 1000+ pages: https://files.ravingbots.com/docs/vr-magic-gestures-ai/

The package has been tested with HTC Vive. Gradually, support for Gear VR and other VR devices is added.

Answer 3

I personally recommend AirSig. It covers more features, such as authentication using controllers. The Vive version and the Oculus version are free. "It would be very useful to approach 100% accuracy under any circumstances." My experience is that its built-in gestures have an accuracy of more than 90%, a signature of more than 96%. Hope this meets your requirements.