What is a computer vision library and algorithm for analyzing human behavior?

Question

Purpose: Detection / determination of human actions, p. collection / removal of items to read the label and hold it on a rack (in the store), sitting, mounting / ascending atoms. sa chair, bench, stairs, etc.

Environment: A store / shop that is mostly well lit. Cameras (VGA → 1MP), fixed (i.e. Not PTZ).

Limitations:

• The presence of known and unknown people.
• Possible rearrangement of objects (items for sale) in the store for a certain period of time.
• Possible changes in lighting over time. For example: the front areas of the store can receive sufficient sunlight during the day, which changes to artificial light at night. In addition, the inclusion of additional light sources is possible during the night.

Question:

• Although I understand that OpenCV is great for face recognition and can be used for face recognition, can it be used to analyze "actions", p. the act of sitting, the act of removing an item from a shelf? If so, what are some of these algorithms, should I delve into?

• Since cameras in stores are mainly at the height of the ceiling, they usually do not have a frontal view of the face, but mostly from top to bottom. I understand that Haar Cascade (PCA) is not quite usable, but other methods are needed sa Defining the geometry of the 3D head. Are there other libraries (other than OpenCV) that need to be used for such tasks? Are there open source libraries for the same?

Answer 1

From time to time, some people come here and ask for help (or better, code) to solve some of the most difficult research problems in computer vision. Problems that have not been resolved by the most famous scientists and scientists. Sometimes they ask for the algorithms they saw in SF films. Then they leave disappointment because OpenCV is not friendly enough.

Now, seriously, if you work at PhD Image Processing, working on some ingenious project, you don't need advice here. And if you do not, the chance to do so is very low.

What you can do with reasonable resources and accuracy is to keep track of the people in the store: use a moving average background subtracter (available in OpenCV) to determine what the empty store looks like, and subtract this background from each frame to see the items that come in and disappear. You can extract them using blob analisys lib. The Kalman filter (or a simpler tracker) helps you keep track of moving objects.

Good luck

Answer 2

This is a rather difficult problem. From my personal conversations with these guys http://www.picar.us/ I will find out again that they have some routines that detect human actions in the video, such as dancing or skateboarding. This material is not included in their open source library, but they can help you if you ask them beautifully.

Answer 3

The problem of recognizing human action is usually handled using a description of word bags and a linear (or non-linear) controlled classification scheme using hundreds of tagged data.