Extract slides from video lectures using OpenCV

Question

I would like to extract all the slides from the video lecture using OpenCV. Here is an example lecture: http://www.youtube.com/watch?v=-hxOpz9c0bY .

What approaches would you recommend? So far I have tried:

• Comparison of changes in the intensity of shades of gray from frame to frame. This can have problems when the object in the foreground moves. For example, in this lecture there is a hand that moves: http://www.youtube.com/watch?v=mNzu42FrlHo#t=07m00s .

• Using SURF functions and performing frame-by-frame comparisons. This approach seems slow.

Does anyone have any other ideas?

Answer 1

Most of this work is most likely already done using a video encoder. You just need to extract the keyframes and check how well the frames between them are compressed.

It should also be fairly easy to distinguish between still images. You can save a lot of time by exploring only keyframes. The slides probably have high contrast, solid shapes, solid background. The lecture hall has blurry shapes and low contrast.

Answer 2

What you need is a detection of scene changes. After that you will have to classify the scenes as a “lecture hall” or “presentation”. As for the hand problem, you can use background subtraction with an adaptive background (just make sure you mask the foreground ... you don't want the foreground to become part of the background).

Answer 3

You can try edge detection and look for a rectangular object - slides (above a certain area threshold). You can also reduce the FP by looking at some text in the rectangle.