Improved thumbnail extraction from video

Question

I use FFmpeg to find the middle frame of the h264 video file and extract the jpg thumbnail for use on the streaming portal. This is done automatically for each uploaded video.

Sometimes a frame turns out to be a black frame or simply semantically bad, that is, a background or a blurry picture that is not very well connected with video content.

I wonder if I can use openCV or some other method / library to programmatically find the best thumbnails using face recognition or frame analysis.

Answer 1

I ran into this problem myself and came up with a very crude but simple algorithm to make my sketches more “interesting”. How?

• Create an x-number of thumbnails at different points. For example. 5 sketches
• Use the largest (in bytes) file and discard the rest

Why does it work? Since jpeg files of a monotonous "boring" image, like the entire black screen, are compressed into much smaller files than an image with many objects and colors.

This is not ideal, but is a viable 80/20 solution. (Solves 80% of the problem with 20% of the work.) The coding of what actually analyzes the image will be much larger.

Answer 2

If someone needs two liners (using ffmpeg and imagemagick):

(he selects a maximum of 20 frames from the video and uses gt (scene) to select transition points. He uses ffmpeg to create 120px wide png files and then imagemagick to create gif (because ffmpeg gifs are ugly as you know ..) It may fail if nothing happens in the film, but then you should not call it a film - right?

ffmpeg -i $1 -loglevel error -vf "select=gt(scene\,0.1), scale=120:-1" -frames:v 20 -f image2 -vsync 0 -an ./tmp/img%05d.png convert -delay 25 -loop 0 ./tmp/img*.png thumb.gif 

Answer 3

Libavfilter has a thumbnail filter that is designed to select the most representative frame from a series of frames. Not sure how this works, but heres docs http://ffmpeg.org/libavfilter.html#thumbnail