John Fremlin's blog: Better video thumbnails by facing the facts

Waiting for updates: connected

Today, I made a nasty little prototype program that generates thumbnails from videos by looking at the faces in them. I called it fumbnailer.

At the moment, it successfully hooks up ffmpeg to opencv, but nothing works very well. ffmpeg (even the development version) seems to utterly fail to seek in flv streams, even when given an exact byte offset to seek to, which is rather irritating. And opencv doesn't give a confidence score for finding a face, which results in the face detection thumbnail generally not including a good face.

fumbnailer implements a second thumbnail selection algorithm (mest), which simply returns the first of the pair of keyframes with the highest intervening datarate. This is supposed to be a proxy for discovering the scenes with the highest movement.

Yesterday, I submitted a patch for ffmpegthumbnailer, (to allow creating thumbnails the same size as the original video), which Dirk (the maintainer) kindly accepted. In the end however, I didn't base fumbnailer on it though, because of its (for my purposes) overly complicated design — adding a single command line option would require modifying about three files, for example. I wanted to bash something out that would let me quickly get feedback on a few ideas.

It seems that modern C++ with the auto keyword seems quite fun. Here's hoping lambdas make it into g++ soon!

Finding a good thumbnail from a video is a kind of trivial sounding problem that for some reason appeals to me. I'd like to develop the two current approaches until they demonstrate their potential (seem to sporadically make good results on the videos I tried — is this just me?), and try a few more ideas out — maybe quality of focus measures next?

Please suggest something radical!