HECATE
Hecate [hek-uh-tee] is a video processing library that auto-magically generates thumbnails, animated GIFs, and video summaries from videos. This library is developed and maintained by Yahoo Research, New York.
The source code is Copyright 2016 Yahoo Inc. and is licensed under the terms of the Apache 2.0 License. See the LICENSE in the project root file for terms.
The technology behind this library is based on our research work. If you find this library useful in your work, we ask you to cite our research paper:
"To Click or Not To Click: Automatic Selection of Beautiful Thumbnails from Videos."
Yale Song, Miriam Redi, Jordi Vallmitjana, Alejandro Jaimes,
Proceedings of the 25th ACM International on Conference on Information and Knowledge Management, CIKM 2016
Installation
Hecate has one dependency: OpenCV library with an FFMPEG support. You will need to install the library properly before trying out Hecate!
Once you install the dependenct library correctly, follow the instruction below:
$ git clone https://github.com/yahoo/hecate.git
$ cd hecate
$ vim Makefile.config
- Set INCLUDE_DIRS and LIBRARY_DIRS to where your
opencv library is installed. Usually under /usr/local.
- If your OpenCV version is 2.4.x, comment out the line
OPENCV_VERSION := 3
- Save and exit
$ make all
$ make distribute
Once you've successfully compiled hecate, it will generate a binary executable under distribute/bin/. Run the following command to check if everything works properly:
$ ./distribute/bin/hecate
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
HECATE Copyright 2016 Yahoo Inc.
Licensed under the terms of the Apache 2.0 License.
Developed by : Yale Song ([email protected])
Built on : 11:46:03 Aug 11 2016
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
USAGE: hecate -i infile [options]
-i --in_video (string) Input video file
-o --out_dir (string) Output directory (./output)
-s --step (int) Frame subsampling step size (1)
-n --njpg (int) Number of thumbnails to be generated (5)
-q --ngif (int) Number of GIFs to be generated (5)
-r --lmov (int) Length of video summary to be generated (in seconds) (15)
-u --jpg_width_px (int) Pixel width of thumbnail images (360)
-v --gif_width_px (int) Pixel width of animated GIFs (360)
-w --mov_width_px (int) Pixel width of summary video (360)
--generate_jpg Generate thumbnail images
--generate_gif Generate animated GIFs
--generate_mov Generate a summary video
--generate_gifsum Generate animated GIFs summary
--generate_gifall Generate all possible animated GIFs
--print_shot_info Print shot boundary detection results
--print_keyfrm_info Print keyframe indices
Congratulations! You have successfully installed hecate!
Get started
In order to get started, we will need a video file to play with. In this example, we will use the video "The Spirit of '43" by Walt Disney from The Internet Archive.
Let's download the video and save it as examples/video.mp4:
$ wget https://archive.org/download/TheSpiritOf43_56/The_Spirit_of__43_512kb.mp4 \
--output-document examples/video.mp4 --no-check-certificate
Hecate provides three main functionalities through a binary executable hecate: Thumbnail extraction, GIF generation, and video summarization. There are various other functionalities the library provides, such as shot boundary detection and keyframe extraction.
We will explain each case below.
Shot boundary detection and keyframe extraction
Shot boundary detection and keyframe extraction are often the first steps towards various video processing methods. With Hecate, obtaining shot and keyframe information is easier than ever! Simply run the following command to get the result:
$ ./distribute/bin/hecate -i examples/video.mp4 --print_shot_info --print_keyfrm_info
Below is the results we obtained on our dev machine (OS X 10.10 with OpenCV v3.1):
shots: [0:81],[84:93],[96:102],[108:270],[272:418],...,[9966:10131],[10135:10164]
keyframes: [52,85,98,128,165,208,242,259,265,273,...,10127,10141]
The units are frame indices (zero-based). You will notice that shot ranges are non-continuous; there are "gaps" between shots, e.g., two frames are missing between the first two shots [0:81] and [84:93]. This is normal and intentional: Hecate discards low-quality frames that aren't ideal in producing nicely looking thumbnails, animated GIFs, and video summaries. We refer to our CIKM 2016 paper for the rational behind our reason to invalidate low-quality frames.
Thumbnail generation
Hecate uses computer vision to determine frames that are most "suitable" as video thumbnails. By suitable, we mean a frame that is the most relevant to the video content and that is the most beautiful in terms of computational aesthetics; technical details are explained in our CIKM 2016 paper.
You can generate thumbnail images using Hecate. Run the following command to generate one thumbnail image from the video.
$ ./distribute/bin/hecate -i examples/video.mp4 --generate_jpg --njpg 1
You will see the generated thumbnail image under the output directory (set as output by default; you can change this using the option --out_dir YOUR_DIRECTORY). On our dev machine we get this thumbnail image:
In the above example, we generated only one thumbnail image. Are you not satisfied with the thumbnail image? Hecate can generate any number of thunbmail images! Let's generate five thumbnail images.
$ ./distribute/bin/hecate -i examples/video.mp4 --generate_jpg --njpg 3
The output files are named <video_filename>_<rank>.jpg. The files are ranked by their quality (rank 0 means it's the best one).
Animated GIF generation
Do you want to create animated GIFs from a video without the hassle of using manual tools? Hecate can automatically create them for you! Run the following command to create one animated GIF from the video.
$ ./distribute/bin/hecate -i examples/video.mp4 --generate_gif --ngif 1
On our dev machine, we get this animated GIF:
You can, of course, create more than just one GIF by setting the paramter --ngif N with an appropriate number N. When there are multiple GIFs, you can also generate a "summary GIF" by concatenating them, using this command:
$ ./distribute/bin/hecate -i examples/video.mp4 --generate_gif --ngif 3 --generate_gifsum
If you'd rather want to obtain all available GIFs from the video, use the following command:
$ ./distribute/bin/hecate -i examples/video.mp4 --generate_gifall
Video summary generation
Last but not least, Hecate can summarize a video! Run the following command to create a video summary of length 15 seconds.
$ ./distribute/bin/hecate -i examples/video.mp4 --generate_mov --lmov 15
We included the video summary generated on our dev machine here: https://github.com/yahoo/hecate/blob/master/examples/video_sum.mp4