Junheum Park, Chul Lee, and Chang-Su Kim

Official PyTorch Code for "Asymmetric Bilateral Motion Estimation for Video Frame Interpolation" [paper]

• PyTorch 1.7
• CUDA 11.0
• CuDNN 8.0.5
• python 3.8

Create conda environment:

    $ conda create -n ABME python=3.8 anaconda
    $ conda activate ABME
    $ pip install opencv-python
    $ conda install pytorch==1.7 torchvision cudatoolkit=11.0 -c pytorch

Download repository:

    $ git clone https://github.com/JunHeum/ABME.git

Download pre-trained model parameters:

    $ unzip ABME_Weights.zip

Check your nvcc version:

    $ nvcc --version

• To install correlation layer, you should match your nvcc version with cudatoolkit version of your conda environment. [nvcc_setting]

Install correlation layer:

    $ cd correlation_package
    $ python setup.py install

Generate an intermediate frame on your pair of frames:

    $ python run.py --first images/im1.png --second images/im3.png --output images/im2.png

1. Download the datasets.
2. Copy the path of the test dataset. (e.g., /hdd/vimeo_interp_test)
3. Parse this path into the --dataset_root argument.
4. (optional) You can ignore the --is_save. But, it yields a slightly different performance than evaluation on saved images.

    $ python test.py --name ABME --is_save --Dataset ucf101 --dataset_root /where/is/your/ucf101_dataset/path
    $ python test.py --name ABME --is_save --Dataset vimeo --dataset_root /where/is/your/vimeo_dataset/path
    $ python test.py --name ABME --is_save --Dataset SNU-FILM-all --dataset_root /where/is/your/FILM_dataset/path
    $ python test.py --name ABME --is_save --Dataset Xiph_HD --dataset_root /where/is/your/Xiph_dataset/path
    $ python test.py --name ABME --is_save --Dataset X4K1000FPS --dataset_root /where/is/your/X4K1000FPS_dataset/path

We provide interpolated frames on test datasets for fast comparison or users with limited GPU memory. Especially, the test on X4K1000FPS requires at least 20GB of GPU memory.

• UCF101
• Vimeo90K
• SNU-FILM
• Xiph
• X4K1000FPS

We plan to share train codes soon!

Please cite the following paper if you feel this repository useful.

    @inproceedings{park2021ABME,
        author    = {Park, Junheum and Lee, Chul and Kim, Chang-Su}, 
        title     = {Asymmetric Bilateral Motion Estimation for Video Frame Interpolation}, 
        booktitle = {International Conference on Computer Vision},
        year      = {2021}
    }

See MIT License