pytorch-deep-video-prior (DVP)
Official PyTorch implementation for NeurIPS 2020 paper: Blind Video Temporal Consistency via Deep Video Prior
TensorFlow implementation | paper | project website
Introduction
Our method is a general framework to improve the temporal consistency of video processed by image algorithms.
For example, combining single image colorization or single image dehazing algorithm with our framework, we can achieve the goal of video colorization or video dehazing.
Dependency
Environment
This code is based on PyTorch. It has been tested on Ubuntu 18.04 LTS.
Anaconda is recommended: Ubuntu 18.04 | Ubuntu 16.04
After installing Anaconda, you can setup the environment simply by
conda env create -f environment.yml
Inference
Demo
bash test.sh
The results will be saved in ./result
Use your own data
For the video with unimodal inconsistency:
python main_IRT.py --max_epoch 25 --input PATH_TO_YOUR_INPUT_FOLDER --processed PATH_TO_YOUR_PROCESSED_FOLDER --model NAME_OF_YOUR_MODEL --with_IRT 0 --IRT_initialization 0 --output ./result/OWN_DATA
For the video with multimodal inconsistency:
python main_IRT.py --max_epoch 25 --input PATH_TO_YOUR_INPUT_FOLDER --processed PATH_TO_YOUR_PROCESSED_FOLDER --model NAME_OF_YOUR_MODEL --with_IRT 1 --IRT_initialization 1 --output ./result/OWN_DATA
Citation
If you find this work useful for your research, please cite:
@inproceedings{lei2020dvp,
title={Blind Video Temporal Consistency via Deep Video Prior},
author={Lei, Chenyang and Xing, Yazhou and Chen, Qifeng},
booktitle={Advances in Neural Information Processing Systems},
year={2020}
}
@article{lei2022deep,
title={Deep video prior for video consistency and propagation},
author={Lei, Chenyang and Xing, Yazhou and Ouyang, Hao and Chen, Qifeng},
journal={IEEE Transactions on Pattern Analysis and Machine Intelligence},
volume={45},
number={1},
pages={356--371},
year={2022},
publisher={IEEE}
}
Contact
Feel free to contact me if there is any question. (Yazhou Xing, [email protected])