Video Super Resolution
A collection of state-of-the-art video or single-image super-resolution architectures, reimplemented in tensorflow.
Project uploaded to PyPI now. Try install from PyPI:
pip install VSR
Pretrained weights is uploading now.
Several referenced PyTorch implementations are also included now.
Quick Link:
Network list and reference (Updating)
The hyperlink directs to paper site, follows the official codes if the authors open sources.
All these models are implemented in ONE framework.
Model | Published | Code* | VSR (TF)** | VSR (Torch) | Keywords | Pretrained |
---|---|---|---|---|---|---|
SRCNN | ECCV14 | -, Keras | Y | Y | Kaiming | β |
RAISR | arXiv | - | - | - | Google, Pixel 3 | |
ESPCN | CVPR16 | -, Keras | Y | Y | Real time | β |
VDSR | CVPR16 | - | Y | Y | Deep, Residual | β |
DRCN | CVPR16 | - | Y | Y | Recurrent | |
DRRN | CVPR17 | Caffe, PyTorch | Y | Y | Recurrent | |
LapSRN | CVPR17 | Matlab | Y | - | Huber loss | |
EDSR | CVPR17 | - | Y | Y | NTIRE17 Champion | β |
SRGAN | CVPR17 | - | Y | - | 1st proposed GAN | |
VESPCN | CVPR17 | - | Y | Y | VideoSR | β |
MemNet | ICCV17 | Caffe | Y | - | ||
SRDenseNet | ICCV17 | -, PyTorch | Y | - | Dense | β |
SPMC | ICCV17 | Tensorflow | T | Y | VideoSR | |
DnCNN | TIP17 | Matlab | Y | Y | Denoise | β |
DCSCN | arXiv | Tensorflow | Y | - | ||
IDN | CVPR18 | Caffe | Y | - | Fast | β |
RDN | CVPR18 | Torch | Y | - | Deep, BI-BD-DN | |
SRMD | CVPR18 | Matlab | - | Y | Denoise/Deblur/SR | β |
DBPN | CVPR18 | PyTorch | Y | Y | NTIRE18 Champion | β |
ZSSR | CVPR18 | Tensorflow | - | - | Zero-shot | |
FRVSR | CVPR18 | T | Y | VideoSR | β | |
DUF | CVPR18 | Tensorflow | T | - | VideoSR | |
CARN | ECCV18 | PyTorch | Y | Y | Fast | β |
RCAN | ECCV18 | PyTorch | Y | Y | Deep, BI-BD-DN | |
MSRN | ECCV18 | PyTorch | Y | Y | β | |
SRFeat | ECCV18 | Tensorflow | Y | Y | GAN | |
NLRN | NIPS18 | Tensorflow | T | - | Non-local, Recurrent | |
SRCliqueNet | NIPS18 | - | - | - | Wavelet | |
FFDNet | TIP18 | Matlab | Y | Y | Conditional denoise | |
CBDNet | CVPR19 | Matlab | T | - | Blind-denoise | |
SOFVSR | ACCV18 | PyTorch | - | Y | VideoSR | β |
ESRGAN | ECCVW18 | PyTorch | - | Y | 1st place PIRM 2018 | β |
TecoGAN | arXiv | Tensorflow | - | T | VideoSR GAN | β |
RBPN | CVPR19 | PyTorch | - | Y | VideoSR | β |
DPSR | CVPR19 | Pytorch | - | - | ||
SRFBN | CVPR19 | Pytorch | - | - | ||
SRNTT | CVPR19 | Tensorflow | - | - | Adobe | |
SAN | CVPR19 | empty | - | - | AliDAMO SOTA | |
AdaFM | CVPR19 | Pytorch | - | - | SenseTime Oral |
*The 1st repo is by paper author.
**Y: included; -: not included; T: under-testing.
You can download pre-trained weights through prepare_data
, or visit the hyperlink at β.
Link of datasets
(please contact me if any of links offend you or any one disabled)
Name | Usage | # | Site | Comments |
---|---|---|---|---|
SET5 | Test | 5 | download | jbhuang0604 |
SET14 | Test | 14 | download | jbhuang0604 |
SunHay80 | Test | 80 | download | jbhuang0604 |
Urban100 | Test | 100 | download | jbhuang0604 |
VID4 | Test | 4 | download | 4 videos |
BSD100 | Train | 300 | download | jbhuang0604 |
BSD300 | Train/Val | 300 | download | - |
BSD500 | Train/Val | 500 | download | - |
91-Image | Train | 91 | download | Yang |
DIV2K | Train/Val | 900 | website | NTIRE17 |
Waterloo | Train | 4741 | website | - |
MCL-V | Train | 12 | website | 12 videos |
GOPRO | Train/Val | 33 | website | 33 videos, deblur |
CelebA | Train | 202599 | website | Human faces |
Sintel | Train/Val | 35 | website | Optical flow |
FlyingChairs | Train | 22872 | website | Optical flow |
DND | Test | 50 | website | Real noisy photos |
RENOIR | Train | 120 | website | Real noisy photos |
NC | Test | 60 | website | Noisy photos |
SIDD(M) | Train/Val | 200 | website | NTIRE 2019 Real Denoise |
RSR | Train/Val | 80 | download | NTIRE 2019 Real SR |
Vimeo-90k | Train/Test | 89800 | website | 90k HQ videos |
Other open datasets: Kaggle ImageNet COCO
VSR package
This package offers a training and data processing framework based on TF. What I made is a simple, easy-to-use framework without lots of encapulations and abstractions. Moreover, VSR can handle raw NV12/YUV as well as a sequence of images as inputs.
Install
-
Prepare proper tensorflow and pytorch(optional). For example, GPU and CUDA10.0 (recommend to use
conda
):conda install tensorflow-gpu==1.15.0 # optional # conda install pytorch
-
Install VSR package
# For someone see this doc online # git clone https://github.com/loseall/VideoSuperResolution && cd VideoSuperResolution pip install -e .
Getting Started
-
Download pre-trained weights and (optinal) training datasets. For instance, let's begin with VESPCN and vid4 test data:
python prepare_data.py --filter vespcn vid4
-
Customize backend cd ~/.vsr/ touch config.yml
backend: tensorflow # (tensorflow, pytorch) verbose: info # (debug, info, warning, error)
-
Evaluate
cd Train python eval.py srcnn -t vid4 --pretrain=/path/srcnn.pth
-
Train
python prepare_data.py --filter mcl-v cd Train python train.py vespcn --dataset mcl-v --memory_limit 1GB --epochs 100
OK, that's all you need. For more details, use --help
to get more information.
More documents can be found at Docs.