codeslake/IFAN

IFAN: Iterative Filter Adaptive Network for Single Image Defocus Deblurring
_{Official PyTorch Implementation of the CVPR 2021 Paper
Project | Paper | arXiv | Supp | Poster | Slide}

This repo contains training and evaluation code for the following paper:

Iterative Filter Adaptive Network for Single Image Defocus Deblurring
Junyong Lee, Hyeongseok Son, Jaesung Rim, Sunghyun Cho, and Seungyong Lee
POSTECH
IEEE Computer Vision and Pattern Recognition (CVPR) 2021

Getting Started

Prerequisites

Tested environment

1. Environment setup

• Option 1. install from scratch

  $ git clone https://github.com/codeslake/IFAN.git
  $ cd IFAN
  
  ## for CUDA10.2
  $ conda create -y --name IFAN python=3.6 && conda activate IFAN
  $ sh install_CUDA10.2.sh
  
  ## for CUDA11.1 or CUDA11.3
  $ conda create -y --name IFAN python=3.8 && conda activate IFAN
  # CUDA11.1
  $ sh install_CUDA11.1.sh
  
  # CUDA11.3 (for amp)
  $ sh install_CUDA11.3.sh

• Option 2. docker

  $ nvidia-docker run --privileged --gpus=all -it --name IFAN --rm codeslake/IFAN:CVPR2021 /bin/zsh
  $ git clone https://github.com/codeslake/IFAN.git
  $ cd IFAN
  
  # for CUDA10.2
  $ coda activate IFAN_CUDA10.2
  
  # for CUDA11.1
  $ coda activate IFAN_CUDA11.1
  
  # for CUDA11.3 (for amp)
  $ coda activate IFAN_CUDA11.3

2. Datasets

Download and unzip datasets under [DATASET_ROOT]:

• DPDD dataset: Google Drive | Dropbox
• PixelDP test set: Google Drive | Dropbox
• CUHK test set: Google Drive | Dropbox
• RealDOF test set: Google Drive | Dropbox

[DATASET_ROOT]
 ├── DPDD
 ├── PixelDP
 ├── CUHK
 └── RealDOF

[DATASET_ROOT] can be modified with config.data_offset in ./configs/config.py.

3. Pre-trained models

Download and unzip pretrained weights (Google Drive | Dropbox) under ./ckpt/:

.
├── ...
├── ./ckpt
│   ├── IFAN.pytorch
│   ├── ...
│   └── IFAN_dual.pytorch
└── ...

Testing models of CVPR 2021

Important Note: There is a mistake in the IAC layer. The current code uses isotropic kernels in which the two separable kernels are exactly the same (per iteration). However, the current IFAN still works, as defocus blur is usually in the isotropic shape, which the current IAC layer can handle with the isotropic kernels.

If you are to train the code from scratch, please update the IAC layer to use kernel1 and kernel2 as commented in the code. I will update the experimental results based on the corrected IAC layer.

## Table 2 in the main paper
# Our final model used for comparison
CUDA_VISIBLE_DEVICES=0 python run.py --mode IFAN --network IFAN --config config_IFAN --data DPDD --ckpt_abs_name ckpt/IFAN.pytorch --data_offset /data_offset --output_offset ./output

## Table 4 in the main paper
# Our final model with N=8
CUDA_VISIBLE_DEVICES=0 python run.py --mode IFAN_8 --network IFAN --config config_IFAN_8 --data DPDD --ckpt_abs_name ckpt/IFAN_8.pytorch --data_offset /data_offset --output_offset ./output

# Our final model with N=26
CUDA_VISIBLE_DEVICES=0 python run.py --mode IFAN_26 --network IFAN --config config_IFAN_26 --data DPDD --ckpt_abs_name ckpt/IFAN_26.pytorch --data_offset /data_offset --output_offset ./output

# Our final model with N=35
CUDA_VISIBLE_DEVICES=0 python run.py --mode IFAN_35 --network IFAN --config config_IFAN_35 --data DPDD --ckpt_abs_name ckpt/IFAN_35.pytorch --data_offset /data_offset --output_offset ./output

# Our final model with N=44
CUDA_VISIBLE_DEVICES=0 python run.py --mode IFAN_44 --network IFAN --config config_IFAN_44 --data DPDD --ckpt_abs_name ckpt/IFAN_44.pytorch --data_offset /data_offset --output_offset ./output

## Table 1 in the supplementary material
# Our model trained with 16 bit images
CUDA_VISIBLE_DEVICES=0 python run.py --mode IFAN_16bit --network IFAN --config config_IFAN_16bit --data DPDD --ckpt_abs_name ckpt/IFAN_16bit.pytorch --data_offset /data_offset --output_offset ./output

## Table 2 in the supplementary material
# Our model taking dual-pixel stereo images as an input
CUDA_VISIBLE_DEVICES=0 python run.py --mode IFAN_dual --network IFAN_dual --config config_IFAN --data DPDD --ckpt_abs_name ckpt/IFAN_dual.pytorch --data_offset /data_offset --output_offset ./output

Testing results will be saved in [LOG_ROOT]/IFAN_CVPR2021/[mode]/result/quanti_quali/[mode]_[epoch]/[data]/.

[LOG_ROOT] can be modified with config.log_offset in ./configs/config.py.

Options

• --data: The name of a dataset to evaluate. DPDD | RealDOF | CUHK | PixelDP | random. Default: DPDD
  • The folder structure can be modified in the function set_eval_path(..) in ./configs/config.py.
  • random is for testing models with any images, which should be placed as [DATASET_ROOT]/random/*.[jpg|png].

Wiki

• Logging
• Training and testing details.

Contact

Open an issue for any inquiries. You may also have contact with [email protected]

License

This software is being made available under the terms in the LICENSE file. Any exemptions to these terms require a license from the Pohang University of Science and Technology.

Citation

If you find this code useful, please consider citing:

@InProceedings{Lee2021IFAN,
    author    = {Junyong Lee and Hyeongseok Son and Jaesung Rim and Sunghyun Cho and Seungyong Lee},
    title     = {Iterative Filter Adaptive Network for Single Image Defocus Deblurring},
    booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
    year      = {2021}
}