codeslake/DMENet

DMENet: Deep Defocus Map Estimation Network
_{Official Implementation of the CVPR 2021 Paper
Project | Paper | Supp | Poster}

This repository contains the official matlab implementation of SYNDOF generation used in the following paper:

Deep Defocus Map Estimation using Domain Adaptation
Junyong Lee¹, Sungkil Lee², Sunghyun Cho³, and Seungyong Lee^1
1POSTECH, ²Sungkyunkwan University, ³DGIST
IEEE Computer Vision and Pattern Recognition (CVPR) 2019

Getting Started

Prerequisites

Tested environment

1. Setup environment

   • Option 1. install from scratch

     $ git clone https://github.com/codeslake/DMENet.git
     $ cd DMENet
     
     # for CUDA10
     $ conda create -y --name DMENet python=3.6 && conda activate DMENet
     $ sh install_CUDA10.0.sh
     
     # for CUDA11 (the name of conda environment matters)
     $ conda create -y --name DMENet_CUDA11 python=3.6 && conda activate DMENet_CUDA11
     $ sh install_CUDA11.1.sh

   • Option 2. docker

     $ nvidia-docker run --privileged --gpus=all -it --name DMENet --rm codeslake/dmenet:CVPR2019 /bin/zsh
     $ git clone https://github.com/codeslake/DMENet.git
     $ cd DMENet
     
     # for CUDA10
     $ conda activate DMENet
     
     # for CUDA11
     $ conda activate DMENet_CUDA11

2. Download and unzip datasets(Google Drive | Dropbox | OneDrive) under [DATASET_ROOT].

   [DATASET_ROOT]
    ├── train
    │   ├── SYNDOF
    │   ├── CUHK
    │   └── Flickr
    └── test
        ├── CUHK
        ├── RTF
        └── SYNDOF
   

   Note:

   • [DATASET_ROOT] is currently set to ./datasets/. It can be specified by modifying config.data_offset in ./config.py.

3. Download pretrained weights of DMENet (Google Drive | Dropbox) and unzip it as in [LOG_ROOT]/DMENet_CVPR2019/DMENet_BDCS/checkpoint/DMENet_BDCS.npz ([LOG_ROOT] is currently set to ./logs/).

4. Download pretrained VGG19 weights (Google Drive | Dropbox) and unzip as in pretrained/vgg19.npy (for training only).

Logs

• Training and testing logs will be saved under [LOG_ROOT]/DMENet_CVPR2019/[mode]/:

  [LOG_ROOT]
   └──DMENet_CVPR2019
      ├── [mode]
      │   ├── checkpoint      # model checkpoint
      │   ├── log             # scalar/image log for tensorboard
      │   ├── sample          # sample images of training
      │   └── result          # resulting images of evaluation
      └── ...
  

  [LOG_ROOT] can be modified with config.root_offset in ./config.py.

Testing final model of CVPR 2019

Please note that due to the server issue, the checkpoint used for the paper is lost.
The provided checkpoint is the new checkpoint that shows the closest evaluation results as in the paper.

Check out updated performance with the new checkpoint.

• Test the final model by:

  python main.py --mode DMENet_BDCS --test_set CUHK

  Testing results will be saved in [LOG_ROOT]/DMENet_CVPR2019/[mode]/result/[test_set]/:

  ...
  [LOG_ROOT]/DMENet_CVPR2019/[mode]/result/
   └── [test_set]
       ├── image                     # input defocused images
       ├── defocus_map               # defocus images (network's direct output in range [0, 1])
       ├── defocus_map_min_max_norm  # min-max normalized defocus images in range [0, 1] for visualization
       └── sigma_map_7_norm          # sigma maps containing normalized standard deviations (in range [0, 1]) for a Gaussian kernel. For the actual standard deviation value, one should multiply 7 to this map.
  

  Quantitative results are computed from matlab. (e.g., evaluation on the RTF dataset).

  • Options
    • --mode: The name of a model to test. The logging folder named with the [mode] will be created as [LOG_ROOT]/DMENet_CVPR2019/[mode]/. Default: DMENet_BDCS
    • --test_set: The name of a dataset to evaluate. CUHK | RTF0 | RTF1 | RTF1_6 | random. Default: CUHK
      • The folder structure can be modified in the function get_eval_path(..) in ./config.py.
      • random is for testing models with any images, which should be placed as [DATASET_ROOT]/test/random/*.[jpg|png].

• Check out the evaluation code for the RTF dataset, and the deconvolution code.

Training & testing the network

• Train the network by:

  python main.py --is_train --mode [mode]

  Note:

  • If you train DMENet with newly generated SYNDOF dataset from this repo, comment this line and uncomment this line before the training.

• Test the network by:

  python main.py --mode [mode] --test_set [test_set]

  • arguments
    • --mode: The name of a model to train. The logging folder named with the [mode] will be created as [LOG_ROOT]/DMENet_CVPR2019/[mode]/. Default: DMENet_BDCS
    • --is_pretrain: Pretrain the network with the MSE loss (True | False). Default: False
    • --delete_log: Deletes [LOG_ROOT]/DMENet_CVPR2019/[mode]/* before training begins (True | False). Default: False

Contact

Open an issue for any inquiries. You may also have contact with [email protected]

Related Links

• CVPR 2021: Iterative Filter Adaptive Network for Single Image Defocus Deblurring [paper][code]
• ICCV 2021: Single Image Defocus Deblurring Using Kernel-Sharing Parallel Atrous Convolutions [paper][code]
• SYNDOF dataset generation repo [link]

License

This software is being made available under the terms in the LICENSE file. Any exemptions to these terms require a license from the Pohang University of Science and Technology.

Citation

If you find this code useful, please consider citing:

@InProceedings{Lee2019DMENet,
    author    = {Junyong Lee and Sungkil Lee and Sunghyun Cho and Seungyong Lee},
    title     = {Deep Defocus Map Estimation Using Domain Adaptation},
    booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
    year      = {2019}
}