Robust Reflection Removal with Reflection-free Flash-only Cues (RFC)

Paper | Project Page | Data | Other datasets To be released: Video

Tensorflow implementation for:
Robust Reflection Removal with Reflection-free Flash-only Cues
Chenyang Lei, Qifeng Chen
HKUST

in CVPR 2021

News

2022.03.15 PyTorch version is released

To Do

TL;DR quickstart

To setup a conda environment, test on demo data:

conda env create -f environment.yml
conda activate flashrr-rfc
bash download.sh
python test.py

Setup

Environment

This code is based on tensorflow. It has been tested on Ubuntu 18.04 LTS.

Anaconda is recommended: Ubuntu 18.04 | Ubuntu 16.04

After installing Anaconda, you can setup the environment simply by

conda env create -f environment.yml

Download checkpoint and VGG model

You can download the ckpt and VGG model by

bash download.sh

Quick inference

You can get the results for the demo data by:

python test.py

If you prepare your own dataset, note that each data sample must contains an ambient image and a flash-only iamge:

python test.py --testset /path/to/your/testset

Training

Reproduce our results

First, download the dataset:

bash download_data.sh

Then, you can train a model by

python train.py --model YOUR_MODEL_NAME

Raw data preprocessing

First, download raw images on OneDrive (70MB for each iamge).

Then,

python rawdata_processing.py

Three rgb images will be saved in ./ dir. You can modify the resolutions by yourself in the code.

What is a RFC (Reflection-free Flash-only Cue)?

The reflection-free cue exploits a flash-only image obtained by subtracting the ambient image from the corresponding flash image in raw data space. The flash-only image is equivalent to an image taken in a dark environment with only a flash on. The reflection disappears in this flash-only image.

Please check our Project Page for detailed explanation.

Citation

If you find our work useful for your research, please consider citing the following papers :)

@InProceedings{Lei_2021_RFC,
     title={Robust Reflection Removal with Reflection-free Flash-only Cues}, 
     author={Chenyang Lei and Qifeng Chen},
     booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
     year = {2021}
}
@misc{lei2022robust,
      title={Robust Reflection Removal with Flash-only Cues in the Wild}, 
      author={Chenyang Lei and Xudong Jiang and Qifeng Chen},
      year={2022},
      eprint={2211.02914},
      archivePrefix={arXiv},
      primaryClass={cs.CV}
}

If you are also interested in the polarization reflection removal, please refer to this work.

If you use the synthetic dataset, please cite these two papers since we use their data to synthesize the images:

Contact

Please contact me if there is any question (Chenyang Lei, [email protected])

License

TBD

ChenyangLEI/flash-reflection-removal

ChenyangLEI

Reviews

Repository Details