• Stars
    star
    709
  • Rank 63,849 (Top 2 %)
  • Language
  • License
    MIT License
  • Created almost 4 years ago
  • Updated about 2 years ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Low-Light Image and Video Enhancement Using Deep Learning: A Survey

teaser

This repository provides:

1) a unified online platform, LLIE-Platform http://mc.nankai.edu.cn/ll/, that covers many popular deep learning-based LLIE methods, of which the results can be produced through a user-friendly web interface, contains a low-light image and video dataset.

2) a new dataset LLIV-Phone https://drive.google.com/file/d/1QS4FgT5aTQNYy-eHZ_A89rLoZgx_iysR/view?usp=sharing, in which the images and videos are taken by various phones' cameras under diverse illumination conditions and scenes, and

3) collects deep learning-based low-light image and video enhancement methods, datasets, and evaluation metrics.

More content and details can be found in our Survey Paper: Low-Light Image and Video Enhancement Using Deep Learning: A Survey.

我们提供中文翻译版:基于深度学习的低照度图像与视频增强综述.

We provide the comparison results on the real low-light videos taken by different mobile phones’ cameras at YouTube https://www.youtube.com/watch?v=Elo9TkrG5Oo&t=6s.

We will periodically update the content. Welcome to let us know if we miss your work that is published in top-tier Journal or conference. We will add it.

✌Our LLIE-Platform supports the function of download. Please right click and then save the figure.

If you use this dataset or platform, please cite our paper. Please hit the star at the top-right corner. Thanks!

Please hit the star in the repo when you ask for any code. Thanks!

📣News

  1. The survey is accepted by TPAMI.

  2. We newly add the Zero-DCE++ to the LLIE-Platform. Have Fun!

Zero-DCE++: C. Li, C. Guo, and C. C. Loy, Learning to enhance low-light image via zero-reference deep curve estimation, TPAMI, 2021.

🌱Contents

  1. LLIE-Platform
  2. LLIV-Phone Dataset
  3. Methods
  4. Datasets
  5. Metrics
  6. Citation

📋LoLi-Platform

Currently, the LLIE-Platform covers 14 popular deep learning-based LLIE methods including LLNet, LightenNet, Retinex-Net, EnlightenGAN, MBLLEN, KinD, KinD++, TBEFN, DSLR, DRBN, ExCNet, Zero-DCE, Zero-DCE++, and RRDNet, where the results of any inputs can be produced through a user-friendly web interface. Have fun: LLIE-Platform.

📋LLIV-Phone

Overview LLIV-Phone dataset contains 120 videos (45,148 images) taken by 18 different phones' cameras including iPhone 6s, iPhone 7, iPhone7 Plus, iPhone8 Plus, iPhone 11, iPhone 11 Pro, iPhone XS, iPhone XR, iPhone SE, Xiaomi Mi 9, Xiaomi Mi Mix 3, Pixel 3, Pixel 4, Oppo R17, Vivo Nex, LG M322, OnePlus 5T, Huawei Mate 20 Pro under diverse illumination conditions (e.g., weak illumination, underexposure, dark, extremely dark, back-lit, non-uniform light, color light sources, etc.) in the indoor and outdoor scenes.

Anyone can access the LLIV-Phone dataset via

Google Drive: https://drive.google.com/file/d/1QS4FgT5aTQNYy-eHZ_A89rLoZgx_iysR/view?usp=sharing or

Baidu Cloud:https://pan.baidu.com/s/1-8PF3dfbtlHlmk9y5ZKx_w, Password: s0b9)

📋Methods

Overview

Date Publication Title Abbreviation Code Platform
2017 PR LLNet: A deep autoencoder approach to natural low-light image enhancement paper LLNet Code Theano
2018 PRL LightenNet: A convolutional neural network for weakly illuminated image enhancement paper LightenNet Code Caffe & MATLAB
2018 BMVC Deep retinex decomposition for low-light enhancement paper Retinex-Net Code TensorFlow
2018 BMVC MBLLEN: Low-light image/video enhancement using CNNs paper MBLLEN Code TensorFlow
2018 TIP Learning a deep single image contrast enhancer from multi-exposure images paper SCIE Code Caffe & MATLAB
2018 CVPR Learning to see in the dark paper Chen et al. Code TensorFlow
2018 NeurIPS DeepExposure: Learning to expose photos with asynchronously reinforced adversarial learning paper DeepExposure TensorFlow
2019 ICCV Seeing motion in the dark paper Chen et al. Code TensorFlow
2019 ICCV Learning to see moving object in the dark paper Jiang and Zheng Code TensorFlow
2019 CVPR Underexposed photo enhancement using deep illumination estimation paper DeepUPE Code TensorFlow
2019 ACMMM Kindling the darkness: A practical low-light image enhancer paper KinD Code TensorFlow
2019 ACMMM (IJCV) Kindling the darkness: A practical low-light image enhancer paper (Beyond brightening low-light images paper) KinD (KinD++) Code TensorFlow
2019 ACMMM Progressive retinex: Mutually reinforced illumination-noise perception network for low-light image enhancement paper Wang et al. Caffe
2019 TIP Low-light image enhancement via a deep hybrid network paper Ren et al. Caffe
2019(2021) arXiv(TIP) EnlightenGAN: Deep light enhancement without paired supervision paper arxiv EnlightenGAN Code PyTorch
2019 ACMMM Zero-shot restoration of back-lit images using deep internal learning paper ExCNet Code PyTorch
2020 CVPR Zero-reference deep curve estimation for low-light image enhancement paper Zero-DCE Code PyTorch
2020 CVPR From fidelity to perceptual quality: A semi-supervised approach for low-light image enhancement paper DRBN Code PyTorch
2020 ACMMM Fast enhancement for non-uniform illumination images using light-weight CNNs paper Lv et al. TensorFlow
2020 ACMMM Integrating semantic segmentation and retinex model for low light image enhancement paper Fan et al.
2020 CVPR Learning to restore low-light images via decomposition-and-enhancement paper Xu et al. Code PyTorch
2020 AAAI EEMEFN: Low-light image enhancement via edge-enhanced multi-exposure fusion network paper EEMEFN PyTorch
2020 TIP Lightening network for low-light image enhancement paper DLN PyTorch
2020 TMM Luminance-aware pyramid network for low-light image enhancement paper LPNet PyTorch
2020 ECCV Low light video enhancement using synthetic data produced with an intermediate domain mapping paper SIDGAN TensorFlow
2020 TMM TBEFN: A two-branch exposure-fusion network for low-light image enhancement paper TBEFN Code TensorFlow
2020 ICME Zero-shot restoration of underexposed images via robust retinex decomposition paper RRDNet Code PyTorch
2020 TMM DSLR: Deep stacked laplacian restorer for low-light image enhancement paper DSLR Code PyTorch
2021 TPAMI Learning to enhance low-light image via zero-reference deep curve estimation paper Zero-DCE++ Code PyTorch
2021 CVPR Retinex-inspired unrolling with cooperative prior architecture search for low-light image enhancement paper RUAS Code PyTorch
2021 CVPR Learning temporal consistency for low light video enhancement from single images paper Zhang et al. Code PyTorch
2021 CVPR Nighttime visibility enhancement by increasing the dynamic range and suppression of light effects paper Sharma and Tan
2021 TCSVT RetinexDIP: A unified deep framework for low-light image enhancement paper RetinexDIP Code PyTorch
2021 TIP Sparse gradient regularized deep retinex network for robust low-light image enhancement paper Retinex-Net PyTorch
2021 TCSVT Low-light image enhancement via progressive-recursive network paper PRIEN PyTorch
2021 TIP Band representation-based semi-supervised low-light image enhancement: Bridging the gap between signal fidelity and perceptual quality paper DRBN PyTorch
2021 ICCV Adaptive unfolding total variation network for low-light image enhancement paper UTVNet Code PyTorch
2021 ICCV Seeing dynamic scene in the dark: A high-quality video dataset with mechatronic alignment paper Wang et al. Code PyTorch
2022 CVPR SNR-aware Low-Light Image Enhancement paper SNR-Aware Code PyTorch
2022 CVPR Toward Fast, Flexible, and Robust Low-Light Image Enhancement paper SCI Code PyTorch
2022 CVPR URetinex-Net: Retinex-Based Deep Unfolding Network for Low-Light Image Enhancement paper URetinex-Net Code PyTorch

📋Datasets

Abbreviation Number Format Real/Synetic Video Paired/Unpaired/Application Dataset
LOL paper 500 RGB Real No Paired Dataset
SCIE paper 4413 RGB Real No Paired Dataset
MIT-Adobe FiveK paper 5000 Raw Real No Paired Dataset
SID paper 5094 Raw Real No Paired Dataset
DRV paper 202 Raw Real Yes Paired Dataset
SMOID paper 179 Raw Real Yes Paired Dataset
SDSD paper 150 RGB Real Yes Paired Dataset
LIME paper 10 RGB Real No Unpaired Dataset
NPE paper 84 RGB Real No Unpaired Dataset
MEF paper 17 RGB Real No Unpaired Dataset
DICM paper 64 RGB Real No Unpaired Dataset
VV 24 RGB Real No Unpaired Dataset
ExDARK paper 7363 RGB Real No Application (Object Detection) Dataset
BBD-100K paper 10,000 RGB Real Yes Application (Driving with diverse kinds of annotations) Dataset
DARK FACE paper 6000 RGB Real No Application (Face Recognition) Dataset
NightCity paper 4297 RGB Real No Application (Semantic Segmentation) Dataset

📋Metrics

Abbreviation Full-/Non-Reference Platform Code
MAE (Mean Absolute Error) Full-Reference
MSE (Mean Square Error) Full-Reference
PSNR (Peak Signal-to-Noise Ratio) Full-Reference
SSIM (Structural Similarity Index Measurement) Full-Reference MATLAB Code
LPIPS (Learned Perceptual Image Patch Similarity) Full-Reference PyTorch Code
LOE (Lightness Order Error) Non-Reference MATLAB Code
NIQE (Naturalness Image Quality Evaluator) Non-Reference MATLAB Code
PI (Perceptual Index) Non-Reference MATLAB Code
SPAQ (Smartphone Photography Attribute and Quality) Non-Reference PyTorch Code
NIMA (Neural Image Assessment) Non-Reference PyTorch/TensorFlow Code/Code
MUSIQ (Multi-scale Image Quality Transformer) Non-Reference TensorFlow Code

📜License

The code, platform, and dataset are made available for academic research purpose only.

📚Citation

If you find the repository helpful in your resarch, please cite the following paper.

@article{LoLi,
  title={Low-Light Image and Video Enhancement Using Deep Learning: A Survey},
  author={Li, Chongyi and Guo, Chunle and Han, Linghao and Jiang, Jun and Cheng, Ming-Ming and Gu, Jinwei and Loy, Chen Change},
  journal={IEEE Transactions on Pattern Analysis and Machine Intelligence},
  year={2021}
}

📋Paper

Official Version

arXiv Version

Chinese Version

📭Contact