Journal of Applied Optics, Volume. 45, Issue 6, 1095(2024)
Review of low-illuminance image enhancement algorithm based on deep learning
Figures & Tables(8)
Fig. 1. Examples of low-illuminance images in some paired low-light datasets (row 1 refers to low-light images, and row 2 refers to reference images)
Fig. 2. Examples of low-illuminance images in some unpaired low-light datasets
Fig. 3. Contrast of low-illuminance images enhancement effect in MIT-Adobe FiveK-test dataset with different methods
Fig. 4. Contrast of low-illuminance images enhancement effect in LOL-test dataset with different methods
Table 1. Representative low-illuminance image enhancement methods based on deep learning
View tableView in ArticleTable 1. Representative low-illuminance image enhancement methods based on deep learning
Method Learinng Network structure(model) Loss function Format Code Publication Project address LLNet[ 14 ]SL Stacked sparse denoising autoencoder SRR loss RGB Theano PR(2017) https://github.com/kglore/llnet_color LightenNet[ 27 ]SL Four layers loss RGB Caffe PR(2018) https://github.com/Li-Chongyi/low-light-codes MBLLEN[ 19 ]SL Feature extraction module; Enhancement module; Fusion module SSIM loss; Region loss; Perceptual loss RGB TensorFlow BNVC(2018) https://github.com/Lvfeifan/MBLLEN RetinexNet[ 26 ]SL Multi-scale network loss; Smoothness loss; Invariable reflectance loss RGB TensorFlow BMVC(2018) https://github.com/weichen582/RetinexNet SICE[ 40 ]SL Low frequency luminance; High frequency detail loss; loss; SSIM loss RGB Caffe TIP(2018) https://github.com/csjcai/SICE SID[ 20 ]SL Amplification ratio loss RAW TensorFlow CVPR(2018) https://github.com/cchen156/Learning-to-See-in-the-Dark SMD[ 41 ]SL Filtered results; Siamese network Recovery loss; Self-Consistency loss RAW TensorFlow ICCV(2019) https://github.com/cchen156/Seeing-Motion-in-the-Dark SMOID[ 42 ]SL 3D U-Net loss RAW TensorFlow ICCV(2019) https://github.com/MichaelHYJiang/Learning-to-See-Moving-Objects-in-the-Dark DeepUPE[ 23 ]SL Illumination map loss; Color loss; Smoothness loss RGB TensorFlow ACM(2019) https://github.com/dvlab-research/DeepUPE EnlightenGAN[ 30 ]UL Attention map; Self-regularzation Adversarial loss; Self feature preserving loss RGB PyTorch arXiv(2019) https://github.com/VITA-Group/EnlightenGAN KinD[ 29 ]SL Reflectance layers loss; SSIM loss; Reflectance similarity loss; loss; smoothness loss; RGB TensorFlow ACMMM(2019) https://github.com/zhangyhuaee/KinD ExCNet[ 37 ]ZSL Fully connected layers Energy minimization loss RGB PyTorch ACMMM(2019) https://cslinzhang.github.io/ExCNet/ DSLR[ 22 ]SL Laplacian pyramid; U-Net like network loss; Color loss; Laplacian loss RGB PyTorch IEEE(2020) https://github.com/SeokjaeLIM/DSLR-release TBEFN[ 44 ]SL Three stages; U-Net like network SSIM loss; Perceptual loss Smoothness loss; RGB PyTorch IEEE(2020) https://github.com/lukun199/TBEFN Zero-DCE[ 39 ]ZSL Fully connected network Illumination smoothness loss; Spatial consistency loss; Color constancy loss RGB PyTorch CVPR(2020) https://github.com/soumik12345/Zero-DCE DRBN[ 35 ]SSL Recursive network SSIM loss; Perceptual loss; Adversarial loss RGB PyTorch CVPR(2021) https://github.com/flyywh/CVPR-2020-Semi-Low-Light Retinex DIP[ 45 ]ZSL Encoder-decoder network Reflectance loss; Smoothness loss RGB PyTorch IEEE(2021) https://github.com/zhaozunjin/RetinexDIP RUAS[ 46 ]ZSL Neural architecture search Cooperative loss; Similar Loss; Total variation loss RGB PyTorch CVPR (2021) https://github.com/KarelZhang/RUAS SCI[ 34 ]UL Self-Calibrated Illumination Fidelity loss; Smoothness loss RGB PyTorch CVPR(2022) https://github.com/tengyu1998/SCI SNR-aware[ 43 ]SL SNR-guided attention Charbonnier loss; Perceptual loss RGB PyTorch CVPR(2022) https://github.com/dvlab-research/SNR-Aware-Low-Light-Enhance Dimma[ 36 ]SSL Mixture density network; U-Net like network Mean squared error loss; Perceptual loss RGB PyTorch arXiv(2023) https://github.com/WojciechKoz/Dimma PairLIE[ 15 ]UL Encoder-decoder networks Projection Loss; Retinex Loss; Reflectance consistency loss RGB PyTorch CVPR(2023) https://github.com/zhenqifu/PairLIE CUE[ 62 ]SL Masked autoencoder; Customized learnable priors Illumination smoothness loss; Noise prior loss RGB PyTorch ICCV(2023) https://github.com/zheng980629/CUE
Table 2. Summary of low-illuminance image enhancement dataset
View tableView in ArticleTable 2. Summary of low-illuminance image enhancement dataset
Dataset Date Number Resolution/pixel Format Real/ Synthetic Paired Download link LIME[47] 2017 10 326×326～2 000×15000 RGB Real No https://github.com/estija/LIME NPE[12] 2013 85 267×304～749×492 RGB Real No https://github.com/Spirals-Team/npe-dataset DICM[48] 2013 64 481×321 RGB Real No https://github.com/JoshuaEbenezer/LDR ExDark[49] 2019 7363 500×332～1600×1066 RGB Real No https://github.com/cs-chan/Exclusively-Dark-Image-Dataset VE-LOL-H[50] 2021 10940 1080×720 RGB Real No https://flyywh.github.io/IJCV2021LowLight_VELOL/ SID[20] 2018 5094 4240×2832 or 6000×4000 RAW Real Yes https://github.com/cchen156/Learning-to-See-in-the-Dark LOL[26] 2018 789 400×600 RGB Real Yes https://daooshee.github.io/BMVC2018website/ SICE[40] 2018 4413 3000×2 000 or 6000×4 000 RGB Real Yes https://github.com/csjcai/SICE MIT-Adobe Fivek[51] 2011 5000 ～6048×4032 RAW Real+ Synthetic Yes https://data.csail.mit.edu/graphics/fivek/ DRV[41] 2019 202 3672×5496 RAW Real Yes https://github.com/cchen156/Seeing-Motion-in-the-Dark VE-LOL-L[50] 2021 2500 400×600 RGB Real+ Synthetic Yes https://flyywh.github.io/IJCV2021LowLight_VELOL/ UHD-LOL[52] 2023 11065 4000 or 8000×8000 RGB Real Yes https://github.com/TaoWangzj/LLFormer
Table 3. Performance comparison on MIT-Adobe FiveK-test dataset
View tableView in ArticleTable 3. Performance comparison on MIT-Adobe FiveK-test dataset
Metnod MSE↓ PSNR↑ SSIM↑ NIQE↓ BRISQUE↓ input[ 51 ]1.723 14.825 0.764 6.124 31.183 LLNet[ 14 ]4.241 9.698 0.473 6.688 38.794 lightenNet[ 27 ]4.172 15.139 0.635 6.971 29.159 MBLLEN[ 19 ]1.267 14.965 0.863 6.947 32.318 RetinexNet[ 26 ]3.943 11.333 0.492 4.036 26.355 KinD[ 29 ]1.609 14.054 0.558 4.217 35.661 TBEFN[ 44 ]3.690 9.760 0.461 5.308 30.181 EnlightenGAN[ 30 ]3.837 14.598 0.793 3.915 31.345 SCI[ 34 ]3.608 14.820 0.723 3.801 29.395 ExCNet[ 37 ]2.927 12.698 0.473 7.648 38.794 Zero-DCE[ 39 ]3.360 12.416 0.736 4.037 31.637 RRDNet[ 38 ]6.199 9.966 0.416 5.173 37.278 RUAS[ 46 ]3.376 9.588 0.460 4.194 27.971 DRBN[ 35 ]3.410 13.639 0.754 4.361 38.904
Table 4. Performance comparison on LOL-test dataset
View tableView in ArticleTable 4. Performance comparison on LOL-test dataset
Metnod MSE↓ PSNR↑ SSIM↑ NIQE↓ BRISQUE↓ input[ 26 ]9.712 7.951 0.131 7.948 42.153 LLNet[ 14 ]1.080 16.583 0.715 5.453 38.549 lightenNet[ 27 ]7.993 11.876 0.428 5.394 13.440 MBLLEN[ 19 ]1.172 18.257 0.723 5.318 12.238 RetinexNet[ 26 ]1.714 16.141 0.464 5.702 38.681 KinD[ 29 ]1.133 16.240 0.704 4.764 27.438 TBEFN[ 44 ]1.445 16.002 0.730 3.983 10.956 EnlightenGAN[ 30 ]1.824 15.997 0.715 3.720 11.070 SCI[ 34 ]1.503 15.654 0.491 3.308 14.863 ExCNet[ 37 ]2.752 15.370 0.538 4.648 19.134 Zero-DCE[ 39 ]2.851 15.278 0.535 5.596 15.408 RRDNet[ 38 ]5.993 13.982 0.461 4.909 14.071 RUAS[ 46 ]3.144 14.667 0.416 5.625 12.287 DRBN[ 35 ]2.622 15.047 0.432 5.121 22.781
Tools
Get Citation
Copy Citation Text
Ziwei LI, Jinlong LIU, Huizhen YANG, Zhiguang ZHANG. Review of low-illuminance image enhancement algorithm based on deep learning[J]. Journal of Applied Optics, 2024, 45(6): 1095
Paper Information
Category:
Received: Oct. 17, 2023
Accepted: --
Published Online: Jan. 14, 2025
The Author Email: Jinlong LIU (刘金龙)
© Copyright 2018-2021 | Chinese Laser Press.
All Rights Reserved 沪ICP备15018463号-20