Laser & Optoelectronics Progress, Volume. 60, Issue 22, 2200003(2023)
Review of Multi-Exposure Image Fusion Methods
Figures & Tables(11)
Table 1. Summary of multi-exposure image fusion methods for static scenes
View tableTable 1. Summary of multi-exposure image fusion methods for static scenes
Type Category Name Year/Source Contribution Spatial domain method method
Pixel-based
Raman et al.［ 6 ］2009/EUROGRAPHICS Compositing based bilateral filter Li et al.［ 7 ］2012/IEEE Fusion with median filter and recursive filter Lee et al.［ 8 ］2018/IEEE Adaptive weighting reflects relative pixel intensity and global gradients Xu et al.［ 9 ］2020/Optik Using patterns of oriented edge agnitudes to extract local contrast Ulucan et al.［ 10 ］2021/Signal Processing Using linear embeddings and watershed masking to fusion Li et al.［ 11 ］2021/China Sciencepaper Adaptive weights constructed from pixel intensity and global gradients Kinoshita et al.［ 12 ］2019/IEEE Segmentation based on luminance distribution Wang et al.［ 13 ］2021/Visual Computer Determined the region of enhancement（RoE）for each image Patch-based method Goshtasbyet al.［ 14 ］2005/Image and Vision Computing Firstly multi-exposure fusion using image blocks Ma et al.［ 15 ］2015/IEEE Decomposing image patch into signal strength，signal structure，and mean intensity Huang et al.［ 16 ］2018/IEEE Three weight measurements build signal structure and mean intensity Li et al.［ 17 ］2020/Journal of Computer Applications Local variance，local saliency features，and local visibility build weight graph Li et al.［ 18 ］2020/IEEE Non-normalized operations Li et al.［ 19 ］2021/IEEE Incorporating the edge-preserving factors into mean intensity Wang et al.［ 20 ］2020/IEEE Using the super-pixel segmentation approach Optimization-based method Shen et al.［ 21 ］2011/IEEE A generalized random walk framework Li et al.［ 22 ］2012/IEEE Using quadratic optimization method Liu et al.［ 24 ］2019/IEEE Using optimal weighted multi-exposure fusion mechanism Ma et al.［ 25 ］2018/IEEE Describing a gradient ascent-based algorithm Transform domain
method
Multi-scale decomposition-based method Burt et al.［ 26 ］1993/IEEE Applying a gradient pyramid model to multi-exposure fusion firstly Mertens et al.［ 27 ］2007/IEEE Weighted Gaussian pyramid to Laplacian pyramid Shen et al.［ 28 ］2014/IEEE Building weight graphs using local weights，global weights，and significance weights Li et al.［ 29 ］2013/IEEE Weighted guided filtering on Gaussian pyramids Yan et al.［ 30 ］2019/Pattern Recognition Letters A simulated exposure model for generating multiple images Li et al.［ 31 ］2017/IEEE Using guided filtering to break down image into base layer and detail layer Singh et al.［ 32 ］2014/Scientific World Journal Adding details to Laplacian pyramid Wang et al.［ 33 ］2020/IEEE Laplacian pyramid in the YUV color space Kou et al.［ 34 ］2018/Journal of Visual Communication and Image Representation Edge-preserving smoothing pyramid Yang et al.［ 35 ］2018/IEEE Generating a moderately exposed analog mapping image Tang et al.［ 36 ］2022/Laser & Optoelectronics Progress Proposing a high dynamic range imaging method based on YCbCr spatial fusion Liu et al.［ 37 ］2022/Laser & Optoelectronics Progress A full sequence based on images is proposed multi-exposure image fusion method for feature weights Wu et al.［ 38 ］2021/Laser & Optoelectronics Progress Proposing a multi-exposure image fusion method based on improved exposure evaluation and double pyramid Gradient-based method Zhang et al.［ 39 ］2012/IEEE A two-dimensional Gaussian filter to calculate the gradient value and gradient direction Paul et al.［ 40 ］2016/Journal of Circuits，Systems and Computers Add the gradient value of the chroma to the gradient value of the light value Liu et al.［ 41 ］2020/IET Image Processing Computing luminance levels in the gradient domain Gu et al.［ 42 ］2012/Journal of Visual Communication & Image Representation Modifying the gradient field iteratively with twice average filtering and nonlinearly compressing in multi-scales Sparse representation-based method Wang et al.［ 43 ］2014/Neurocomputing A novel recognition framework based on the discriminative sparse representation Shao et al.［ 44 ］2018/Applied Sciences A halo-free multi-exposure fusion method based on sparse representation of gradient features Yang et al.［ 45 ］2020/IEEE An exposure fusion method with sparse decomposition and a sparsity exposure dictionary Deep learning
method
CNN Kalantari et al.［ 46 ］2017/ACM Transactions on Graphics The first learning-based technique to produce an HDR image Li et al.［ 47 ］2018/IEEE Using CNN to extract the features of each image Liu et al.［ 48 ］2019/Information Fusion Proposing the FusionNet Chen et al.［ 49 ］2019/Journal of Visual Communication and Image Representation Constructing a dual network cascade model Cai et al.［ 50 ］2018/IEEE A large-scale multi-exposure image data Prabhakaret al.［ 51 ］2017/IEEE First ever unsupervised deep learning method（DeepFuse） Qi et al.［ 52 ］2021/Information Fusion An unsupervised deep learning approach based on quantitative evaluation Han et al.［ 53 ］2022/ Information Fusion Proposing a depth-aware enhancement network Zhang et al.［ 54 ］2020/ Information Fusion Proposing a general image fusion framework based on the convolutional neural network Ma et al.［ 55 ］2019/IEEE Multi-exposure fusion network based on deep guided learning Xu et al.［ 56 ］2022/IEEE An unsupervised end-to-end image fusion network（U2Fusion） Gao et al.［ 57 ］2021/Electronics Applying to the identification of traffic signs GAN Xu et al.［ 58 ］2020/IEEE Proposing a GAN-based multi-exposure image fusion network firstly（MEF-GAN） Yang et al.［ 59 ］2021/Neural Computing
and Applications
A novel GAN-based multi-exposure image fusion method（GANFuse） Le et al.［ 60 ］2022/ Information Fusion Generative adversarial networks based on continuous learning（UIFGAN） Zhou et al.［ 61 ］2022/ Information Fusion Proposing an image fusion method based on GAN（GIDGAN）
Table 2. Summary of multi-exposure image fusion methods for dynamic scenes
View tableTable 2. Summary of multi-exposure image fusion methods for dynamic scenes
Category Name Year/Source Contribution Based on motion detection Khan et al.［ 63 ］2006/ICIP Computed weights iteratively to remove ghost Jacobs et al.［ 64 ］2008/IEEE Calculate local information entropy to detect moving pixels Pece et al.［ 5 ］2010/IEEE Using the median threshold to design the bitmap motion detection algorithm Zhang et al.［ 65 ］2010/IEEE Using gradient direction consistency as the basis for motion detection Zhang et al.［ 66 ］2017/Information Sciences Detects motion by comparing the structural consistency of images Ma et al.［ 67 ］2017/IEEE Calculate the structural consistency of the direction of the components of the signal structure Hu et al.［ 68 ］2019/IEEE Proposing a convolutional neural network based deghosting algorithm Liu et al.［ 69 ］2015/ Journal of Visual Communication and Image Representation A method based on dense scale invariant feature transform Ye et al.［ 70 ］2022/Electronics Proposing a ghost-free multi-exposure image fusion technique Based on image registration Kang et al.［ 71 ］2003/ACM Transactions on Graphics Gradient-based optical flow estimation of the motion field Hu et al.［ 72 ］2012/ECCV Aligning images with local consistency in geometry and luminosity Zimmer et al.［ 73 ］2011/Computer Graphics Forum A modern energy-based optic flow approach Sie et al.［ 74 ］2014/IEEE Determining the fusing map via Markov Random Field Ulucan et al.［ 75 ］2023/Signal Processing Exposure intensity mapping using histogram matching operators Xue et al.［ 76 ］2016/Asian Conference on Computer Vision Introducing a random walk model Trinidad et al.［ 77 ］2019/IEEE A cascade feature extraction method based on optical flow Khan et al.［ 78 ］2021/Journal of Visual Communication and Image Representation A method that relies on the accurate detection and matching of feature points across adjacent viewpoints Based on deep learning Peng et al.［ 79 ］2018/IEEE Using networks with deep network optical flow registration for alignment registration Prabhakar et al.［ 81 ］2019/IEEE A CNN-based fast，scalable image deghosting method Deng et al.［ 82 ］2020/SPIE A multi-scale contextual attention guided alignment network（CAHDRNet） Yan et al.［ 83 ］2020/IEEE An attention-guided end-to-end deep neural net-work（AHDRNet） Yan et al.［ 84 ］2022/International Journal of Computer Vision A dual-attention-guided end-to-end deep neural network（DAHDRNet）
Table 3. Summary of static multi-exposure image sequences
View tableTable 3. Summary of static multi-exposure image sequences
Image set Size Number Image origin Image set Size Number Image origin Arno 339×521 3 Kede Ma［ 25 ］Laurenziana 512×356 3 Kede Ma［ 25 ］Balloons 339×521 9 Erik.Reinhard［ 85 ］Lighthouse 340×512 3 HDRsoft［ 86 ］Belgium house 384×521 9 Dani Lischinski［ 87 ］Mask 800×1200 3 HDRsoft［ 86 ］Cafe 247×371 3 Fei Kou［ 34 ］Office 340×512 6 Matlab［ 88 ］Candle 364×512 6 Kede Ma［ 86 ］Preschool 247×371 3 Fei Kou［ 34 ］Cave 384×512 4 Kede Ma［ 86 ］Room 467×700 3 Pangeasoft［ 89 ］Chinese garden 340×512 3 Kede Ma［ 86 ］Set 341×512 3 Kede Ma［ 25 ］Church 335×512 3 Jianbbing Shen［ 28 ］Sports centre 247×371 3 Fei Kou［ 34 ］Farmhouse 341×512 3 HDR Projects［ 90 ］Tower 512×341 3 Kede Ma［ 7 ］House 500×752 4 Mertens［ 91 ］Tree 408×271 3 Fei Kou［ 34 ］Kluki 341×512 3 Kede Ma［ 86 ］Venice 341×512 3 HDRsoft［ 86 ］Laurenziana 512×356 3 Kede Ma［ 25 ］Window 384×512 3 Kede Ma［ 25 ］Lighthouse 340×512 3 HDRsoft［ 86 ］Yellow hall 339×512 3 Kede Ma［ 25 ］
Table 4. Summary of dynamic multi-exposure image sequences
View tableTable 4. Summary of dynamic multi-exposure image sequences
Image set Size Number Image origin Image set Size Number Image origin Arch 1024×669 5 Orazio Gallo［ 92 ］Office 768×1024 3 Kede Ma［ 58 ］Brunswick 683×1024 3 Fabrizio Pece［ 5 ］Prof.JeonEigth 681×1024 7 Zhengguo Li［ 17 ］Campus 648×1011 6 Wei Zhang［ 36 ］Puppets 1024×812 5 Orazio Gallo［ 92 ］Cliffs1 683×1024 3 Fabrizio Pece［ 5 ］Russ1 683×1024 3 Fabrizio Pece［ 5 ］Forrest 683×1024 4 Orazio Gallo［ 92 ］Sculpture garden 754×1024 5 Orazio Gallo［ 92 ］Horse 690×1024 3 Kang［ 71 ］Square 683×1024 3 Fabrizio Pece［ 5 ］Lady 1024×686 3 Jun Hu［ 93 ］Tate3 683×1024 3 Fabrizio Pece［ 5 ］Noise camera 480×640 10 Orazio Gallo［ 92 ］Wroclav 683×1024 3 Fabrizio Pece［ 5 ］Noise camera 480×640 10 Orazio Gallo［ 92 ］YWFusionopolis 681×1024 6 Zhengguo Li［ 17 ］
Table 5. Summary of performance evaluation indicators of multi-exposure image fusion method for static scenes
View tableTable 5. Summary of performance evaluation indicators of multi-exposure image fusion method for static scenes
Method MEF-SSIM MI PSNR SD He QAB/F NIQE Raman et al.［ 6 ］Li et al.［ 7 ］√ Lee et al.［ 8 ］√ Xu et al.［ 9 ］√ Ulucan et al.［ 10 ］√ Li et al.［ 11 ］√ √ Kinoshita et al.［ 12 ］√ √ Wang et al.［ 13 ］√ Goshtasby et al.［ 14 ］Ma et al.［ 15 ］√ Huang et al.［ 16 ］√ Li et al.［ 17 ］√ Li et al.［ 18 ］√ Li et al.［ 19 ］√ Wang et al.［ 20 ］√ √ Shen et al.［ 21 ］Li et al.［ 22 ］Liu et al.［ 24 ］√ Ma et al.［ 25 ］√ Burt et al.［ 26 ］Mertens et al.［ 27 ］Shen et al.［ 28 ］Li et al.［ 29 ］Yan et al.［ 30 ］Li et al.［ 31 ］√ Singh et al.［ 32 ］√ √ Wang et al.［ 33 ］√ Kou et al.［ 34 ］√ Yang et al.［ 35 ］√ Zhang et al.［ 39 ］Paul et al.［ 40 ］Liu et al.［ 41 ］√ Gu et al.［ 42 ］Shao et al.［ 44 ］√ √ Wang et al.［ 43 ］√ Yang et al.［ 45 ］√ Kalantari et al.［ 46 ］√ Li et al.［ 47 ］√ Liu et al.［ 48 ］√ Chen et al.［ 49 ］√ √ √ Cai et al.［ 50 ］√ Prabhakar et al.［ 51 ］√ Qi et al.［ 52 ］√ Xu et al.［ 56 ］√ √ Gao et al.［ 57 ］√ √ √ Yang et al.［ 59 ］√ √ √
Table 6. Summary of performance evaluation indicators of multi-exposure image fusion method for dynamic scenes
View tableTable 6. Summary of performance evaluation indicators of multi-exposure image fusion method for dynamic scenes
Method MEF-SSIM MI PSNR SD He QAB/F NIQE Khan et al.［ 63 ］Jacobs et al.［ 64 ］√ Pece et al.［ 5 ］Zhang et al.［ 65 ］Zhang et al.［ 66 ］Ma et al.［ 67 ］√ Hu et al.［ 68 ］Liu et al.［ 69 ］√ Ye et al.［ 70 ］√ √ Kang et al.［ 71 ］Hu et al.［ 72 ］Zimmer et al.［ 73 ］Sie et al.［ 74 ］Ulucan et al.［ 75 ］√ √ Xue et al.［ 76 ］Trinidad et al.［ 77 ］√ √ Khan et al.［ 78 ］√ √ √ Peng et al.［ 79 ］√ Prabhakar et al.［ 81 ］√ √ Deng et al.［ 82 ］√ Yan et al.［ 83 ］√ Yan et al.［ 84 ］√
Tools
Get Citation
Copy Citation Text
Xinli Zhu, Yasheng Zhang, Yuqiang Fang, Xitao Zhang, Jieping Xu, Di Luo. Review of Multi-Exposure Image Fusion Methods[J]. Laser & Optoelectronics Progress, 2023, 60(22): 2200003
Paper Information
Category: Reviews
Received: Mar. 17, 2023
Accepted: Apr. 3, 2023
Published Online: Nov. 3, 2023
The Author Email: Yuqiang Fang (fangyuqiang@nudt.edu.cn)
© Copyright 2018-2021 | Chinese Laser Press.
All Rights Reserved 沪ICP备15018463号-20