Opto-Electronic Engineering, Volume. 51, Issue 6, 240093-1(2024)

LF-UMTI: unsupervised multi-exposure light field image fusion based on multi-scale spatial-angular interaction

Yulong Li1... Yeyao Chen1, Yueli Cui2 and Mei Yu1,* |Show fewer author(s)
Author Affiliations
  • 1Faculty of Information Science and Engineering, Ningbo University, Ningbo, Zhejiang 315211, China
  • 2School of Electronic and Information Engineering, Taizhou University, Taizhou, Zhejiang 318000, China
  • show less

    Light field imaging can simultaneously capture the intensity and direction information of light in a real-world scene. However, due to the limited capacity of imaging sensors, light field images captured with a single exposure struggle to fully record all the details in the real scene. To address the aforementioned issue, an unsupervised multi-exposure light field imaging method based on multi-scale spatial-angular interactions is proposed in this paper. A multi-scale spatial-angular interaction strategy is adopted to effectively extract spatial-angular features of the light field. Additionally, a channel-wise modeling strategy is employed to reduce computational complexity and adapt to the high-dimensional structure of the light field. Furthermore, a light field reconstruction module guided by reversible neural networks is constructed to avoid fusion artifacts and recover more detailed information. Lastly, an angle consistency loss is designed, considering the disparity variations between boundary sub-aperture images and the central sub-aperture image, to ensure the disparity structure of the fusion result. To evaluate the performance of the proposed method, a benchmark dataset for multi-exposure light field imaging is created, targeting real-world scenes. Experimental results demonstrate that the proposed method can reconstruct light field images with high contrast and rich details while ensuring angular consistency. Compared with the existing methods, the proposed method achieves superior results in both objective quality and subjective visual perception.

    Keywords
    Tools

    Get Citation

    Copy Citation Text

    Yulong Li, Yeyao Chen, Yueli Cui, Mei Yu. LF-UMTI: unsupervised multi-exposure light field image fusion based on multi-scale spatial-angular interaction[J]. Opto-Electronic Engineering, 2024, 51(6): 240093-1

    Download Citation

    EndNote(RIS)BibTexPlain Text
    Save article for my favorites
    Paper Information

    Category:

    Received: Apr. 23, 2024

    Accepted: Jun. 3, 2024

    Published Online: Oct. 21, 2024

    The Author Email: Yu Mei (郁梅)

    DOI:10.12086/oee.2024.240093

    Topics