Optics and Precision Engineering, Volume. 33, Issue 2, 282(2025)

Multiscale semantic injective fusion of nighttime infrared and visible

Yanchun YANG*, Jialong LI, Yi LI, and Zeyu WANG
Author Affiliations
  • School of Electronical and Information Engineering,Lanzhou Jiaotong University, Lanzhou730070, China
  • show less

    Aiming at the problems of unclear texture details and poor visual perception due to neglecting illumination in infrared and visible image fusion under low-light conditions, a low-light enhancement and semantic injection multi-scale infrared and visible image fusion method was proposed. Firstly, a network suitable for low-light enhancement was designed to realize the enhancement of visible images in nighttime scenes through repeated iterations of residual models. Then, a feature extractor based on the Nest architecture was used as the encoder and decoder of the network, in which the deep features could capture the complex structure and semantic information of the images. A semantic prior learning module was designed to further extract the semantic information of the deep infrared and visible images through cross-attention, and a semantic injection unit was adopted to inject the enhancement features into each scale step by step. Thirdly, a gradient enhancement branch was designed, where the mainstream features were first passed through hybrid attention, and then the Sobel operator stream and Laplacian operator stream were divided from the mainstream as a way to enhance the gradient of the fused image. Finally, the features at each scale were reconstructed by dense connections between the same layers and jump connections between different layers in the decoder. Experimental results show that this method improves the visual information fidelity, mutual information, disparity correlation coefficient, and spatial frequency, on average, by 23.1%, 16.3%, 18%, and 39.8%, respectively, in comparison with the nine methods, which effectively enhances the quality of fused images in low-light environments, and helps to improve the performance of the advanced visual tasks.

    Keywords
    Tools

    Get Citation

    Copy Citation Text

    Yanchun YANG, Jialong LI, Yi LI, Zeyu WANG. Multiscale semantic injective fusion of nighttime infrared and visible[J]. Optics and Precision Engineering, 2025, 33(2): 282

    Download Citation

    EndNote(RIS)BibTexPlain Text
    Save article for my favorites
    Paper Information

    Category:

    Received: May. 21, 2024

    Accepted: --

    Published Online: Apr. 30, 2025

    The Author Email: Yanchun YANG (yangyanchun102@sina.com)

    DOI:10.37188/OPE.20253302.0282

    Topics