Optics and Precision Engineering, Volume. 33, Issue 2, 282(2025)
Multiscale semantic injective fusion of nighttime infrared and visible
Aiming at the problems of unclear texture details and poor visual perception due to neglecting illumination in infrared and visible image fusion under low-light conditions, a low-light enhancement and semantic injection multi-scale infrared and visible image fusion method was proposed. Firstly, a network suitable for low-light enhancement was designed to realize the enhancement of visible images in nighttime scenes through repeated iterations of residual models. Then, a feature extractor based on the Nest architecture was used as the encoder and decoder of the network, in which the deep features could capture the complex structure and semantic information of the images. A semantic prior learning module was designed to further extract the semantic information of the deep infrared and visible images through cross-attention, and a semantic injection unit was adopted to inject the enhancement features into each scale step by step. Thirdly, a gradient enhancement branch was designed, where the mainstream features were first passed through hybrid attention, and then the Sobel operator stream and Laplacian operator stream were divided from the mainstream as a way to enhance the gradient of the fused image. Finally, the features at each scale were reconstructed by dense connections between the same layers and jump connections between different layers in the decoder. Experimental results show that this method improves the visual information fidelity, mutual information, disparity correlation coefficient, and spatial frequency, on average, by 23.1%, 16.3%, 18%, and 39.8%, respectively, in comparison with the nine methods, which effectively enhances the quality of fused images in low-light environments, and helps to improve the performance of the advanced visual tasks.
Get Citation
Copy Citation Text
Yanchun YANG, Jialong LI, Yi LI, Zeyu WANG. Multiscale semantic injective fusion of nighttime infrared and visible[J]. Optics and Precision Engineering, 2025, 33(2): 282
Category:
Received: May. 21, 2024
Accepted: --
Published Online: Apr. 30, 2025
The Author Email: Yanchun YANG (yangyanchun102@sina.com)