Infrared and Visible Image Fusion Based on Adaptive Feature Enhancement and Generator Path Interaction

Yejun Yang; Gang Liu; Gang Xiao; Xinjie Gu

doi:10.3788/LOP222204

Laser & Optoelectronics Progress, Volume. 60, Issue 14, 1410018(2023)

Infrared and Visible Image Fusion Based on Adaptive Feature Enhancement and Generator Path Interaction

Yejun Yang¹, Gang Liu^1、*, Gang Xiao², and Xinjie Gu¹

Author Affiliations

¹School of Automation Engineering, Shanghai University of Electric Power, Shanghai 200090, China

²School of Aeronautics and Astronautics, Shanghai Jiao Tong University, Shanghai 200240, China

show less

Abstract Get PDF(in Chinese)

Figures & Tables(12)

Fig. 1. Fusion network architecture based on adaptive feature enhancement and generator path interaction

Download full size

Fig. 2. Schematic of adaptive enhancement block

Download full size

Fig. 3. Network architecture of generator

Download full size

Fig. 4. Network architecture of discriminator

Download full size

Fig. 5. Fusion results of different methods in bench images

Download full size

Fig. 6. Qualitative comparison of different methods on 6 pairs of typical infrared and visible images

Download full size

Fig. 7. Quantitative comparison of six indicators in 18 pairs of TNO test sets

Download full size

Fig. 8. Kaptein_1123 images for ablation experiments of enhanced blocks. (a) Infrared image; (b) visible image; (c) fused image without the enhancement block; (d) fused image of the proposed method

Download full size

Fig. 9. Effects before and after weight action. (a) Infrared image; (b) local infrared image; (c) local infrared gradient map; (d) effect of weights acting on infrared gradient; (e) visible image; (f) local visible image; (g) local visible gradient map; (h) effect of weights acting on visible gradient

Download full size

Fig. 10. Experimental analysis of ablation with hyperparameters

Download full size

Table 1. Training process of proposed method

View table

Table 1. Training process of proposed method

Algorithm：training process of proposed method

Input：training set infrared image $I_{i}$ and visible image $I_{v}$

Output：fused image $I_{f}$

1）for M epochs do

2）for m steps do

3）for p times do

4）Select b infrared patches｛ $I_{i}^{1}$ ， $I_{i}^{2}$ ，…， $I_{i}^{b}$ ｝

5）Select b visible patches｛ $I_{v}^{1}$ ， $I_{v}^{2}$ ，…， $I_{v}^{b}$ ｝

6）Select b fused patches｛ $I_{f}^{1}$ ， $I_{f}^{2}$ ，…， $I_{f}^{b}$ ｝

7）Update the parameters of the discriminator by Adam optimizer

8）end

9）Select b infrared patches｛ $I_{i}^{1}$ ， $I_{i}^{2}$ ，…， $I_{i}^{b}$ ｝

10）Select b visible patches｛ $I_{v}^{1}$ ， $I_{v}^{2}$ ，…， $I_{v}^{b}$ ｝

11）Update the parameters of the generator by Adam optimizer

12）end

13）end

Table 2. Average running time of different methods on the selected 18 pairs of TNO test set
View table
Table 2. Average running time of different methods on the selected 18 pairs of TNO test set
Method TNO Method TNO
MDLatLRR 70.5981 DenseFuse 5.0580
FPDE 2.9546 U2Fusion 6.0424
LatLRR 154.4078 FusionGAN 7.1445
GTF 6.8297 GANMcC 14.3756
DDcGAN 15.6905 Proposed method 1.6773

Tools

Get Citation

Copy Citation Text

Yejun Yang, Gang Liu, Gang Xiao, Xinjie Gu. Infrared and Visible Image Fusion Based on Adaptive Feature Enhancement and Generator Path Interaction[J]. Laser & Optoelectronics Progress, 2023, 60(14): 1410018

Download Citation

EndNote(RIS)BibTex Plain Text

Set citation alerts for article

Save article for my favorites