Infrared and Laser Engineering, Volume. 53, Issue 8, 20240198(2024)
Spatial and frequency domain feature decoupling for infrared and visible image fusion
Figures & Tables(15)
Fig. 1. (a) Framework of proposed algorithm: It contains a frequency decoupling branch, a spatial decoupling branch and a multi-frequency convolution attention block; (b) Framework of multi-frequency convolution attention block
Fig. 5. Visualization results of frequency domain disentangling branches
Fig. 6. Visualization results of spatial domain disentangling branches
Fig. 7. Visualization results of ablation comparison of different modules
Fig. 8. The visualization analysis and comparison of image pair "00959 N" in the MSRS dataset
Fig. 9. The visualization analysis and comparison of image pair "FLIR_04688" in the RoadScene dataset
Table 1. Objective evaluation results of RoadScene dataset comparison experiment (Bold, underlined, and italicized correspond to the first, second, and third places respectively)
View tableView in ArticleTable 1. Objective evaluation results of RoadScene dataset comparison experiment (Bold, underlined, and italicized correspond to the first, second, and third places respectively)
Method EN↑ SF↑ AG↑ MI↑ SCD↑ VIF↑ $ \mathrm{\mathit{Q}}^{{{{A}{B}}}/\mathrm{\mathit{F}}} $ SSIM↑ RFN-Nest 7.33 7.85 3.34 1.90 1.73 0.50 0.30 0.78 SwinFusion 7.00 14.36 5.55 1.79 1.43 0.6 0.59 1.02 SDNet 7.32 15.61 6.19 2.25 1.45 0.60 0.51 1.01 U2Fusion 6.80 11.81 4.77 1.82 1.20 0.53 0.47 0.98 Densefuse 6.84 9.50 3.69 1.88 1.36 0.51 0.37 0.93 FusionGAN 7.07 8.73 3.37 1.91 1.13 0.37 0.26 0.59 DATFusion 6.72 11.36 4.02 2.54 1.15 0.60 0.47 0.92 Ours 7.55 16.88 6.33 2.08 1.77 0.62 0.49 0.96
Table 2. Objective evaluation results of MSRS dataset comparison experiment (Bold, underlined, and italicized correspond to the first, second, and third places respectively)
View tableView in ArticleTable 2. Objective evaluation results of MSRS dataset comparison experiment (Bold, underlined, and italicized correspond to the first, second, and third places respectively)
Method EN↑ SF↑ AG↑ MI↑ SCD↑ VIF↑ $ {{Q}}^{{{{{A}{B}}}}/{{F}}} $ SSIM↑ RFN-Nest 6.20 6.16 2.11 1.70 1.47 0.66 0.39 0.77 SwinFusion 6.07 10.79 3.46 1.58 1.32 0.72 0.60 0.95 SDNet 5.25 8.67 2.67 1.18 0.99 0.50 0.38 0.72 U2Fusion 4.95 6.71 2.09 1.35 1.01 0.47 0.32 0.61 Densefuse 5.93 6.02 2.05 1.84 1.25 0.69 0.37 0.90 FusionGAN 5.43 4.35 1.45 1.31 0.98 0.44 0.14 0.50 DATFusion 6.48 10.93 3.56 2.70 1.41 0.91 0.62 0.91 Ours 6.71 12.28 3.94 2.65 1.69 0.94 0.64 0.95
Table 3. Objective evaluation results of TNO dataset comparison experiment (Bold, underlined, and italicized correspond to the first, second, and third places respectively)
View tableView in ArticleTable 3. Objective evaluation results of TNO dataset comparison experiment (Bold, underlined, and italicized correspond to the first, second, and third places respectively)
Method EN↑ SF↑ AG↑ MI↑ SCD↑ VIF↑ $ {{Q}}^{{A}{B}/{F}} $ SSIM RFN-Nest 6.96 5.87 2.66 1.46 1.78 0.56 0.33 0.81 SwinFusion 6.49 10.22 4.00 1.31 1.64 0.62 0.55 1.04 SDNet 6.69 11.64 4.56 1.56 1.56 0.58 0.43 0.97 U2Fusion 7.00 11.86 5.00 1.39 1.78 0.62 0.43 0.94 Densefuse 6.82 8.99 3.54 1.59 1.78 0.66 0.45 1.02 FusionGAN 6.56 6.28 2.41 1.62 1.38 0.42 0.23 0.66 DATFusion 6.45 9.61 3.54 2.17 1.50 0.68 0.48 0.93 Ours 7.31 13.14 5.19 1.84 1.79 0.75 0.50 0.99
Table 4. The impact of different spectral components on the fusion performance in MFCA (Bold, underlined, and italicized correspond to the first, second, and third places respectively)
View tableView in ArticleTable 4. The impact of different spectral components on the fusion performance in MFCA (Bold, underlined, and italicized correspond to the first, second, and third places respectively)
Dataset n EN $ \uparrow $ SF $ \uparrow $ AG $ \uparrow $ MI $ \uparrow $ SCD $ \uparrow $ VIF $ \uparrow $ $ {{Q}}^{{A}{B}/{F}}\uparrow $ SSIM $ \uparrow $ TNO n=2 6.99 13.4 4.88 1.42 0.74 0.32 0.40 0.09 n=4 7.09 15.33 5.93 1.41 0.37 0.24 0.42 0.15 n=8 7.10 13.52 5.13 1.54 0.85 0.36 0.46 0.11 n=16 7.31 13.14 5.19 1.84 1.79 0.75 0.50 0.99 n=32 7.29 12.35 5.34 1.68 1.53 0.47 0.39 0.72
Table 5. Evaluation results of ablation experiments for each component. (Bold, underlined, and italicized correspond to the first, second, and third places respectively)
View tableView in ArticleTable 5. Evaluation results of ablation experiments for each component. (Bold, underlined, and italicized correspond to the first, second, and third places respectively)
$ \mathrm{D}\mathrm{a}\mathrm{t}\mathrm{a}\mathrm{s}\mathrm{e}\mathrm{t} $ $ \mathrm{M}\mathrm{e}\mathrm{t}\mathrm{h}\mathrm{o}\mathrm{d}\mathrm{s} $ $ \mathrm{E}\mathrm{N}\uparrow $ $ \mathrm{A}\mathrm{G}\uparrow $ $ \mathrm{S}\mathrm{S}\mathrm{I}\mathrm{M}\uparrow $ $ \mathrm{V}\mathrm{I}\mathrm{F}\uparrow $ $ {{Q}}^{{A}{B}/{F}}\uparrow $ TNO Spa+MFCA 7.06 4.64 0.98 0.72 0.50 Fre+MFCA 7.15 4.81 0.99 0.72 0.51 Fre+Spa 7.18 5.04 1.01 0.76 0.53 Fre+Spa+MFCA 7.31 5.19 1.02 0.78 0.55 RoadScene Spa+MFCA 7.34 5.69 0.96 0.66 0.52 Fre+MFCA 7.36 4.94 0.97 0.69 0.50 Fre+Spa 7.50 6.33 0.99 0.67 0.50 Fre+Spa+MFCA 7.55 6.69 1.00 0.71 0.55 MSRS Spa+MFCA 6.67 3.74 0.97 0.97 0.65 Fre+MFCA 6.73 3.66 0.99 1.04 0.66 Fre+Spa 6.71 3.91 0.96 1.01 0.65 Fre+Spa+MFCA 6.80 3.92 1.00 1.04 0.70
Table 6. Evaluation results of the ablation study on loss functions (Bold, underlined, and italicized correspond to the first, second, and third places respectively)
View tableView in ArticleTable 6. Evaluation results of the ablation study on loss functions (Bold, underlined, and italicized correspond to the first, second, and third places respectively)
$ \mathrm{D}\mathrm{a}\mathrm{t}\mathrm{a}\mathrm{s}\mathrm{e}\mathrm{t} $ $ \mathrm{L}\mathrm{o}\mathrm{s}\mathrm{s} $ $ \mathrm{E}\mathrm{N}\uparrow $ $ \mathrm{M}\mathrm{I}\uparrow $ $ \mathrm{S}\mathrm{S}\mathrm{I}\mathrm{M}\uparrow $ $ \mathrm{V}\mathrm{I}\mathrm{F}\uparrow $ $ {{Q}}^{{A}{B}/{F}}\uparrow $ TNO $ {\mathcal{L}}_{{\mathrm{spa}}}+{\mathcal{L}}_{{\mathrm{recon}}} $ 7.19 1.36 0.15 0.37 0.39 $ {\mathcal{L}}_{{\mathrm{fre}}}+{\mathcal{L}}_{{\mathrm{recon}}} $ 7.12 1.73 0.94 0.60 0.40 $ {\mathcal{L}}_{{\mathrm{fre}}}+{\mathcal{L}}_{{\mathrm{spa}}}+{\mathcal{L}}_{{\mathrm{recon}}} $ 7.31 1.84 1.02 0.78 0.55
Tools
Get Citation
Copy Citation Text
Yan FAN, Qiao LIU, Di YUAN, Yunpeng LIU. Spatial and frequency domain feature decoupling for infrared and visible image fusion[J]. Infrared and Laser Engineering, 2024, 53(8): 20240198
Paper Information
Category:
Received: May. 8, 2024
Accepted: --
Published Online: Oct. 29, 2024
The Author Email:
© Copyright 2018-2021 | Chinese Laser Press.
All Rights Reserved 沪ICP备15018463号-20