Acta Optica Sinica, Volume. 45, Issue 6, 0628010(2025)
Nonlinear Radiation and Geometric Invariant Matching for Multimodal Imagery
Figures & Tables(17)
Fig. 1. Multi-scale, multi-directional Log-Gabor convolution results for an optical-depth image pair
Fig. 2. Example of feature detection results. (a) An optical-depth image pair; (b) maximum moment maps; (c) minimum moment maps; (d) weighted moment maps; (e) detected feature points
Fig. 4. Comparison of direction index maps for an optical-near infrared image pair under different rotation angles. (a) Original image pair; (b) 0°; (c) 60°; (d) 120°
Fig. 5. Original and modified direction index maps under different rotation angles. (a) Optical image rotated 60°; (b) optical image after modification; (c) near infrared image rotated 60°; (d) near infrared image after modification
Fig. 6. Visualization results of feature matching at different stages for an optical-near infrared image pair. (a) Original multimodal image pair; (b) extracted feature points; (c) original index map and main feature directions; (d) modified index map and main feature directions; (e) matching results.
Fig. 7. Template matching results. (a) Original multimodal image pair; (b) coarsely corrected image pair; (c) template features; (d) matching point pairs obtained on corrected image; (e) matching point pairs obtained on original image; (f) coordinates of partially matched points
Fig. 8. Visualization comparison of different algorithms on vision multimodal images (left image shows matching results, and right image shows fusion results)
Fig. 9. Visualization comparison of different algorithms on medical multimodal images (left image shows matching results, and right image shows fusion results)
Fig. 10. Visualization comparison of different algorithms on remote sensing multimodal images (left image shows matching results, and right image shows fusion results)
Fig. 11. Performance of algorithm on NCM and RMSE under different rotation angles
Table 1. Experimental data
View tableTable 1. Experimental data
Aera Dataset No. Type Number Vision 1 Low light RGB‑IR 20 2 RGB‑Depth 20 3 RGB‑NIR 20 Medical 1 PD‑T1 20 2 Multisource‑Retina 20 3 T1‑T2 20 Remote
sensing
1 Multitemporal‑Optical 20 2 Optical‑SAR 20 3 Optical‑IR 20
Table 2. Average performance of NRGM algorithm under different parameter settings across various datasets
View tableTable 2. Average performance of NRGM algorithm under different parameter settings across various datasets
s o l NCM Pre Rec F1 RMSE SR 2 12 72 1681.24 91.22 76.32 82.28 1.49 94.44 3 12 72 2261.24 97.80 84.04 89.63 1.16 98.89 4 12 72 2384.59 96.60 83.96 89.21 1.19 97.78 5 12 72 2152.28 92.18 81.35 85.71 1.45 95.56 4 8 72 2267.76 96.73 83.52 88.93 1.20 97.78 4 10 72 2294.10 93.44 81.31 86.17 1.34 95.56 4 12 72 2384.59 96.60 83.96 89.21 1.19 97.78 4 16 72 2387.49 95.82 84.21 88.77 1.27 98.89 4 12 60 2243.63 97.00 82.83 88.61 1.20 98.89 4 12 72 2384.59 96.60 83.96 89.21 1.19 97.78 4 12 84 2447.78 97.86 86.17 90.82 1.16 100.00 4 12 96 2476.72 93.56 83.62 87.59 1.37 96.67
Table 3. Quantitative experimental results of different algorithms on visual multimodal images
View tableTable 3. Quantitative experimental results of different algorithms on visual multimodal images
Dataset Metrics SIFT RIFT HOWP ASS MatchFormer SemLA GIFT NRGM 1 NCM 8.15 8.50 4.00 112.30 0.80 11.05 1189.60 2686.40 Pre 73.51 24.48 12.78 64.01 4.67 7.39 71.27 95.92 Rec 81.31 26.13 16.17 67.60 6.66 10.29 86.59 69.74 F1 77.08 24.94 14.00 65.71 5.43 8.58 76.38 80.66 RMSE 2.44 4.80 4.74 2.90 4.88 4.84 2.67 1.48 SR 85 25 10 90 10 15 85 95 2 NCM 0 25.95 6.00 59.50 3.30 8.80 968.55 2188.00 Pre 0 40.43 11.71 40.25 5.45 8.56 68.39 96.26 Rec 0 41.89 12.35 40.48 9.71 10.78 80.76 75.59 F1 0 40.19 11.95 40.35 6.24 9.53 72.56 84.37 RMSE 5.00 4.03 4.72 3.69 4.85 4.87 2.85 1.39 SR 0 55 15 55 5 10 80 100 3 NCM 376.25 712.20 731.65 1679.25 1817.55 261.70 2719.10 3889.40 Pre 93.53 85.48 89.96 87.09 91.04 48.02 95.05 91.96 Rec 97.76 84.92 92.51 90.13 96.85 51.07 98.98 96.31 F1 95.15 83.46 90.89 88.20 93.28 49.40 96.48 93.47 RMSE 1.40 2.13 2.00 1.89 1.87 3.72 1.25 1.23 SR 95 90 95 90 95 75 95 90
Table 4. Quantitative experimental results of different algorithms on medical multimodal images
View tableTable 4. Quantitative experimental results of different algorithms on medical multimodal images
Dataset Metrics SIFT RIFT HOWP ASS MatchFormer SemLA GIFT NRGM 1 NCM 9.80 100.15 38.90 285.25 73.85 68.50 309.50 1987.05 Pre 41.13 47.29 16.44 81.84 19.01 15.64 27.35 96.90 Rec 44.77 52.46 16.95 87.32 19.46 16.24 36.53 99.53 F1 42.52 49.52 16.66 84.44 19.23 15.93 30.57 97.57 RMSE 3.47 3.63 4.50 2.21 4.32 4.48 4.28 0.88 SR 40 60 20 95 20 20 30 100 2 NCM 29.10 352.10 254.85 302.85 588.15 316.20 1498.00 3664.80 Pre 22.88 45.84 42.44 44.18 36.51 28.87 47.38 98.26 Rec 25.08 47.05 43.75 45.15 41.59 38.41 49.61 96.80 F1 23.51 46.40 43.07 44.64 38.60 32.55 48.21 97.49 RMSE 4.25 3.54 3.60 3.50 3.77 4.12 3.13 1.11 SR 20 50 50 50 50 45 50 100 3 NCM 5.65 300.55 41.65 414.30 41.20 74.25 1802.85 2242.00 Pre 41.16 68.64 11.93 89.14 9.58 10.25 87.23 99.94 Rec 44.54 74.33 11.90 93.09 9.49 10.59 92.80 99.91 F1 41.99 71.18 11.90 91.03 9.53 10.41 89.78 99.92 RMSE 3.57 2.83 4.59 1.88 4.67 4.68 1.79 0.65 SR 35 80 15 100 10 15 95 100
Table 5. Quantitative experimental results of different algorithms on remote sensing multimodal images
View tableTable 5. Quantitative experimental results of different algorithms on remote sensing multimodal images
Dataset Metrics SIFT RIFT HOWP ASS MatchFormer SemLA GIFT NRGM 1 NCM 20.30 31.25 18.75 195.05 40.15 29.20 2163.15 3950.80 Pre 53.16 30.59 15.31 68.41 15.22 13.00 85.31 99.95 Rec 58.27 34.04 18.10 69.50 18.51 16.52 91.24 92.41 F1 55.51 31.99 16.51 68.78 16.69 14.47 87.34 95.80 RMSE 3.17 4.29 4.51 2.80 4.48 4.58 1.93 0.98 SR 60 40 20 90 20 25 90 100 2 NCM 0 5.40 7.85 58.20 0.05 9.10 662.90 1519.65 Pre 0 14.36 14.62 52.61 0.55 18.27 61.15 87.70 Rec 0 16.59 15.52 51.49 0.83 22.74 78.10 59.64 F1 0 15.21 15.03 51.89 0.66 20.18 67.75 70.72 RMSE 5.00 4.87 4.58 3.31 5.00 4.44 3.11 1.74 SR 0 10 25 85 0 35 90 90 3 NCM 0.15 22.05 0.55 6.55 25.20 18.95 41.70 438.85 Pre 3.75 53.04 6.78 15.04 16.78 23.19 5.81 89.95 Rec 3.75 63.67 6.78 17.60 19.26 22.86 8.27 89.23 F1 3.75 56.80 6.78 16.18 17.83 22.98 6.58 89.57 RMSE 4.91 3.39 4.82 4.46 4.66 4.59 4.84 1.40 SR 5 75 10 20 25 15 5 90
Table 6. Average running time of different algorithms
View tableTable 6. Average running time of different algorithms
Aera SIFT
（C++）
RIFT
（MATLAB）
HOWP
（MATLAB）
ASS
（MATLAB）
MatchFormer
（Python）
SemLA
（Python）
GIFT
（MATLAB）
NRGM
（MATLAB）
Vision 0.94 72.71 8.62 21.40 7.26 3.77 28.44 23.23 Medical 1.55 43.52 6.56 19.81 5.15 2.92 22.13 21.68 Remote sensing 0.83 36.33 7.26 8.73 6.87 3.53 16.24 13.51
Tools
Get Citation
Copy Citation Text
Yuxuan Liu, Li Zhang, Zhongli Fan, Yushan Sun, Haibin Ai, Xueqing Ban. Nonlinear Radiation and Geometric Invariant Matching for Multimodal Imagery[J]. Acta Optica Sinica, 2025, 45(6): 0628010
Paper Information
Category: Remote Sensing and Sensors
Received: Jul. 17, 2024
Accepted: Sep. 30, 2024
Published Online: Mar. 17, 2025
The Author Email: Fan Zhongli (fanzhongli@whu.edu.cn)
© Copyright 2018-2021 | Chinese Laser Press.
All Rights Reserved 沪ICP备15018463号-20