Optics and Precision Engineering, Volume. 33, Issue 6, 945(2025)

Cross-modality image matching algorithm based on policy gradient and pseudo-twin network

Jian ZHANG1,2,3, Ao LIANG1,2,3, Haiyang HUA1,2、*, Tianci LIU1,2, and Shihan LI1,2,3
Author Affiliations
  • 1Key Laboratory of Opto-Electronic Information Processing,Chinese Academy of Sciences, Shenyang006, China
  • 2Shenyang Institute of Automation, Chinese Academy of Sciences, Shenyang110016, China
  • 3University of Chinese Academy of Sciences, Beijing100049, China
  • show less
    Figures & Tables(17)
    Framework of the proposed method
    Differences between visible and infrared images
    Keypoints acquisition
    Pseudo-twin neural network framework
    Fusion layer
    Data set presentation (The first line pertain to the VEDAI datasets, and the second line pertain to the MTV datasets.)
    Experimental results of PCMM-Net on VEDAI dataset
    Experimental results of PCMM-Net on MTV dataset
    Matching accuracy of different algorithms at various pixel thresholds ε in the VEDAI and MTV Datasets
    MTV image registration results
    Matching results of VEDAI dataset original map and affine transform
    • Table 1. VEDAI matches results

      View table
      View in Article

      Table 1. VEDAI matches results

      数据集指标SIFT[5RIFT[23CMM-Net[13D2-Net[24DALF[18PCMM-Net
      VEDAIMA/%86.5470.5941.7941.0595.5197.77
      NCM286.1810.7263.01 026.1890.21 326.7
      RMSE0.441.295.1442.4210.640.55
      T/s0.212.180.2800.490.190.35
    • Table 2. MTV matches results

      View table
      View in Article

      Table 2. MTV matches results

      数据集指标SIFT[5RIFT[23CMM-Net[13D2-Net[24DALF[18PCMM-Net
      MTVMA/%5.8279.6277.3176.3781.8895.88
      NCM3.144720.6283.0542.485.1934.4
      RMSE2.5013.2905.5923.3632.8902.03
      T/s0.1204.1580.3550.6050.4700.714
    • Table 3. Sensitivity analysis experiment on the MTV Dataset

      View table
      View in Article

      Table 3. Sensitivity analysis experiment on the MTV Dataset

      网格大小W/pixel图像块大小M/pixel
      67891011121314283032343638404244
      MA/%82.2384.4988.1295.8890.1388.0487.4087.8588.0794.1393.3195.8895.1194.0493.9293.5594.1593.67
      NCM310.5372.3627934.4871.7794.6462332.9245.9865.9860.4934.4880.3890.3900.6843.9872.9790.3
    • Table 4. Ablation experiment

      View table
      View in Article

      Table 4. Ablation experiment

      数据集指标NBN+NFMSBN+NFMPNN+NFMSBN+FMPNN+FM
      VEDAIMA/%78.7393.4694.9895.1297.77
      NCM839.851031.08950.981101.021326.79
      MTVMA/%80.5793.1893.4294.1595.88
      NCM765.3829.6726.1850.6934.4
    • Table 5. Influence of the number of key points on the result of the experiment

      View table
      View in Article

      Table 5. Influence of the number of key points on the result of the experiment

      数据集关键点数量MA/%NCMNRMA/%
      MTV10095.4929.729.70
      50096.14206.941.38
      1 00095.90461.546.15
      1 50095.44746.349.75
      1 93095.39924.547.7
    • Table 6. Experiment on image scale and image rotation adaptabilit

      View table
      View in Article

      Table 6. Experiment on image scale and image rotation adaptabilit

      旋转适应性实验
      旋转度数05101520253035
      MA/%95.8887.8584.8578.5766.1051.8420.0206.17
      尺度适应性实验
      尺寸大小10.90.80.70.60.50.40.3
      MA/%95.8890.6290.0384.4866.2938.2211.423.12
    Tools

    Get Citation

    Copy Citation Text

    Jian ZHANG, Ao LIANG, Haiyang HUA, Tianci LIU, Shihan LI. Cross-modality image matching algorithm based on policy gradient and pseudo-twin network[J]. Optics and Precision Engineering, 2025, 33(6): 945

    Download Citation

    EndNote(RIS)BibTexPlain Text
    Save article for my favorites
    Paper Information

    Category:

    Received: May. 11, 2024

    Accepted: --

    Published Online: Jun. 16, 2025

    The Author Email: Haiyang HUA (c3ill@sia. cn)

    DOI:10.37188/OPE.20253306.0945

    Topics