Journal of Infrared and Millimeter Waves, Volume. 43, Issue 6, 859(2024)

Progressive spatio-temporal feature fusion network for infrared small-dim target detection

Dan ZENG1, Jian-Ming WEI1, Jun-Jie ZHANG1, Liang CHANG2, and Wei HUANG1、*
Author Affiliations
  • 1School of Communication and Information Engineering,Shanghai University,Shanghai 200444,China
  • 2Innovation Academy for Microsatellites,Chinese Academy of Sciences,Shanghai 201203,China
  • show less
    Figures & Tables(14)
    Progressive spatio-temporal feature fusion network structure:(a)overall architecture of PSTFNet;(b)progressive temporal accumalation module;(c)multi-scale spatial feature fusion module
    Progressive temporal accumulation module:(a)architecture of the P2DConv module;(b)architecture of the M3DConv module
    SHU-MIRST dataset simulation flowchart:(a)background shooting;(b)target template production;(c)target 3D modeling;(d)image fusion algorithm for region resampling;(e)target template embedding
    SHU-MIRST dataset statistical information: (a) distribution of target sizes;(b) distribution of mean SCR
    Examples of target motion trajectory in the SHU-MIRST dataset
    ROC curves of PSTFNet under different mSCR: (a) mSCR≤3;(b) mSCR>3;(c) all sequences
    Qualitative comparison results of PSTFNet and 6 benchmark algorithms on the SHU-MIRST Dataset
    Visualization map of PSTFNet and the backbone network ResUNet at different stage of decoder
    • Table 1. Quantitative comparison of different algorithms on the SHU-MIRST dataset

      View table
      View in Article

      Table 1. Quantitative comparison of different algorithms on the SHU-MIRST dataset

      方法SHU-MIRST(mSCR≤3)SHU-MIRST(mSCR>3)SHU-MIRST(all)
      IoU/(%)Pd/(%)Fa(10-6IoU/(%)Pd/(%)Fa(10-6IoU/(%)Pd/(%)Fa(10-6
      Top-Hat0.000.83856.812.6711.17185.810.934.45621.96
      IPI0.192.7580.232.7214.7557.341.086.9572.22
      PSTNN0.000.14122.942.4110.31129.360.843.70125.19
      WSLCM0.4545.804 623.485.6180.223 562.332.2657.854 252.08
      WSNM-STIPI9.6153.6135.9513.6766.0136.3511.0357.9536.09
      IMNN-LWEC0.000.0032.240.123.96139.760.041.3869.87
      ASTTV-NTLA0.000.3080.290.405.0234.670.141.9564.34
      RDIAN36.4052.0736.4667.3684.8415.4047.2363.5429.09
      DNANet38.7461.8239.7574.1985.5610.6051.1470.1329.55
      ISNet36.1749.0113.1565.3382.4613.2346.3860.7213.18
      UIUNet43.5455.9311.8874.2990.613.2854.3068.078.87
      SSTNet-64.0918.55-93.568.92-74.4015.17
      ResUNet-DTUM51.7868.5113.3275.5393.836.6060.0977.3710.97
      DNANet-DTUM51.9169.1921.6376.7193.982.6760.5977.8615.00
      Ours57.6875.8010.8076.2895.082.6964.1982.557.97
    • Table 2. Quantitative comparison of different algorithms on the IRDST-Real dataset

      View table
      View in Article

      Table 2. Quantitative comparison of different algorithms on the IRDST-Real dataset

      方法IoU/(%)Pd/(%)Fa(10-6
      Top-Hat5.3924.66489.28
      IPI9.3836.5537.11
      PSTNN5.7917.5857.05
      WSLCM4.9237.441 389.62
      WSNM-STIPI17.7959.6638.92
      IMNN-LWEC3.107.99641.05
      ASTTV-NTLA0.271.82395.59
      RDIAN47.6986.043.95
      DNANet50.3482.575.15
      ISNet50.3582.383.86
      UIUNet48.7381.542.70
      SSTNet-85.114.83
      ResUNet-DTUM50.3186.192.87
      DNANet-DTUM50.9887.033.62
      Ours53.9391.252.26
    • Table 3. Results of the ablation experiment for the PSTFNet component modules

      View table
      View in Article

      Table 3. Results of the ablation experiment for the PSTFNet component modules

      方法IoU/(%)Pd/(%)Fa(10-6
      Backbone37.1750.4124.89
      Backbone + PTAM58.1076.2312.13
      Backbone + MSFM40.5856.7811.08
      PSTFNet64.1982.557.97
    • Table 4. Results of the PTAM layer ablation experiment

      View table
      View in Article

      Table 4. Results of the PTAM layer ablation experiment

      方法IoU/(%)Pd/(%)Fa(10-6
      PSTFNet w/o PTAM40.5856.7811.08
      PSTFNet w/o PTAM L12343.2658.9513.14
      PSTFNet w/o PTAM L1249.0464.029.95
      PSTFNet w/o PTAM L155.3372.6010.29
      PSTFNet64.1982.557.97
    • Table 5. Results of the PTAM composition ablation experiment

      View table
      View in Article

      Table 5. Results of the PTAM composition ablation experiment

      方法IoU/(%)Pd/(%)Fa(10-6
      PSTFNet w/o PTAM40.5856.7811.08
      PSTFNet w/o M3DConv58.7474.997.17
      PSTFNet w/o P2Dconv53.8366.5113.23
      PSTFNet64.1982.557.97
    • Table 6. Results of the MSFM composition ablation experiment

      View table
      View in Article

      Table 6. Results of the MSFM composition ablation experiment

      方法IoU/(%)Pd/(%)Fa(10-6
      PSTFNet w/o MSFM58.1076.2312.13
      PSTFNet w/o MC60.5279.169.25
      PSTFNet w/o SA62.4781.3817.18
      PSTFNet64.1982.557.97
    Tools

    Get Citation

    Copy Citation Text

    Dan ZENG, Jian-Ming WEI, Jun-Jie ZHANG, Liang CHANG, Wei HUANG. Progressive spatio-temporal feature fusion network for infrared small-dim target detection[J]. Journal of Infrared and Millimeter Waves, 2024, 43(6): 859

    Download Citation

    EndNote(RIS)BibTexPlain Text
    Save article for my favorites
    Paper Information

    Category: Interdisciplinary Research on Infrared Science

    Received: Mar. 24, 2024

    Accepted: --

    Published Online: Dec. 13, 2024

    The Author Email: HUANG Wei (lyxhw@shu.edu.cn)

    DOI:10.11972/j.issn.1001-9014.2024.06.017

    Topics