Infrared and Laser Engineering, Volume. 52, Issue 2, 20220338(2023)

Transformer-based multi-source images instance segmentation network for composite materials

Yan Ke1, Yun Fu2, Weizhu Zhou1, and Weidong Zhu1
Author Affiliations
  • 1School of Mechanical Engineering, Zhejiang University, Hangzhou 310027, China
  • 2Xizi Spirit Aerospace Industry (Zhejiang) Ltd, Hangzhou 310018, China
  • show less
    Figures & Tables(16)
    Acquisition platform of composite material defect image
    (a) Infrared image; (b) Enhance the reflection of the calibration plate; (c) Infrared image before distortion correction; (d) Infrared image after distortion correction
    (a) Before transformation; (b) After transformation
    (a) Visible image; (b) Infrared image; (c) Defect label
    Network structure of Yolact
    Network structure of Trans-Yolact
    Multi-Head Self-Attention structure
    FPN structure based on Transformer
    (a) Self-Transformer; (b) Grounding Transformer; (c) Rendering Transformer
    Loss of Trans-Yolact
    PR test curves of six composite defects
    (a) Visible image; (b) Infrared image; (c) Yolact detection results; (d) Trans-Yolact detection results
    Performance of Yolact and Trans-Yolact under different network depth layers
    Field detecting experiment
    • Table 1. AP of six composite defects

      View table
      View in Article

      Table 1. AP of six composite defects

      TwistWrinkleBridgeBubbleMissForeignTotal
      Yolact IR VI0.7710.8430.9590.7830.7360.8560.825
      Yolact IR0.7150.8570.9040.6320.7220.8990.788
      Yolact VI0.5010.8720.9290.7300.6850.8200.756
      Backbone improved0.7900.8700.9330.8070.8360.9100.857
      Transformer improved0.8060.8860.9520.7400.8570.9050.858
      Trans-Yolact0.8220.8890.9480.7880.8880.9470.880
    • Table 2. Prune-optimized network structure and test data

      View table
      View in Article

      Table 2. Prune-optimized network structure and test data

      Trans-YolactYolact
      C111111111
      C2(SE)3(√)3(√)3(√)3(√)3(√)3(×)3(×)3(×)
      C3(SE)4(√)4(√)4(√)4(√)4(√)4(×)4(×)4(×)
      C42318128423124
      C5(MHSA)3(√)3(√)3(√)3(√)3(√)3(×)3(×)3(×)
      FLOPs79.79 G71.95 G63.74 G58.27 G50.11 G79.28 G64.22 G53.27 G
      Params44.69 M38.62 M31.92 M27.45 M22.59 M49.62 M37.34 M28.40 M
      File space176.6 M152.8 M126.5 M108.9 M89.8 M194.5 M146.3 M111.2 M
      mAP88.0387.6386.6986.0383.5082.4882.1080.50
      FPS37.7245.2755.6857.6759.0336.5342.4048.25
    Tools

    Get Citation

    Copy Citation Text

    Yan Ke, Yun Fu, Weizhu Zhou, Weidong Zhu. Transformer-based multi-source images instance segmentation network for composite materials[J]. Infrared and Laser Engineering, 2023, 52(2): 20220338

    Download Citation

    EndNote(RIS)BibTexPlain Text
    Save article for my favorites
    Paper Information

    Category: Photoelectric measurement

    Received: Jun. 20, 2022

    Accepted: --

    Published Online: Mar. 13, 2023

    The Author Email:

    DOI:10.3788/IRLA20220338

    Topics