Acta Photonica Sinica, Volume. 53, Issue 8, 0810001(2024)

A Multi-scale Hierarchical Residual Network-based Method for Tiny Object Detection in Optical Remote Sensing Images

Xiangjin ZENG1...2, Genghuan LIU1, Jianming CHEN1,2, Jiazhen DOU1, Zhenbo REN3, Jianglei DI1,*, and Yuwen QIN12,** |Show fewer author(s)
Author Affiliations
  • 1Key Laboratory of Photonic Technology for Integrated Sensing and Communication, Ministry of Education, and Guangdong Provincial Key Laboratory of Information Photonics Technology, Institute of Advanced Photonics Technology, School of Information Engineering, Guangdong University of Technology, Guangzhou 510006, China
  • 2Southern Marine Science and Engineering Guangdong Laboratory (Zhuhai), Zhuhai 519082, China
  • 3Key Laboratory of Light Field Manipulation and Information Acquisition, Ministry of Industry and Information Technology, and Shaanxi Key Laboratory of Optical Information Technology, School of Physical Science and Technology, Northwestern Polytechnical University, Xi'an 710129, China
  • show less
    Figures & Tables(15)
    The basic framework of MHRM-YOLO detection algorithm
    The structure of MHRM module
    Diagram of calculation factors for bounding box regression indicators
    Partial image instances and tiny objects in the AI-TODv2 dataset
    The generated heatmap at the P2 layer based on the CSPNet module and MHRM module
    Visual examples of detection results of different detection algorithms on AI-TODv2 dataset
    Visual examples of detection results of different detection algorithms on the TinyPerson dataset
    • Table 1. Comparison of the combination results of MHRM module and CSPNet module with P3-5 layer and P2-4 layer

      View table
      View in Article

      Table 1. Comparison of the combination results of MHRM module and CSPNet module with P3-5 layer and P2-4 layer

      Backbone moduleLayersAP50AP75mAP
      CSPNet (baseline)21P3, P4, P562.221.328.0
      CSPNet (baseline)P2, P3, P467.625.231.4
      MHRM (ours)P3, P4, P563.925.230.2
      MHRM (ours)P2, P3, P467.627.632.5
    • Table 2. Comparison between localization loss functions

      View table
      View in Article

      Table 2. Comparison between localization loss functions

      MethodAP50AP75mAP
      CIoU26 (baseline)67.627.632.5
      IoU loss + CDβ=0.5 (ours)66.126.131.5
      IoU loss + CDβ=1.0 (ours)67.327.832.6
      IoU loss + CD + BDβ=0.5 (ours)66.829.133.1
      IoU loss + CD + BDβ=1.0 (ours)68.129.233.5
    • Table 3. Comparison of three combinations of values for the number of bottleneck block stacks n in MHRM at layers C2, C3, C4 and C5

      View table
      View in Article

      Table 3. Comparison of three combinations of values for the number of bottleneck block stacks n in MHRM at layers C2, C3, C4 and C5

      n(C2, C3, C4, C5)Params (M)GFLOPs (G)AP50AP75mAPInference/ms
      1, 2, 2, 128.63181.167.229.133.114.5
      1, 3, 3, 130.93199.467.428.933.115.8
      1, 2, 3, 130.47190.568.129.233.515.3
    • Table 4. Comparison of five different backbone modules

      View table
      View in Article

      Table 4. Comparison of five different backbone modules

      BackboneParams (M)GFLOPs (G)AP50AP75mAPInference/ms
      ResNet538.30191.665.623.129.617.1
      CSPNet2134.79183.867.625.231.416.8
      SwinT-t*2835.60671.963.121.828.223.9
      Res2Net2435.66181.665.025.030.617.1
      MHRM (ours)30.47190.567.627.632.515.3
    • Table 5. Comparison of detection results between MHRM-YOLO ported to the YOLOv8 detection algorithm and the benchmark algorithm

      View table
      View in Article

      Table 5. Comparison of detection results between MHRM-YOLO ported to the YOLOv8 detection algorithm and the benchmark algorithm

      MethodLayersParams (M)GFLOPs (G)AP50AP75mAPInference/ms
      YOLOv829P3, P4, P543.6257.560.324.029.015.8
      YOLOv829P2, P3, P434.7309.864.025.730.718.8
      MHRM-YOLO (ours)P3, P4, P550.1225.963.027.231.410.1
      MHRM-YOLO (ours)P2, P3, P433.8248.965.729.433.312.1
    • Table 6. Comparison of detection performance between benchmark algorithms and MHRM on the AI-TODv2 test set

      View table
      View in Article

      Table 6. Comparison of detection performance between benchmark algorithms and MHRM on the AI-TODv2 test set

      MethodBackboneParams (M)GFLOPsAP50AP75mAPInference/ms
      CasCade-RCNN/FPN30ResNet50569.17324.626.19.111.844.4
      Faster-RCNN/FPN31ResNet5041.38269.124.39.211.239.4
      Sparce RCNN32ResNet50100.60192.224.56.29.749.8
      DetectoRS33ResNet50124.00236.329.312.714.8136.1
      Mask RCNN34SwinT-s2947.43297.626.19.112.178.1
      YOLOX35CSPNet2154.13243.353.715.320.647.4
      YOLOv519CSPNet46.10168.662.221.328.011.7
      YOLOv636CSPBep59.54235.246.417.421.615.0
      YOLOv723ELAN36.50161.442.319.518.09.7
      YOLOv829C2f43.60257.560.324.029.015.8
      MHRM-YOLO (ours)MHRM30.47190.568.129.233.515.3
    • Table 7. The AP and mAP performance of different algorithms on different categories on the AI-TODv2 test dataset

      View table
      View in Article

      Table 7. The AP and mAP performance of different algorithms on different categories on the AI-TODv2 test dataset

      MethodAIBRSTSHSPVEPEWMAP50mAP
      CasCade-RCNN/FPN3021.26.219.721.98.212.65.00.026.111.8
      Faster-RCNN/FPN3121.37.117.918.98.711.04.60.024.311.2
      Sparce RCNN329.19.513.226.15.57.23.63.524.59.7
      DetectoRS3326.614.018.824.614.413.36.20.0129.314.8
      Mask RCNN3422.78.918.422.411.010.22.81.326.112.1
      Cascade R-CNN/GPM 1824.323.942.546.030.130.89.49.458.627.1
      FSANet-SwinT*1130.915.135.040.319.824.98.95.652.822.6
      YOLOX3530.914.339.735.411.029.912.05.453.720.6
      YOLOv51938.618.541.843.824.333.115.97.762.228.0
      YOLOv63637.415.936.633.910.024.112.33.046.421.6
      YOLOv72325.79.035.931.023.528.110.61.242.318.0
      YOLOv82942.122.643.646.419.534.115.58.460.329.0
      MHRM-YOLO (ours)43.623.648.751.030.639.220.410.568.133.5
    • Table 8. Comparison of detection algorithms on the TinyPerson dataset

      View table
      View in Article

      Table 8. Comparison of detection algorithms on the TinyPerson dataset

      MethodParams (M)GFLOPsAP50AP75mAP
      YOLOv51946.11431.242.56.113.7
      YOLOv63659.50600.233.34.210.9
      YOLOv72336.50412.841.08.612.7
      YOLOv82943.60663.042.16.514.5
      MHRM-YOLO (ours)30.45486.846.26.415.5
    Tools

    Get Citation

    Copy Citation Text

    Xiangjin ZENG, Genghuan LIU, Jianming CHEN, Jiazhen DOU, Zhenbo REN, Jianglei DI, Yuwen QIN. A Multi-scale Hierarchical Residual Network-based Method for Tiny Object Detection in Optical Remote Sensing Images[J]. Acta Photonica Sinica, 2024, 53(8): 0810001

    Download Citation

    EndNote(RIS)BibTexPlain Text
    Save article for my favorites
    Paper Information

    Category:

    Received: Jan. 11, 2024

    Accepted: Mar. 5, 2024

    Published Online: Oct. 15, 2024

    The Author Email: DI Jianglei (jiangleidi@gdut.edu.cn), QIN Yuwen (qinyw@gdut.edu.cn)

    DOI:10.3788/gzxb20245308.0810001

    Topics