Laser & Optoelectronics Progress, Volume. 60, Issue 2, 0228011(2023)

Three-Dimensional Pedestrian Detection by Fusing Image Semantics and Point Cloud Spatial Visibility Features

Lu Xiong, Zhenwen Deng, Wei Tian*, and Zhiang Wang
Author Affiliations
  • School of Automotive Studies, Tongji University, Shanghai 201804, China
  • show less
    Figures & Tables(15)
    Schematic of ray traversing grid
    Logic diagram of 2D Raycasting algorithm
    Overall framework of fusion network
    Semantic segmentation and point feature enhancement
    Geometric feature and semantic feature encoding
    Spatial visibility feature encoding
    Feature fusion and detection heads
    Visibility feature visualization. (a) BEV of point cloud; (b) single layer feature
    Comparison results of pedestrian detection (example 1). (a) (c) Benchmark results; (b) (d) results obtained by proposed method
    Comparison results of pedestrian detection (example 2). (a) (c) Benchmark results; (b) (d) results obtained by proposed method
    • Table 1. Performance comparison of different visibility description methods on KITTI dataset

      View table

      Table 1. Performance comparison of different visibility description methods on KITTI dataset

      Visibility code

      [U,O,F]

      3D pedestrian detection AP /%mAP /%
      EasyModerateHard
      [0,1,-1]68.7662.4957.7463.00
      [0.5,0.7,0.4]70.8464.5759.7765.06
    • Table 2. Performance comparison of different density along height direction on nuScenes dataset

      View table

      Table 2. Performance comparison of different density along height direction on nuScenes dataset

      Number of channels3D pedestrian detection AP /%mAP /%
      0.5 m1.0 m2.0 m4.0 m
      162.2464.3666.3668.7865.44
      3269.6871.8773.7375.9772.81
    • Table 3. Performance comparison of different number of frames on nuScenes dataset

      View table

      Table 3. Performance comparison of different number of frames on nuScenes dataset

      Nunber of frames3D pedestrian detection AP /%mAP /%
      0.5 m1.0 m2.0 m4.0 m
      137.4638.2239.2940.4038.84
      1068.3670.6372.3074.5971.47
    • Table 4. Performance comparison of different optimized methods on KITTI validation set

      View table

      Table 4. Performance comparison of different optimized methods on KITTI validation set

      Method3D pedestrian detection AP /%mAP /%
      EasyModerateHard
      PP(Baseline)70.1663.4057.4963.68
      PP+Vis.71.8664.4958.7265.02
      PP+Img.72.0865.5660.3465.99
      PP+Vis.+Img.71.8465.6560.8166.10
    • Table 5. Performance comparison of different methods on KITTI test set

      View table

      Table 5. Performance comparison of different methods on KITTI test set

      Method3D pedestrian detection AP /%Speed /Hz
      EasyModerateHard
      VoxelNet739.4833.6931.514.4
      AVOD336.1027.8625.7610
      SECOND1851.0742.5637.2920
      F-PointNet450.5342.1538.085.9
      PointPainting650.3240.9737.872.5
      PointRCNN1947.9839.3736.0110
      Proposed method51.0741.3637.8330
    Tools

    Get Citation

    Copy Citation Text

    Lu Xiong, Zhenwen Deng, Wei Tian, Zhiang Wang. Three-Dimensional Pedestrian Detection by Fusing Image Semantics and Point Cloud Spatial Visibility Features[J]. Laser & Optoelectronics Progress, 2023, 60(2): 0228011

    Download Citation

    EndNote(RIS)BibTexPlain Text
    Save article for my favorites
    Paper Information

    Category: Remote Sensing and Sensors

    Received: Feb. 14, 2022

    Accepted: Mar. 14, 2022

    Published Online: Feb. 7, 2023

    The Author Email: Wei Tian (tian_wei@tongji.edu.cn)

    DOI:10.3788/LOP220712

    Topics