Laser & Optoelectronics Progress, Volume. 59, Issue 8, 0815002(2022)

Multitarget Detection Algorithm Based on Multimodal Information Fusion

Tong Liu, Sijie Gao*, and Weizhi Nie
Author Affiliations
  • School of Electrical and Information Engineering, Tianjin University, Tianjin 300072, China
  • show less

    Developing information acquisition equipment, such as lidar, has made intelligent target detection increasingly important. Recently, there has been an increasing need for an effective and intelligent target detection technology to realize the intelligent detection and recognition of pedestrians, vehicles, and other targets, and improve the intelligence level of unmanned driving, urban management, and other applications. Thus, to solve the lack of information in the use of light detection and ranging (LiDAR) for 3D target detection, this paper proposes a multimodal information fusion-based multitarget detection algorithm. The network model comprises three modules: LiDAR point cloud data processing module, 2D image data processing module, and information fusion and detection module. The first two extracted the point cloud and RGB image features, respectively, whereas the information fusion and detection module merged the three- and two-dimensional feature maps according to the corresponding positions to mitigate the lack of information in the monomodal data and achieve the complementarity of the feature level. The fused feature map generated the target detection frame using the three- and two-dimensional area generation networks and adopted the post-fusion strategy to fuse the detection frames of both modes to obtain the final target detection result. KITTI and VOC2007 datasets were used for evaluation and analysis. Experimental results demonstrated the superiority of the proposed algorithm.

    Tools

    Get Citation

    Copy Citation Text

    Tong Liu, Sijie Gao, Weizhi Nie. Multitarget Detection Algorithm Based on Multimodal Information Fusion[J]. Laser & Optoelectronics Progress, 2022, 59(8): 0815002

    Download Citation

    EndNote(RIS)BibTexPlain Text
    Save article for my favorites
    Paper Information

    Category: Machine Vision

    Received: Mar. 8, 2021

    Accepted: Apr. 21, 2021

    Published Online: Apr. 11, 2022

    The Author Email: Gao Sijie (gaosijie_0112@tju.edu.cn)

    DOI:10.3788/LOP202259.0815002

    Topics