Laser & Optoelectronics Progress, Volume. 60, Issue 24, 2410003(2023)

Object Detection via Multimodal Adaptive Feature Fusion

Xiaoqiang Gao1, Kan Chang1,2、*, Mingyang Ling1, and Mengyu Yin1
Author Affiliations
  • 1School of Computer and Electronic Information, Guangxi University, Nanning 530004, Guangxi, China
  • 2Guangxi Key Laboratory of Multimedia Communications and Network Technology, Nanning 530004, Guangxi, China
  • show less

    With the advancement of deep learning, object detection methods based on convolutional neural networks (CNNs) have achieved tremendous success. Existing CNN-based object detection models typically employ single-modal RGB images for training and testing; however, their detection performance is significantly degraded in low-light conditions. To address this issue, a multimodal object detection network model built on YOLOv5 is proposed, which integrates RGB and thermal infrared imagery to fully exploit the information provided by the fusion of multi-modal features, increasing the object detection accuracy. To achieve effective fusion of multimodal feature information, a multimodal adaptive feature fusion (MAFF) module is introduced. It facilitated multimodal feature fusion by adaptively selecting diverse modal features and exploiting the complementary information between modalities. The experimental results indicate the efficacy of the proposed algorithm for seamlessly merging features from distinct modalities, which significantly increases the detection accuracy.

    Tools

    Get Citation

    Copy Citation Text

    Xiaoqiang Gao, Kan Chang, Mingyang Ling, Mengyu Yin. Object Detection via Multimodal Adaptive Feature Fusion[J]. Laser & Optoelectronics Progress, 2023, 60(24): 2410003

    Download Citation

    EndNote(RIS)BibTexPlain Text
    Save article for my favorites
    Paper Information

    Category: Image Processing

    Received: Mar. 13, 2023

    Accepted: Apr. 12, 2023

    Published Online: Dec. 4, 2023

    The Author Email: Chang Kan (pandack0619@163.com)

    DOI:10.3788/LOP230856

    Topics