Laser & Optoelectronics Progress, Volume. 60, Issue 24, 2410003(2023)
Object Detection via Multimodal Adaptive Feature Fusion
Figures & Tables(11)
Fig. 5. Qualitative comparison with other algorithms in FLIR dataset. (a) Truth image; (b) CFT; (c) ProbEn; (d) MAFFNet
Table 1. Influence of the number of MAFF modules on object detection results
View tableTable 1. Influence of the number of MAFF modules on object detection results
MAFF module Parameter quantity /106 GFLOPs FPS /（frame·s-1） mAP50 Person Car Bicycle All Without MAFF module 76.47 195.8 25 0.864 0.912 0.652 0.809 MAFF_1 76.34 194.1 24 0.868 0.914 0.676 0.819 MAFF_2 75.95 194.1 24 0.872 0.910 0.681 0.821 MAFF_3 74.37 197.1 24 0.867 0.910 0.668 0.815 MAFF_1+MAFF_2 75.82 192.4 23 0.882 0.918 0.701 0.834 MAFF_1+MAFF_3 74.24 192.4 24 0.879 0.913 0.690 0.827 MAFF_2+MAFF_3 73.85 192.4 23 0.879 0.914 0.736 0.843 MAFF_1+MAFF_2 +MAFF_3 73.72 190.8 23 0.886 0.922 0.740 0.849
Table 2. Influence of single component of fusion module on object detection results
View tableTable 2. Influence of single component of fusion module on object detection results
Module Parameter quantity /106 GFLOPs FPS /（frame·s-1） mAP50 Person Car Bicycle All JAM 76.47 195.8 23 0.857 0.905 0.685 0.816 SFF 73.72 190.7 24 0.883 0.917 0.721 0.840 JAM+SFF 73.72 190.8 23 0.886 0.922 0.740 0.849
Table 3. Influence of different component of fusion module on object detection results
View tableTable 3. Influence of different component of fusion module on object detection results
Method Parameter quantity /106 GFLOPs FPS /（frame·s-1） mAP50 Person Car Bicycle All JAM+SFF-GAP 73.72 190.8 23 0.888 0.919 0.722 0.843 JAM+DSFF 74.23 191.2 21 0.881 0.912 0.734 0.844 JAM-ECA+SFF 73.72 190.8 23 0.891 0.920 0.725 0.845 JAM+SFF 73.72 190.8 23 0.886 0.922 0.740 0.849
Table 4. Performance comparison of different algorithms in FLIR dataset
View tableTable 4. Performance comparison of different algorithms in FLIR dataset
Method Backbone Data mAP50 Person Car Bicycle All YOLOv5 CSPDarkNet RGB 0.581 0.781 0.407 0.590 YOLOv5 CSPDarkNet Thermal 0.791 0.887 0.538 0.739 CFR［ 25 ］VGG16 RGB+T 0.745 0.849 0.578 0.724 GAFF［ 27 ］VGG16 RGB+T 0.727 GAFF［ 27 ］ResNet18 RGB+T 0.729 CFT［ 28 ］CSPDarkNet RGB+T 0.822 0.890 0.640 0.784 ProbEn［ 36 ］ResNet101 RGB+T 0.877 0.901 0.735 0.838 YOLOBase（ours） CSPDarkNet RGB+T 0.864 0.912 0.652 0.809 MAFFNet（ours） CSPDarkNet RGB+T 0.886 0.922 0.740 0.849
Table 5. Comparison of complexity of various models
View tableTable 5. Comparison of complexity of various models
Data Method Detector Parameter quantity /106 GFLOPs FPS /（frame·s-1） mAP50 RGB YOLOv5 YOLOv5 46.64 114.6 38 0.590 Thermal YOLOv5 YOLOv5 46.64 114.6 38 0.739 RGB+T CFT［ 28 ］YOLOv5 206.26 13732.5 14 0.784 RGB+T ProbEn［ 36 ］Faster R-CNN 107.18 339.3 17 0.838 RGB+T YOLOBase（ours） YOLOv5 76.47 195.8 25 0.809 RGB+T MAFFNet（ours） YOLOv5 73.72 190.8 23 0.849
Table 6. Performance comparison of different algorithms in LLVIP dataset
View tableTable 6. Performance comparison of different algorithms in LLVIP dataset
Method Backbone Data mAP50 mAP75 mAP YOLOv3［ 3 ］DarkNet RGB 0.859 0.379 0.433 YOLOv3［ 3 ］DarkNet Thermal 0.897 0.534 0.528 YOLOv5 CSPDarkNet RGB 0.908 0.519 0.505 YOLOv5 CSPDarkNet Thermal 0.946 0.722 0.619 CFT［ 28 ］CSPDarkNet RGB+T 0.975 0.729 0.636 CCIFNet［ 37 ］ResNet50 RGB+T 0.976 0.726 0.641 MAFFNet CSPDarkNet RGB+T 0.977 0.783 0.671
Tools
Get Citation
Copy Citation Text
Xiaoqiang Gao, Kan Chang, Mingyang Ling, Mengyu Yin. Object Detection via Multimodal Adaptive Feature Fusion[J]. Laser & Optoelectronics Progress, 2023, 60(24): 2410003
Paper Information
Category: Image Processing
Received: Mar. 13, 2023
Accepted: Apr. 12, 2023
Published Online: Dec. 4, 2023
The Author Email: Chang Kan (pandack0619@163.com)
© Copyright 2018-2021 | Chinese Laser Press.
All Rights Reserved 沪ICP备15018463号-20