Acta Photonica Sinica, Volume. 54, Issue 6, 0610001(2025)
Lightweight Pedestrian Vehicle Detection Algorithm Based on Visible and Infrared Bimodal Fusion
Figures & Tables(14)
Fig. 1. Overall architecture of lightweight bimodal target detection based on differential modal feature fusion module
Fig. 4. FLIR ADAS detection results（red inverted triangles indicate missed detections and yellow inverted triangles indicate false detections）
Fig. 5. LLVIP detection results（red inverted triangles indicate missed detections and yellow inverted triangles indicate false detections）
Fig. 6. KAIST detection results（red inverted triangles indicate missed detections and yellow inverted triangles indicate false detections）
Fig. 7. Daytime detection results of different scenes for each model in FLIR ADAS dataset（red inverted triangles indicate missed detections and yellow inverted triangles indicate false detections）
Fig. 8. Detection results of different scene nights for each model of FLIR ADAS dataset（red inverted triangles indicate missed detections and yellow inverted triangles indicate false detections）
Fig. 9. Comparison results between fogged and noisy images（red inverted triangles indicate missed detections and yellow inverted triangles indicate false detections）
Table 1. FLIR ADAS dataset ablation experiments result
View tableView in ArticleTable 1. FLIR ADAS dataset ablation experiments result
Mobilenetv2 DMFF IA mAP50/% mAP/% √ -- -- 81.6 40.7 √ √ -- 84.2 43.9 √ -- √ 82.9 43.1 √ √ √ 85.4 45.2
Table 2. Performance comparison of different models in different datasets
View tableView in ArticleTable 2. Performance comparison of different models in different datasets
Dataset Modality Method Params/×106 mAP50/% mAP/% FLIR ADAS RGB YOLOv7-tiny 6.02 70.1 30.6 Thermal YOLOv7-tiny 6.02 80.5 39.6 RGB+T Baseline 9.77 81.6 40.7 RGB+T Ours 8.46 85.4 45.2 LLVIP RGB YOLOv7-tiny 6.02 90 44.9 Thermal YOLOv7-tiny 6.02 95.8 62.3 RGB+T Baseline 9.77 95.6 57.1 RGB+T Ours 8.46 96.9 64.1 KAIST RGB YOLOv7-tiny 6.02 49.3 19.9 Thermal YOLOv7-tiny 6.02 68.4 30.4 RGB+T Baseline 9.77 72.8 32.0 RGB+T Ours 8.46 76.2 34.4
Table 3. Comparison of results of different models on FLIR ADAS dataset
View tableView in ArticleTable 3. Comparison of results of different models on FLIR ADAS dataset
Method Data Params/×106 mAP50/% mAP/% FPS Mono-modality networks YOLOv5s RGB 7.2 73.8 34.2 147 YOLOv5s Thermal 7.2 82.1 42.7 -- YOLOv7-tiny RGB 6.02 70.1 30.6 163.93 YOLOv7-tiny Thermal 6.02 80.5 39.6 -- Multi-modality networks ICAFusion RGB+T 120.21 81.6 38.9 92.59 CFT RGB+T 206.03 83.5 39.6 111.1 SLBAF RGB+T 2.49 79.3 39.7 53.8 Baseline RGB+T 9.77 81.6 40.7 -- Ours RGB+T 8.46 85.4 45.2 208.3
Table 4. Comparison of results of different models on the LLVIP dataset
View tableView in ArticleTable 4. Comparison of results of different models on the LLVIP dataset
Method Data mAP50/% mAP/% FPS Mono-modality networks YOLOv5s RGB 91.5 48.3 113.6 YOLOv5s Thermal 94.9 55.1 -- YOLOv7-tiny RGB 90 44.9 105.3 YOLOv7-tiny Thermal 95.8 62.3 -- Multi-modality networks ICAFusion RGB+T 87.3 53.4 64.9 PCMFNet［ 39 ］RGB+T 91.2 55.5 -- CFT RGB+T 97.5 63.6 83.3 SLBAF RGB+T 95.4 58.8 34.5 Baseline RGB+T 95.6 57.1 -- Ours RGB+T 96.9 64.1 103.3
Table 5. Comparison of results of different models on the KAIST dataset
View tableView in ArticleTable 5. Comparison of results of different models on the KAIST dataset
Method Data mAP50/% FPS YOLOv5s RGB 57.3 103 YOLOv5s Thermal 70.2 -- YOLOv7-tiny RGB 49.3 117.6 YOLOv7-tiny Thermal 68.4 -- SLBAF RGB+T 71.4 35.7 ICAFusion RGB+T 71.9 45.45 CFT RGB+T 72.0 71.4 AR-CNN RGB+T 75.3 -- MBNet RGB+T 75.9 -- Baseline RGB+T 72.8 -- Ours RGB+T 76.2 113.6
Tools
Get Citation
Copy Citation Text
Cuixia GUO, Yongtao XU, Zhanghuang ZOU, Zhijie PAN, Feng HUANG. Lightweight Pedestrian Vehicle Detection Algorithm Based on Visible and Infrared Bimodal Fusion[J]. Acta Photonica Sinica, 2025, 54(6): 0610001
Paper Information
Category:
Received: Nov. 26, 2024
Accepted: Jan. 20, 2025
Published Online: Jul. 14, 2025
The Author Email: Feng HUANG (huangf@fzu.edu.cn)
© Copyright 2018-2021 | Chinese Laser Press.
All Rights Reserved 沪ICP备15018463号-20